Functional Testing, BDD and domain-specific languages

I love Test Driven Development (TDD). If you look back through the posts on this blog that soon becomes apparent. I’m pretty comfortable with using TDD techniques at all levels of a solution, from the tiniest code snippet to multiply-redundant collaborating systems. Of course, the difficulty of actually coding the tests in a test-driven design can vary widely based on the context and complexity of the interaction being tested, and everyone has a threshold at which they decide that the effort is not worth it.

The big difference between testing small code snippets and large systems is the combinatorial growth of test cases. To fully test at a higher (or perhaps more appropriately, outer) layer would require an effectively infinite number of tests. By a happy co-incidence, though, if you have good test coverage at lower/inner layers of the code, you can rely on them to do their job, so it’s only needed to test the additional behaviour at each layer. Even with this simplifying assumption the problem does not fully go away. Very often the nature of outer layers implies a broader range of inputs and outputs, and a greater dependence on internal state from previous actions. It can still seem to need a ridiculous amount of tests to properly cover a complex application. And worst of all, these tests are often boring. Slogging through acres of largely similar sequences, differing only in small details, can be a real turn off for developers, so once again, it can feel that the effort is not worth it.

If fully covering the outer layers of a system, with tests for every combination of data, every sequence of actions, and every broken configuration is prohibitively expensive, time-consuming and downright boring, then it makes sense to be smart about it, and prioritise those tests which have the highest value. Value in this sense is a loose term encompassing cost of failures, importance of features, and scale of use. A scenario with a high business value if successful, a high cost of failure, which is used often by large numbers of people would be an obvious choice for an outer layer test. Something of little value which nobody cares about and which is hardly ever used would be much further down the list.

And this leads neatly to the concept of automated “functional testing”. Functional tests being outer layer tests which exercise these important, valuable interactions with the system as a whole, Arguably there is a qualitative difference between unit tests of internal components and functional tests of outer layers. Internal components have an internal function, and relate mostly with other internal components and their interactions are designed by developers for code purposes. This makes it relatively easy for developers to decide what to test, so specifying tests in similar language to the implementation code is a straightforward and effective way to get the tests written. Outer layers and whole applications have an external function, and business drivers for what they do and how they do it. This can be less amenable to describing tests in the language used by programmers. Add to this the potentially tedious nature of these external, functional, tests and its easy to see why they sometimes get overlooked, despite their potentially high business value.

Many attempts have been made over the years to come up with ways to get users and business experts involved in writing and maintaining external functional test cases. My friend, some-time colleague and agile expert Steve Cresswell has recently blogged about a comparison of Behaviour-Driven Design (BDD) tools. It’s an interesting article, but I can’t help thinking that there is another important dimension to these tools which also needs to be unpicked.

Along this dimension, testing tools range from “raw program code” (with no assistance from a framework); through “library-style” frameworks which just add some extra features to a programming language using its existing extension mechanisms; “internal DSL” (Domain Specific Language) frameworks which use language meta-programming features to re-work the language into something more suitable for expressing business test cases; “external languages” which are parsed (and typically interpreted, but compilation is also an option) by some specialist software; through to “non-textual” tools where the test specification is stored and managed in another way, for example by recording activity, or entering details in a spreadsheet.

In my experience, most “TDD” tools (JUnit and the like) sit comfortably in the “library-style” group. test case specification is done in a general-purpose programming language, with additions to help with common test activities such as assertions and collecting test statistics. Likewise, most “BDD” tools (Cucumber and the like) are largely in the “external language” group. This is slightly complicated by the need to drop “down” to a general-purpose language for the details of particular actions and assertions.

Aside from the “raw program code” option which, by definition, has no frameworks, the great bulk of test tools occupy the “library style” and “external language” groups, with a small but significant number of tools in the “non-textual” group. I find it somewhat surprising how few there seem to be in the “internal DSL” category, especially given how popular internal DSL approaches are for other domains such as web applications. There are some, of course, (coulda, for example) which claim to be internal DSLs for BDD-style testing, but there is still a more subtle issue with these and with many external languages.

The biggest problem I have with most BDD test frameworks is related to the concept of “comments” in programming languages. Once upon a time, when I was new to programming, it was widely assumed that adding comments to code was very important. I recall several courses which stressed this so much that a significant point of the mark was dependent on comments. Student (and by implication, junior developer) code would sometimes contain more comments than executable code, just to be on the safe side. Over the years it seems that the prevailing wisdom has changed. Sure, there are still books and courses which emphasise comments, but many experienced developers try to avoid the need for comments wherever possible, using techniques such as extracting blocks of code to named methods and grouping free-floating variables into semantically meaningful data structures, as well as brute-force approaches such as just deleting comments.

The move away from comments has developed gradually, as more and more programmers have found themselves working on code produced by the above mentioned comment-happy processes. The more you do this, the more you realise that the natural churn and change of code has an unpleasant effect on comments. By their nature comments are not part of the executable code (I am specifically excluding “parsed comments” which function like hints to a compiler or interpreter here). This in turn means that comments can not be tested, and thus cannot automatically be protected against errors and regressions. Add to this the common pressure to get the code working as quickly and efficiently as possible, you can see that comments often go unchanged even when the code being commented is radically different. This effect then snowballs – the more that comments get out of step with the code, the less value they provide, and so the less effort is made to update them. Soon enough most (or all!) comments are at best a useless waste of space, and at worst dangerously misleading.

What does this have to do with BDD languages? If you look in detail at examples of statements in many BDD languages, they have a general form something like keyword "some literal string or regular expression" Typical keywords are things like “Given”, “When”, “Then”, “And”, and so on. For example: Given a payment server with support for Visa and Mastercard. This seems lovely and friendly, phased in business terms. But let’s dig into how this is commonly implemented. Very likely, somewhere in the code will be a “step” definition.

An example from spinach, which is written in Ruby, might be:

  step 'a payment server with support for Visa and Mastercard' do
    @server = deploy_server(:payment, 8099)
    @server.add_module(@@modules[:visa])
    @server.add_module(@@modules[:mc])
    @server.start
  end

This also looks lovely, linking the business terminology to concrete program actions in a nice simple manner. However, this apparent simplicity hides the fact that the textual step name has no actual relationship to the concrete code. Sure, the text is used in at least two places, so it seems as if the system is on the case to prevent typos and accidental mis-edits, but it still says nothing about what the step code actually does. As an extreme example, suppose we changed the step code to be:

  step 'a payment server with support for Visa and Mastercard' do
    Dir.foreach('/') {|f| File.delete(f) if f != '.' && f != '..'}
  end

The test specifications in business language would still be sensible, but running the tests would potentially delete a whole heap of files!

For a less extreme example, imagine that the system has grown a broad BDD test suite, with many cases which use “Given a payment server with support for Visa and Mastercard”. Now we need to add support for PayPal. There are several options including:

  • copy the existing step to a new one with a different name and an extra line to install a PayPal module (including going through all the test code to decide which tests should use the old step and which should use the new one
  • add the extra line to the existing step and modify the name to include PayPal then change all the references to the new name
  • or just add the extra module and leave the step name the same.

The last option happens more often than you might think. Just as with the comments, the BDD step name is not part of the test code, and is not itself tested, so there is nothing in the system to keep it in step with the implementation. And the further this goes, the less the business language of the test specifications can be trusted.

I have worked on a few projects which used these BDD approaches, and all of them fell foul of this problem to some degree. Once this “rot” takes hold, it seems almost inevitable that the BDD specifications either become just another set of “unit” tests, only understood by developers who can dig into the steps to work out what is really happening, or they are progressively abandoned. It seems unlikely that many projects would be willing to take on the costs of going through a large test suite and checking a bunch arbitrary text that’s not part of the deliverable, just to see if it makes sense.

I wonder if the main reason that this is not seen to be more of a problem is that so few projects have progressed with BDD through several iterations of the system, and are still using the approach.

So, is there any way out of this trap? For some cases, using regular expressions for step name matching can help. This can provide a form of parameter-passing to steps, increasing reuse for multiple purposes and reducing churn in the test specifications. This does not solve the overall problem, though, as it still has chunks of un-parsed literal text. For that we would need a more sweeping change.

Which brings me back to internal and external DSLs. To my mind, the only way to address this issue in the longer term is to define the test specifications in a much more flexible language, one which suits both the domain being tested and the domain of testing itself. My aim would be to avoid the need for those comment-like chunks of un-parsed literal text, and align the programming language with the business language well enough that expressions in the business language make sense as programming constructs. If done well, this should allow test specifications to be changed at a business level and still be a valid and correct expression of the desired actions and assertions.

Although some modern programming languages are relatively adept in metaprogramming and internal DSLs (Ruby is well-known for this), mostly they don’t go far enough. Arguably a better choice would be to base the DSL on one of the languages which have very little by way of their own syntax, and thus are more flexible in adapting to a domain. Languages such as LISP and FORTH have hardly any syntax, and have both been used for describing complex domains. It used to be said of FORTH that the task of programming in forth was mainly one of writing a language in which the solution is trivial to express.

I’m afraid I don’t have a final answer to this, though. I have no tool, language or framework to sell, just a hope that somebody, somewhere, is working on this kind of next-generation BDD approach.

For more information about low-syntax languages, you might want to read some of my articles on Raspberry Alpha Omega, such as What is a High Level Language and More Language Thoughts and Delimter-Free Languages.

Also, for Paul Marrington’s take on internal vs external DSLs for testing, see BDD – Use a DSL or natural language?.

Spike Driven Development

test-Driven development (TDD) is a very powerful and beneficial approach to developing software. I routinely recommend it as a solution to untested and hard-to maintain software. However, every rule has its exceptions and there are occasions when a pure TDD approach is not very helpful.

Mark Needham wrote a recent blog post on the subject where he describes a kind of “Spike Driven Development”: Coding: Spike Driven Development at Mark Needham.

I tend to think of this as some sort of experiment-driven development. I start by choosing a goal. Typically this is either finding out about some murky or unknown aspect of a system such as interfacing with a third party, or improving a variable quantity such as performance. This in turn leads to thinking about ways to assess whether we have done enough to meet the goal and how much time to allocate to the exercise. Finally, a period of playing, following where each trial leads. The important bit is to repeatedly check progress against the goal – that’s what links it to the aspects of TDD which work well.

TDD: Driving from the assertion up

I’m doing quite a lot of agile and TDD mentoring at the moment, so such things are always floating near the top of my mind.

A recent article from Mark Needham presents an interesting view on how to construct code within tests, and the impact that might have on how tests drive design of code.

A good aspect of the Oracle/Sun purchase

There has been a lot of speculation and worry about Oracle’s acquisition of Sun, so it’s nice to see some positive aspects. Here’s one about an improved attitude to automated testing.

One of the results of the Oracle purchase of Sun has been an increased focus on testing — not that we didn’t test GlassFish before, but it was mostly manual in my area of the server.

from Jason Lee’s blog Coming Up for Air.

What Developers Need to Know About Agile

Stephan Schmidt writes

Agile is mostly driven driven by managment and consultants, seldom bottom up by developers.

I found this very odd, and completely contrary to my own experience. From the perspective of someone who first encountered agile approaches in the form of Extreme Programming the problem is almost always how to explain the obvious developer benefits in ways which management will understand.

This phrasing of agile for managers seems to be the purpose of Scrum. I tend to recommend that puzzled managers read up on Scrum, but puzzled developers start with XP and Test Driven Development, which provide some much more tangible developer advantages.

via Code Monkeyism: What Developers Need to Know About Agile.

What have we learned from 10 Years of TDD?

Wow, is Test-Driven Developement (TDD) really 10 tears old? Gojko Adzic is blogging the QCon London conference, and has a summary of the TDD lessons-learned session.

Gojko Adzic » QCon London 2009: What have we learned from 10 Years of TDD?.

Steve Freeman compares Test-Driven Development to rock climbing

I love this kind of analogy. Thinking and learning about one area by considering another is a very powerful technique for boosting overall understanding and wisdom. In this case, Steve Freeman has scraped off the climbing terminology from a snippet of an article and replaced it with jargon from software development.

Thought-provoking.

Tips on Test-Driven Development | Steve Freeman

Fun with very strict TDD

I use Test-Driven Development (TDD) every day, and find it very helpful. It can be hard to get to grips with, though. I was pleased to read that acceptance-testing pundit Gojko Adzic had fun with some very strict TDD rules.

Gojko Adzic » Thought-provoking TDD exercise at the Software Craftsmanship conference.

My approach to TDD is really quite similar to these strict rules. I’ll admit that I do (sometmes) cut a few corners. The biggest aid to getting the most from TDD is definately good refactoring tools. Without that the whole thing becomes so much more clumsy.

TDD is very comfortable with Java and Eclipse. Recently I have moved out of that comfort zone into some fairly deep JavaScript work, and really miss the ability to automatically extract or move methods etc.

Testing is Overrated

A conference presentation from RubyFringe, designed to be contentious. There are some good points, particularly about the way that different approaches to testing can complement each other, but I think he misses the point about TDD when he lumps it in with developer unit testing and ignores the design aspects of the technique.

InfoQ: Testing is Overrated.

Test-Driven Development of HTTP and REST interfaces with Mojasef

Test-Driven Development (TDD) of HTTP and REST interfaces can be tricky. This post contains some examples of how to do it using the open source Mojasef toolkit. I was prompted to write this post after recently finding and fixing an irritating bug using just this technique.

First, let’s set the scene. In TDD the rules are very simple: No production code is written without a failing test; just enough code is written to satisfy the current set of tests; duplication is mercilessly refactored. TDD is a very powerful and effective technique which can improve both the quality and delivery time of software. HTTP and REST interfaces provide access to a system or component using the HTTP protocol. This access may be by fetching HTML or WML pages, for example, or it may be by sending, receiving and updating resources in some other format such as XML or JSON.

For a first scenario imagine we have an existing HTTP interface for which we want to write a client using TDD. This service is relatively simple to start with. To send a message requires a POST request to a particular URL with two parameters “destination” and “text”. If the destination is valid, the text message is sent to the destination and a “200″ status code is returned. If the destination is not recognized, a “404″ status code is returned. This interface is so simple that it is tempting to just write the client, test it manually and move on. But if we were to do that, then we might not think of all the test case possibilities, and there would be no regression tests to help protect the software from unexpected side-effects later. So we opt to do the right thing, and use TDD.

For the sake of simplicity, I will assume we are using the Stringtree HTTPClient as described in an earlier post. So let’s start with a simple test to make sure the code compiles and the test libraries and stuff are in place:

import org.stringtree.http.*;
 
public class MessageSendingTest extends junit.framework.TestCase {
  public void testExceptionFromUnknownURL() {
    Form form = new Form();
    form.put("destination", "name@example.com");
    form.put("text", "hello from HTTP");
    HTTPClient client = new HTTPClient();
    try {
      client.post("http://localhost:9999/send", form);
      fail("should throw an exception");
    } catch (Exception e) {}
  }
}

This test serves two purposes. First, compiling it helps drive the structure of the test case, what classes need to be imported and so on. Second it documents the expected behaviour of our client if the server is not running or not where it is expected to be. I always like to start with this sort of “null” test case wherever possible. Once we get this test working we are ready to move on to the next step, testing against an actual server.

It might be possible to test against a real server, perhaps by manually examining log files, or checking the actual receipt of a message. Feel free to do that, but understand that that is really integration testing rather than unit testing, and neither helps us design and debug the client, nor document and enforce the proper behaviour of our code once we move on to other work. To gain these benefits we need a server which may be started, examined and stopped quickly and under the control of a simple unit test. This is where Mojasef comes in.

Mojasef is a server and web application framework designed for efficient test-driven development of web applications. Although usually used for building applications, it is just as suitable for building test harnesses. In this case we will build a simple application which implements the HTTP interface described above. And yes, we will do it using TDD, so we leave the MessageSendingTest for the moment and start a new one to drive our test harness without requiring HTTP. As usual, I start with a “null” test to get started:

public class MessageHarnessTest extends junit.framework.TestCase {
  MessageHarness harness;
 
  public void setUp() {
    harness = new MessageHarness();
  }
 
  public void testNoAction() {
    assertFalse(harness.wasCalled());
  }
}

This won’t compile without a MessageHarness class, so let’s make the minimum necessary.

public class MessageHarness {
  public boolean wasCalled() {
    return false;
  }
}

Good, our test now runs, so let’s add some behaviour. First another test:

  public void testCalledWithoutArguments() {
    harness.send();
    assertTrue(harness.wasCalled());
  }

To make this compile we need to add another method to MessageHarness

  public void send() {
  }

Our new test fails, so we need to make it work.

public class MessageHarness {
  private boolean called = false;
 
  public boolean wasCalled() {
    return called;
  }
 
  public void send() {
    called = true;
  }
}

That seems enough to start with. Now back to the main test case. Add a new test to call the application we have just built over HTTP and check that all the communication stuff works. With a bit of refactoring to keep the tests clean and duplication low.

import org.stringtree.http.*;
import org.stringtree.mojasef.standalone.InlineServer;
 
public class MessageSendingTest extends junit.framework.TestCase {
  HTTPClient client;
  Form form;
  InlineServer server;
  MessageHarness app;
 
  public void setUp() throws Exception {
    client = new HTTPClient();
    form = new Form();
    app = new MessageHarness();
    server = new InlineServer(app, "9998");
    server.start();
  }
 
  public void tearDown() {
	server.halt();
  }
 
  public void testExceptionFromUnknownURL() {
    form.put("destination", "name@example.com");
    form.put("text", "hello from HTTP");
    try {
      client.post("http://localhost:9999/send", form);
      fail("should throw an exception");
    } catch (Exception e) {}
  }
 
  public void testHarnessWiring() throws Exception {
    form.put("destination", "unknown@example.com");
    form.put("text", "hello from HTTP");
    assertFalse(app.wasCalled());
    client.post("http://localhost:9998/send", form);
    assertTrue(app.wasCalled());
  }
}

This probably needs a bit of explanation. The MessageHarness class we produced above is a Mojasef web application. Really, without any other configuration, base classes or nonsense. It’s just a “plain old Java object” (POJO). The job of the Mojasef code is to take an object of that class and make its methods available as HTTP URLs.

To serve it over HTTP we could use a MojasefServlet and run it in any Servlet Container, but for this test we want an ultra-light server which can be started and stopped in a test. So we use the InlineServer provided in the Mojasef jar. It’s a real HTTP server which you can use from a regular web browser, but it’s mainly designed for streamlined use in tests.

This server is configured to serve the MessageHarness application on port 9998. Having configured the server we start it in startUp and stop it in tearDown to ensure that nothing is left behind between tests.

The new test makes a HTTP request to the freshly started server, then checks (by calling the MessageHarness method we created earlier) that the test harness application was successfully called. This verifies that the server has been correctly started and called.

It’s still not really testing the message API. To do that we need more tests.

Now, we know from the introduction above that our resource should return a HTTP 404 (“not found”) error if we try to send to an unrecognised address.

  public void test404FromUnknownDestination() throws Exception {
    form.put("destination", "unknown@example.com");
    form.put("text", "hello from HTTP");
    Document result = client.post("http://localhost:9998/send", form);
    assertEquals("404", result.getHeader(HTTPClient.HTTP_RESPONSE_CODE));
  }

Making this test work is pretty simple, especially as we have no other tests which say anything about the HTTP Response code. The only “cleverness” is that we now need to use some slightly smarter features of the Mojasef code.

import org.stringtree.finder.StringKeeper;
import org.stringtree.mojasef.HTTPConstants;
 
public class MessageHarness {
  private boolean called = false;
 
  public boolean wasCalled() {
    return called;
  }
 
  public void send(StringKeeper context) {
    called = true;
    context.put(HTTPConstants.RESPONSE_CODE, "404");
  }
}

Mojasef will try a variety of ways to call a public method. We have seen the basic one – a method with no parameters mapped directly to a URL. This is another one – a method taking a “context” as an argument. This context will be pre-loaded with any POST or URL parameters, as well as any request headers and plenty of other stuff which is not of interest right now. This context also provides a way of setting information to be sent back to the client.

Now, our application code is passing all its tests, but it does so by returning a “404″ error code for all requests. This is not really desirable behaviour, so we need another test:

  public void test200FromKnownDestination() throws Exception {
    form.put("destination", "known@example.com");
    form.put("text", "hello from HTTP");
    Document result = client.post("http://localhost:9998/send", form);
    assertEquals("200", result.getHeader(HTTPClient.HTTP_RESPONSE_CODE));
  }

This test fails, so we need to modify the application to add some understanding of the supplied form data.

import org.stringtree.finder.StringKeeper;
import org.stringtree.mojasef.HTTPConstants;
 
public class MessageHarness {
  private boolean called = false;
 
  public boolean wasCalled() {
    return called;
  }
 
  public void send(StringKeeper context) {
    called = true;
    String destination = context.get("destination");
    if ("known@example.com".equals(destination)) {
        context.put(HTTPConstants.RESPONSE_CODE, "200");
    } else {
        context.put(HTTPConstants.RESPONSE_CODE, "404");
    }
  }
}

Now we reach an interesting point in the development process. It is perfectly feasible to continue in this vein, step-by-step bringing the application into existence. However, even the streamlined InlineServer still needs to set itself up, bind and unbind ports, deal with network traffic and so on. This is considerably slower than a regular method call, and as more tests are added the test case will continue to slow down.

The advantages of the Mojasef approach to web applications really begin to pay off at this point. The tests we already have prove the basics of HTTP Communication are working, so further tests do not really need to bother with HTTP, provided the tests are testing exactly the same code. Luckily (as pointed out earlier) a Mojasef application is just a regular Java class, which needs no modification to be tested using regular JUnit tests. Here are the same tests as above, but implemented as simple method calls:

import org.stringtree.finder.StringKeeper;
import org.stringtree.mojasef.HTTPConstants;
import org.stringtree.mojasef.HTTPConstants;
 
public class MessageProcessingTest extends junit.framework.TestCase {
  StringKeeper context;
  MessageHarness app;
 
  public void setUp() {
    context = new MapStringKeeper();
    app = new MessageHarness();
  }
 
  public void test404FromUnknownDestination() throws Exception {
    context.put("destination", "unknown@example.com");
    context.put("text", "hello from HTTP");
    app.send(context);
    assertEquals("404", context.get(HTTPConstants.RESPONSE_CODE));
  }
 
  public void test200FromKnownDestination() throws Exception {
    context.put("destination", "known@example.com");
    context.put("text", "hello from HTTP");
    app.send(context);
    assertEquals("200", context.get(HTTPConstants.RESPONSE_CODE));
  }

I shall stop there, leaving the development of the rest of the application to interested readers.

From the above steps, I hope you can see that the normal process of Test-Driven Development is quite possible for both web applications, and web-application clients. When using a framework which gets all the fiddly stuff out of the way it can even be as simple, quick and productive as in-process development.

TDD: Does it make you slower?

I use Test-driven Development (TDD) all the time. Although my immediate work colleagues have “bought in” to the idea, I sometimes find my self explaining about it to other people in other situations. A common question is whether the extra effort of writing all those tests slows down the overall development process. Mark Needham has an article on this very question.

TDD: Does it make you slower? at Mark Needham

How does Architecture fit with TDD

It’s a common conundrum once a team embraces test-driven development (TDD). What becomes of the role of architect? Some suggestions include moving the role of the architect even higher than in a typical waterfall or “over the wall” process.

Sarah Taraporewalla’s Technical Ramblings » How does Architecture fit with TDD

My take on this is a bit different. One of the toughest things when developing software using TDD is choosing which tests to write next. This drives in a very real way the capabilities and design of the resulting system. At a low level the choice of test is often relatively easy, but at a higher level it can be easy to get lost in the detail.

So here’s my suggestion for the role of the architect in TDD-based development: driving the choice of high-level test cases to implement, and moderating the discussions between developers about how such test cases are split into finer tests and eventually implemented.

The benefits of TDD (evolutionary design, scaffolding, etc.) are still there, as is the input from an individual with collected understanding of which possibilities are implemented and which are not.

Thoughts?

TDD and Test-Driven Refactoring

This is a neat observation. One equivalent to test-Driven Development (TDD) when working with an existing codebase might be “test-driven refactoring”. The idea seems to be that at any particular point in the lifecycle of a software system not all of the benefits of TDD are actually being used. So why not just do enough TDD to get the required benefits at the time.

In this case the need is to refactor some existing code in preparation for a change. Rather than either (a) stop to write tests for the whole system, or (b) throw up our hands and abandon TDD for this work, the suggestion is to write a passing test around the exact code being refactored, then use that test to ensure that the change preserved that behaviour. Then, in the TDD style, rinse and repeat for the following refactorings.

As this progresses the result should be a growing body of regression tests. concentrating around the areas which have actually changed, which seems good to me. It’s not as good as TDD from the start, but it’s better than no tests at all.

Agile Tips: Testing private methods, TDD and Test-Driven Refactoring

Testing Will Challenge Your Conventions

I’d not seen this, and if you haven’t either then it’s well worth a read. Tim Ottinger gives a detailed list of the ways that Test-Driven Development (TDD) changes code.

Testing Will Challenge Your Conventions

Increasing Code Coverage May Be Harmful

I develop almost all my code these days using Test Driven Development (TDD). By taking this approach I never bother about unit test code coverage, but some developers seem very concerned by it.

One thing that I have never considered doing is retroactively adding tests just to increase some code-coverage metric. Dan Manges has written in interesting explanation of why that might be, using some examples in Ruby.

Dan Manges’s Blog – Increasing Code Coverage May Be Harmful

The world mocks too much

I’ll put my cards on the table. At heart I am a “classic TDD” guy. My first attempt at any new code is to test return values or side-effects rather than immediately jump to setting up mock objects and testing interactions. Although I do use mock objects as a testing technique from time to time I have never liked any of the popular mock frameworks. The nearest I have found to what I would like is probably Mockito, but even so I still use my own recording framework.

thekua.com@work » The world mocks too much

TDD on Three Index Cards

This is a great little summary. I think I’ll have to make a set of these cards to show and remind at any opportunity.

TDD on Three Index Cards

Husbanding willpower

Steve Freeman has put together one of the most thought-provoking software development articles I have read in a long time – based on application of ideas from a New Scientist article.

Husbanding willpower | Steve Freeman

Small and Simple Web Applications – the Friki Way

A few years go I wrote a series of articles for The Java Ranch Journal entitled “Small and Simple Web Applications – the Friki Way”. The series showed an example of developing a very simple wiki (collaborative editable web site) in Java using Test-Driven Development (TDD) and a form of agile project management.

From time to time I refer someone to these articles but it has been tedious to find and gather all the individual article links. This post contains those links, so I can just give out a single URL in future.

Unfortunately I never got around to continuing the series beyond that point. Maybe one day I should …

mockito 1.5 and spying on objects

Mockito is a Java Mock objects framework which seems more usable than either of the main contenders (JMock and EasyMock). The project has just released a new version which now allows the attachment of a mock “overlay” to an existing object.

This has been on my “evaluate” stack for months, but (as Harry Pynn pointed out when he “vouched for” me) I tend to be more comfortable with “classic” TDD and don’t usually find much use for Mock Objects.

I have had a quick play with the examples of use on the Mockito site with mixed success. many obvious cases work well, but there are also some gaps in the way it works. For example, consider this small example test:

import static org.mockito.Mockito.spy;
 
public class PlaypenTest extends TestCase {
	public void testMockito() {
		List real = new ArrayList(); 
		real.add("first");
		real.add("second");
		List ml = spy(real);
 
		stub(ml.get(0)).toReturn("huh");
		assertEquals("huh", ml.get(0));
		assertEquals("second", ml.get(1));
	}
}

It works correctly. However, the following idiomatic and functionally similar simplification throws a complicated internal exception in the “spy” method.

import static org.mockito.Mockito.spy;
 
public class PlaypenTest extends TestCase {
	public void testMockito() {
		List real = Arrays.asList(new String[] {"first", "second"});
		List ml = spy(real);
 
		stub(ml.get(0)).toReturn("huh");
		assertEquals("huh", ml.get(0));
		assertEquals("second", ml.get(1));
	}
}

Maybe I’ll get around to using it for something. I’m still not sure.

mockito – Google Code