Frank Carver's Punch Barrel

Test-Driven Development (TDD) of HTTP and REST interfaces can be tricky. This post contains some examples of how to do it using the open source Mojasef toolkit. I was prompted to write this post after recently finding and fixing an irritating bug using just this technique.

First, let’s set the scene. In TDD the rules are very simple: No production code is written without a failing test; just enough code is written to satisfy the current set of tests; duplication is mercilessly refactored. TDD is a very powerful and effective technique which can improve both the quality and delivery time of software. HTTP and REST interfaces provide access to a system or component using the HTTP protocol. This access may be by fetching HTML or WML pages, for example, or it may be by sending, receiving and updating resources in some other format such as XML or JSON.

For a first scenario imagine we have an existing HTTP interface for which we want to write a client using TDD. This service is relatively simple to start with. To send a message requires a POST request to a particular URL with two parameters “destination” and “text”. If the destination is valid, the text message is sent to the destination and a “200″ status code is returned. If the destination is not recognized, a “404″ status code is returned. This interface is so simple that it is tempting to just write the client, test it manually and move on. But if we were to do that, then we might not think of all the test case possibilities, and there would be no regression tests to help protect the software from unexpected side-effects later. So we opt to do the right thing, and use TDD.

For the sake of simplicity, I will assume we are using the Stringtree HTTPClient as described in an earlier post. So let’s start with a simple test to make sure the code compiles and the test libraries and stuff are in place:

import org.stringtree.http.*;
 
public class MessageSendingTest extends junit.framework.TestCase {
  public void testExceptionFromUnknownURL() {
    Form form = new Form();
    form.put("destination", "name@example.com");
    form.put("text", "hello from HTTP");
    HTTPClient client = new HTTPClient();
    try {
      client.post("http://localhost:9999/send", form);
      fail("should throw an exception");
    } catch (Exception e) {}
  }
}

This test serves two purposes. First, compiling it helps drive the structure of the test case, what classes need to be imported and so on. Second it documents the expected behaviour of our client if the server is not running or not where it is expected to be. I always like to start with this sort of “null” test case wherever possible. Once we get this test working we are ready to move on to the next step, testing against an actual server.

It might be possible to test against a real server, perhaps by manually examining log files, or checking the actual receipt of a message. Feel free to do that, but understand that that is really integration testing rather than unit testing, and neither helps us design and debug the client, nor document and enforce the proper behaviour of our code once we move on to other work. To gain these benefits we need a server which may be started, examined and stopped quickly and under the control of a simple unit test. This is where Mojasef comes in.

Mojasef is a server and web application framework designed for efficient test-driven development of web applications. Although usually used for building applications, it is just as suitable for building test harnesses. In this case we will build a simple application which implements the HTTP interface described above. And yes, we will do it using TDD, so we leave the MessageSendingTest for the moment and start a new one to drive our test harness without requiring HTTP. As usual, I start with a “null” test to get started:

public class MessageHarnessTest extends junit.framework.TestCase {
  MessageHarness harness;
 
  public void setUp() {
    harness = new MessageHarness();
  }
 
  public void testNoAction() {
    assertFalse(harness.wasCalled());
  }
}

This won’t compile without a MessageHarness class, so let’s make the minimum necessary.

public class MessageHarness {
  public boolean wasCalled() {
    return false;
  }
}

Good, our test now runs, so let’s add some behaviour. First another test:

  public void testCalledWithoutArguments() {
    harness.send();
    assertTrue(harness.wasCalled());
  }

To make this compile we need to add another method to MessageHarness

  public void send() {
  }

Our new test fails, so we need to make it work.

public class MessageHarness {
  private boolean called = false;
 
  public boolean wasCalled() {
    return called;
  }
 
  public void send() {
    called = true;
  }
}

That seems enough to start with. Now back to the main test case. Add a new test to call the application we have just built over HTTP and check that all the communication stuff works. With a bit of refactoring to keep the tests clean and duplication low.

import org.stringtree.http.*;
import org.stringtree.mojasef.standalone.InlineServer;
 
public class MessageSendingTest extends junit.framework.TestCase {
  HTTPClient client;
  Form form;
  InlineServer server;
  MessageHarness app;
 
  public void setUp() throws Exception {
    client = new HTTPClient();
    form = new Form();
    app = new MessageHarness();
    server = new InlineServer(app, "9998");
    server.start();
  }
 
  public void tearDown() {
	server.halt();
  }
 
  public void testExceptionFromUnknownURL() {
    form.put("destination", "name@example.com");
    form.put("text", "hello from HTTP");
    try {
      client.post("http://localhost:9999/send", form);
      fail("should throw an exception");
    } catch (Exception e) {}
  }
 
  public void testHarnessWiring() throws Exception {
    form.put("destination", "unknown@example.com");
    form.put("text", "hello from HTTP");
    assertFalse(app.wasCalled());
    client.post("http://localhost:9998/send", form);
    assertTrue(app.wasCalled());
  }
}

This probably needs a bit of explanation. The MessageHarness class we produced above is a Mojasef web application. Really, without any other configuration, base classes or nonsense. It’s just a “plain old Java object” (POJO). The job of the Mojasef code is to take an object of that class and make its methods available as HTTP URLs.

To serve it over HTTP we could use a MojasefServlet and run it in any Servlet Container, but for this test we want an ultra-light server which can be started and stopped in a test. So we use the InlineServer provided in the Mojasef jar. It’s a real HTTP server which you can use from a regular web browser, but it’s mainly designed for streamlined use in tests.

This server is configured to serve the MessageHarness application on port 9998. Having configured the server we start it in startUp and stop it in tearDown to ensure that nothing is left behind between tests.

The new test makes a HTTP request to the freshly started server, then checks (by calling the MessageHarness method we created earlier) that the test harness application was successfully called. This verifies that the server has been correctly started and called.

It’s still not really testing the message API. To do that we need more tests.

Now, we know from the introduction above that our resource should return a HTTP 404 (“not found”) error if we try to send to an unrecognised address.

  public void test404FromUnknownDestination() throws Exception {
    form.put("destination", "unknown@example.com");
    form.put("text", "hello from HTTP");
    Document result = client.post("http://localhost:9998/send", form);
    assertEquals("404", result.getHeader(HTTPClient.HTTP_RESPONSE_CODE));
  }

Making this test work is pretty simple, especially as we have no other tests which say anything about the HTTP Response code. The only “cleverness” is that we now need to use some slightly smarter features of the Mojasef code.

import org.stringtree.finder.StringKeeper;
import org.stringtree.mojasef.HTTPConstants;
 
public class MessageHarness {
  private boolean called = false;
 
  public boolean wasCalled() {
    return called;
  }
 
  public void send(StringKeeper context) {
    called = true;
    context.put(HTTPConstants.RESPONSE_CODE, "404");
  }
}

Mojasef will try a variety of ways to call a public method. We have seen the basic one – a method with no parameters mapped directly to a URL. This is another one – a method taking a “context” as an argument. This context will be pre-loaded with any POST or URL parameters, as well as any request headers and plenty of other stuff which is not of interest right now. This context also provides a way of setting information to be sent back to the client.

Now, our application code is passing all its tests, but it does so by returning a “404″ error code for all requests. This is not really desirable behaviour, so we need another test:

  public void test200FromKnownDestination() throws Exception {
    form.put("destination", "known@example.com");
    form.put("text", "hello from HTTP");
    Document result = client.post("http://localhost:9998/send", form);
    assertEquals("200", result.getHeader(HTTPClient.HTTP_RESPONSE_CODE));
  }

This test fails, so we need to modify the application to add some understanding of the supplied form data.

import org.stringtree.finder.StringKeeper;
import org.stringtree.mojasef.HTTPConstants;
 
public class MessageHarness {
  private boolean called = false;
 
  public boolean wasCalled() {
    return called;
  }
 
  public void send(StringKeeper context) {
    called = true;
    String destination = context.get("destination");
    if ("known@example.com".equals(destination)) {
        context.put(HTTPConstants.RESPONSE_CODE, "200");
    } else {
        context.put(HTTPConstants.RESPONSE_CODE, "404");
    }
  }
}

Now we reach an interesting point in the development process. It is perfectly feasible to continue in this vein, step-by-step bringing the application into existence. However, even the streamlined InlineServer still needs to set itself up, bind and unbind ports, deal with network traffic and so on. This is considerably slower than a regular method call, and as more tests are added the test case will continue to slow down.

The advantages of the Mojasef approach to web applications really begin to pay off at this point. The tests we already have prove the basics of HTTP Communication are working, so further tests do not really need to bother with HTTP, provided the tests are testing exactly the same code. Luckily (as pointed out earlier) a Mojasef application is just a regular Java class, which needs no modification to be tested using regular JUnit tests. Here are the same tests as above, but implemented as simple method calls:

import org.stringtree.finder.StringKeeper;
import org.stringtree.mojasef.HTTPConstants;
import org.stringtree.mojasef.HTTPConstants;
 
public class MessageProcessingTest extends junit.framework.TestCase {
  StringKeeper context;
  MessageHarness app;
 
  public void setUp() {
    context = new MapStringKeeper();
    app = new MessageHarness();
  }
 
  public void test404FromUnknownDestination() throws Exception {
    context.put("destination", "unknown@example.com");
    context.put("text", "hello from HTTP");
    app.send(context);
    assertEquals("404", context.get(HTTPConstants.RESPONSE_CODE));
  }
 
  public void test200FromKnownDestination() throws Exception {
    context.put("destination", "known@example.com");
    context.put("text", "hello from HTTP");
    app.send(context);
    assertEquals("200", context.get(HTTPConstants.RESPONSE_CODE));
  }

I shall stop there, leaving the development of the rest of the application to interested readers.

From the above steps, I hope you can see that the normal process of Test-Driven Development is quite possible for both web applications, and web-application clients. When using a framework which gets all the fiddly stuff out of the way it can even be as simple, quick and productive as in-process development.

From an SOA architectural view, Business Process Management and workflow orchestration seems an obvious component of a large system. However, when I see this idea moved wholesale into the world of REST, I worry that a significant point has been missed.

InfoQ: IBM’s BPM Zero Project: RESTful Worflow Management

The part of BPEL workflow orchestration which makes sense when used with traditional services (presented via SOAP or other RPC mechanisms) is the connecting together of displarate executable chunks into a larger executable chunk. In effect, BPEL used as an over-arching programming language.

The point that is missed when this approach is applied to REST is that REST is not about executable services, but about resources. A REST-style of over-arching connection would somehow represent disparate resources as a larger resource. This is not at all what BPEL orchestration does.

The practical upshot of this is that SOA workflow management can only work with REST resources which may also be considered as executable services. While such things do exist, they are an unusual and very rare subset of REST. Attempting to integrate REST resources using BPEL typically leads to one of three outcomes: (1) the integration is limited to those resources which are also services, and thus is so limited as to be hardly useful. (2) useful resources are converted or wrapped to appear as executable services, defeating the point of REST. (3) The project is abandoned as impossible, or simply fails.

As a simple “litmus test” for whether you are making effective and proper use of REST: Will your application still work if all the collaborating “services” are replaced by simple files or static web pages? If not, then you need to work harder at understanding REST.

I have read a lot of faux-REST APIs recently, which are essentially just HTTP/XML or HTTP/JSON remote services, and still need a client to be pre-built with specialist knowledge of URI structure. “Proper” REST allows a server to change its URIs however and whenever it likes, with client applications seamlessly adapting to the change.

InfoQ has published an excellent article describing an approach to building fully self-describing REST interfaces.

InfoQ: Describing RESTful Applications

I still think more work is needed, though, particularly in the area of content types. The suggestion of application/blah.whatever+xml as per RFC 3023 is certainly better than an unadorned text/xml, but still does not provide the flexibility for client applications to sensibly handle new, yet similar, content types. In particular that proposal is limited to XML, with no equivalent for JSON or other data description formats.

Alternative approaches such as text/xml/blah/whatever might allow a more general support. In this example, transport and storage responsibilities might only care that it is essentially a textual format (something which is worryingly implicit in application/blah.whatever+xml), while parsing components might only care that it is text/xml, and so on. There is then the possibility for further information to be encoded in the content type. Consider a type such text/xml/link@href/whatever, which might indicate a textual format, parsable as XML, in which relationship URIs are indicated as a link element with an href attribute, and so on.

While compelling, such an approach could easily lead to fragility, though. Such a representation implies that the range of available content types is a strict single-inheritance tree ontology, with all the familiar problems
that entails.

Definitely an area for further research.

When I first encountered Duncan Cragg’s “REST dialogues” I was not sure how they would develop. As I have read more, I have become progressively more impressed. Cragg uses the style of a Socratic dialogue with an imaginary “eBay architect” to teach about the nature and use of REST techniques as an alternative to more traditional approaches such as SOAP and SOA.

Recent dialogues on Business Conversations and The Distributed Observer Pattern are particularly thought-provoking, but the whole growing series is definitely worth a read.

The REST Dialogues

I have encountered several web articles recently which attempt to describe REST verbs (“get”, “put”, “post”, “delete”) in terms of traditional database CRUD (“create”, “read”, “update”, “delete”) operations, and then get stuck trying to map one set to the other.

On the one hand, it’s easy to see why the confusion happens. Both sets have four basic operations, and both sets are concerned with manipulation of stored resources, both have an operation called “delete”, and “read” sounds pretty much like “get”. However, that’s where the similarity stops.

For some recent examples of confusing REST and CRUD, see:

REST and usablity shouldn’t be mixed

Two stage resource oriented web application architectures

Hazem Ahmed Saleh: The rules behind designing a good RESTful service

Even the, otherwise very good, REST Dialogues sometimes fall into this trap.

So, what is the difference?

“Get” and “read” are essentially the same, at least when it comes to fetching raw resource data.

There is a difference in usage, though. In REST a “get” fetches a single resource. CRUD, on the other hand, is often associated with relational databases, which excel in returning sets of results. The common REST approach to multiple results is to return a resource which includes URIs for each resource in the collection (perhaps also with some summary information); the REST client is then free to individually “get” any or all referenced resources which it requires, or if necessary to instead fetch them from a local cache. As an example, consider the loading of images in a web page.

“Delete”, although similar on the surface has some important differences underneath.

In CRUD, the relational heritage often implies the ability to do complex deletes, such as deleting all resources which meet some criteria. In REST the “delete” operation is a simple delete of the resource at a single URI. To delete multiple items using REST typically requires multiple, separate, “delete” requests.

“Put” and “update” can be similar or different depending on interpretation.

The REST “put” operation has strict and well-defined semantics. It ensures that the specified URI has the supplied resource. If the URI is invalid it fails; if the URI contains no resource the supplied resource is placed there; if the URI contains a different resource it is replaced with the supplied resource; if the URI contains the same resource there is no apparent change to the resource at the URI. The CRUD “update” operation, on the other hand is less well-defined. SQL standards do not include a “replace”, “put”, or “insert or update if exists” operation (although many vendor implementations implement such features in a non-standard way). With this in mind, some people interpret the CRUD “update” operation as equivalent to an SQL “update” statement (which only applies if the resource already exists), while others treat it more generally as any form of change which is not a “delete”.

This only leaves “post” and “create”. These are the most different of all.

In REST, “post” is used for any action which results in a resource change or side-effect, and cannot be repeated without the change or side-effect happening again. An SQL “insert” operation would be implemented as a REST “post”, for example, but so would an SQL “update” which increments a value, or the sending of an email. A practical rule of thumb is that a “post” should be simple and quick, leaving the way for future changes using “put” as soon as possible. A CRUD “create” is a quite different thing. “Create” explicitly creates a new resource, and is undefined if that resource already exists.

REST has no direct analogue of “create”. If it is needed (for example) to create and populate a new resource, it is typical to call “post” on some parent resource which results in the return of, or redirect to, a new (empty) URI, and then populate the URI with one or more “put” operations. This approach has the big advantage that the impact of an accidental re-”post” is minimised. In a well designed system, the creating of a new URI could be as low-impact as incrementing an integer, only requiring storage of resource data if/when it is later populated using “put”.

CRUD, on the other hand, has no direct analogue of “post”. A CRUD “create” will probably have side effects, as will some CRUD “update” operations. CRUD is also limited in that it does not include the concept of side-effects outside the resources being managed. There is no way to describe sending an email in terms of CRUD, for example.

In summary, the REST and CRUD are different in many ways, some subtle and some less so.

REST is an approach designed to enable scalable distributed applications to clearly distinguish actions which make no change and thus may be cached or repeated (“get”) from those which make a change but may be casually repeated without problems (“put”, “delete”) from those which should neither be repeated nor cached (“post”). This in turn enables intermediate software (be it in the form of libraries, reusable components, or entirely separate systems) to implement sensible optimisation and error-handling strategies without needing knowledge of the content or relationships of the resources.

CRUD is a description of some generic operations on stored data, with no particular intention of serving as a distribution protocol, and thus little or no planning for the mitigation of problems such as accidental duplicates, time-outs or lost requests.

Attempts, however well-intentioned, to explain one by analogy with the other run a great risk of increasing confusion rather than clearing it. If you are considering using such an analogy, please think twice.

REST is a very powerful way of thinking about application design, but it can be very hard to really understand. Several people including the originator of REST, Roy Fielding, have written on the subject, and InfoQ has attempted to summarise some of these articles.

InfoQ: What Makes Good REST?

This looks interesting. Next Monday (20th October 2008) the London JAVAWUG will be holding a “Birds-of-a-Feather” session on the topic of Web applications and REST web services. Both are subjects near to my heart, so I might try and arrange a trip to “the smoke” that evening.

Anyone else likely to be attending?

JAVAWUG :: Java Web User Group :: London, UK :: Peter Pilgrim, Java Champion : Weblog

An intriguing and detailled example of implementing a real-world asynchronous workflow using REST. I’m not sure that I entirely agree with all the choices, but it’s well worth studying to see how capable REST can be.

InfoQ: How to GET a Cup of Coffee

Another interesting-looking Qcon presentation for the “to watch” queue. I’m getting a bit bored with the X eye for the Y guy title meme, though.

InfoQ: REST Eye for the SOA Guy

This is interesting, in that it is a backlash against REST, which has otherwise been one of the darlings of the software world. I’m not entirely sure if this article is an honest attempt at a refutal, or whether it is merely an attempt to stir up some discussion. But it’s worth reading anyway.

Could we be looking at it all wrong?: Putting REST to rest

A typical InfoQ summary of an interesting discussion, this time about the merits and problems of polling REST resources, and how some other approaches might help solve these problems.

InfoQ: Beyond Polling? Consider PubSub, Push and MOM

When I first saw this it looked great: a ruby REST wrapper which supports a lot of useful test and integration possibilities. However, the deeper I looked, the more disappointed I became. I’m now saddened to believe that this is based on yet another misunderstanding of what REST is.

As far as I can tell, the ActiveResource concept, on which this approach is based, is merely an attempt to impose a constrained CRUD data-model over HTTP. There is no concept of content negotiation – all data is XML according to pre-assumed schemas. There is no support for automatic discovery and use of hrefs between resources. There is even the suggestion that it’s a good idea to casually extend the HTTP protocol with extra custom methods.

All of these are typical problems found in projects which use the name REST without the key bits which make it really work. The end result is just another fragile, application-specific RPC protocol. Sigh.

nutrun » Blog Archive » Testing web services with ActiveResource

Ted Neward has posted a long and detailed discussion of the potential merits and disadvantages of Google’s “protocol buffers” approach compared with XML as a way of offering rich remote APIs.

I learned a lot from this article, and I’m still digesting it, so no snap opinions this time …

Interoperability Happens – So You Say You Want to Kill XML….

And adding to the pot, InfoQ have also written about the same subject:

InfoQ: Google Introduces Binary Encoding Format: Protocol Buffers

I don’t need to say much about this one. Re-implementing a clumsy version of HTTP on top of SOAP seems somewhat crazy.

mnot’s Web log: The WS-Empire Strikes Back… feebly

Well worth reading even if you think you understand what REST is all about.

InfoQ: REST Anti-Patterns

Martin Fowler’s “bliki” is often an interesting read. This recent article is very thought-provoking, offering a clever way to bridge the gap between fully dynamic and fully-static web pages.

In regular web development, there is a stark choice between marking a web page as fully dynamic (typically by adding a sprinkle of “don’t cache me” headers) and marking it as static and unchanging. Dynamic pages can bypass web-caches to provide up-to-the-minute information, but force the server to handle the load of every request from every user. Static pages gain the speed and scalability advantages from distributed caching but can fall foul of many problems including stale data and user-session-confusion.

Fowler proposes a technique of using separate HTTP requests controlled by JavaScript to fetch dynamic page fragments and merge them with the main, static, content.

The new bit about this is the solution of using JavaScript to fetch fragments and modify the HTML. The overall approach has been used successfully since the dawn of modern web browsers. – it’s very similar in concept to the way the old faithful “img” tag has always been handled.

MF Bliki: Segmentation by Freshness

Saddening.

It seems that the team responsible for Astoria have decided to go for the traditional Microsoft “embrace and extend” approach to the beautifully simple world of REST.

The whole point of REST is that a GET is a GET and a PUT is a PUT. GET reads a resource, PUT updates it. These operations are idempotent and simple to implement both for the server and for the client. Any requests which non-idempotently change state should use POST.

It seems that, rather than put some thought into how to expose resources in a way which allows a combination of these basic operations to perform any needed task, the Astoria team have decided that new HTTP verbs are required, and are now championing the addition of a MERGE verb.

This is (in my opinion, of course) flatly wrong.

Conceptually MERGE is a loosely defined operation, which will by its nature be implemented differently in different contexts, so nether the client software nor the network infrastructure will be able to make any simplifying assumptions. Worse, if it comes into any kind of common usage it will effectively require all servers and clients to be aware of it.

This verb is simply not needed. If the resources being addressed are too big for a GET/modify/PUT cycle on a whole resource to be effective, just provide a way of addressing parts of the resource.

Sigh.

InfoQ: Merge, Replace, or Patch: How Astoria Handles Changing Data

It seems as if the term SOA (Service-Oriented Architecture) has become confused with the implementation technology (often WS-* web services). Some pundits are now trying to create a new term “WOA” (Web-Oriented Architecture) to describe a service-oriented approach using native web technologies such as HTTP, URLs and REST.

ZapThink have taken a look at these terms and their implications: WOA is Me – Another Acronym? WOA and SOA

I can’t decide if this is a good idea, or a bizarre attempt at box-ticking. An alphabet soup of acronyms to try and merge the laid-back world of REST with the world of business process workflow.

InfoQ: WfXML-R: REST based process integration

Another attempt to compare REST and WS-*. Like so many others it ignores important distinctions (such as the resource-based nature of REST against the service-based nature of WS-*) to focus on a particular common subset. However, it is an interesting indicator to how the discussion is moving.

InfoQ: A Fair Comparison of REST and WS-* using an Architectural Decision Framework: is the Debate Over?