Frank Carver's Punch Barrel

I have encountered several web articles recently which attempt to describe REST verbs (“get”, “put”, “post”, “delete”) in terms of traditional database CRUD (“create”, “read”, “update”, “delete”) operations, and then get stuck trying to map one set to the other.

On the one hand, it’s easy to see why the confusion happens. Both sets have four basic operations, and both sets are concerned with manipulation of stored resources, both have an operation called “delete”, and “read” sounds pretty much like “get”. However, that’s where the similarity stops.

For some recent examples of confusing REST and CRUD, see:

REST and usablity shouldn’t be mixed

Two stage resource oriented web application architectures

Hazem Ahmed Saleh: The rules behind designing a good RESTful service

Even the, otherwise very good, REST Dialogues sometimes fall into this trap.

So, what is the difference?

“Get” and “read” are essentially the same, at least when it comes to fetching raw resource data.

There is a difference in usage, though. In REST a “get” fetches a single resource. CRUD, on the other hand, is often associated with relational databases, which excel in returning sets of results. The common REST approach to multiple results is to return a resource which includes URIs for each resource in the collection (perhaps also with some summary information); the REST client is then free to individually “get” any or all referenced resources which it requires, or if necessary to instead fetch them from a local cache. As an example, consider the loading of images in a web page.

“Delete”, although similar on the surface has some important differences underneath.

In CRUD, the relational heritage often implies the ability to do complex deletes, such as deleting all resources which meet some criteria. In REST the “delete” operation is a simple delete of the resource at a single URI. To delete multiple items using REST typically requires multiple, separate, “delete” requests.

“Put” and “update” can be similar or different depending on interpretation.

The REST “put” operation has strict and well-defined semantics. It ensures that the specified URI has the supplied resource. If the URI is invalid it fails; if the URI contains no resource the supplied resource is placed there; if the URI contains a different resource it is replaced with the supplied resource; if the URI contains the same resource there is no apparent change to the resource at the URI. The CRUD “update” operation, on the other hand is less well-defined. SQL standards do not include a “replace”, “put”, or “insert or update if exists” operation (although many vendor implementations implement such features in a non-standard way). With this in mind, some people interpret the CRUD “update” operation as equivalent to an SQL “update” statement (which only applies if the resource already exists), while others treat it more generally as any form of change which is not a “delete”.

This only leaves “post” and “create”. These are the most different of all.

In REST, “post” is used for any action which results in a resource change or side-effect, and cannot be repeated without the change or side-effect happening again. An SQL “insert” operation would be implemented as a REST “post”, for example, but so would an SQL “update” which increments a value, or the sending of an email. A practical rule of thumb is that a “post” should be simple and quick, leaving the way for future changes using “put” as soon as possible. A CRUD “create” is a quite different thing. “Create” explicitly creates a new resource, and is undefined if that resource already exists.

REST has no direct analogue of “create”. If it is needed (for example) to create and populate a new resource, it is typical to call “post” on some parent resource which results in the return of, or redirect to, a new (empty) URI, and then populate the URI with one or more “put” operations. This approach has the big advantage that the impact of an accidental re-”post” is minimised. In a well designed system, the creating of a new URI could be as low-impact as incrementing an integer, only requiring storage of resource data if/when it is later populated using “put”.

CRUD, on the other hand, has no direct analogue of “post”. A CRUD “create” will probably have side effects, as will some CRUD “update” operations. CRUD is also limited in that it does not include the concept of side-effects outside the resources being managed. There is no way to describe sending an email in terms of CRUD, for example.

In summary, the REST and CRUD are different in many ways, some subtle and some less so.

REST is an approach designed to enable scalable distributed applications to clearly distinguish actions which make no change and thus may be cached or repeated (“get”) from those which make a change but may be casually repeated without problems (“put”, “delete”) from those which should neither be repeated nor cached (“post”). This in turn enables intermediate software (be it in the form of libraries, reusable components, or entirely separate systems) to implement sensible optimisation and error-handling strategies without needing knowledge of the content or relationships of the resources.

CRUD is a description of some generic operations on stored data, with no particular intention of serving as a distribution protocol, and thus little or no planning for the mitigation of problems such as accidental duplicates, time-outs or lost requests.

Attempts, however well-intentioned, to explain one by analogy with the other run a great risk of increasing confusion rather than clearing it. If you are considering using such an analogy, please think twice.

InfoQ have published an article which offers a compelling argument that common corporate performance reviews are fundamentally broken, and should be abandoned. While I largely agree with this premise, I don’t see a great deal of improvement in the proposed solutions.

Perhaps it reflects differences in the kinds of places we have worked, but for me an effective performance review should concentrate on a holistic view of how the individual has contributed to the success of the organization. I am worried that neither of the two proposals encourages “thinking outside the box” and doing whatever is most effective for the business. Instead, they concentrate on reinforcing existing decisions such as pigeon-holing staff into “team members” and “managers”.

InfoQ: Performance Reviews Banished

Agile processes are an obvious choice for development work within a business, but have traditionally not sat well when dealing with customers and contracts who like to specify detail, delivery and price before work starts. Negotiators and lawyers have no obvious answers to hand and, rather than take the risky route of creating a new agile contract, usually fall back to the apparent safety net of an implicit waterfall process.

If there were some example “agile” business contracts available, some of the risk for the contract negotiators might be removed.

InfoQ: Working Group Formed to Produce Reusable Agile Contracts

I’m currently working on some software which sends notifications to users (using SMS, email, or whatever) and have faced the inevitable problems with testing it. On balance I’d prefer not to receive a test SMS on my mobile phone every time our continuous integration system runs an end-to-end test.

Gojko Adzic has some thoughts about how to make such a system more straightforward to test.

Gojko Adzic » How to test e-mail notifications properly

BBC News | Hi-tech brings families together

Looks like an interesting list, especially if you know a company which is looking for a new CEO right now.

The Seven Things That Surprise New CEOs — HBS Working Knowledge

I try to avoid getting involved in US politics, but I really enjoyed the cleverness of this video (was it actually a TV ad, or is it just an internet meme?) and mentioned it to some colleagues yesterday, so it deserves to be here, I guess.

YouTube – Wassup 2008

As a comparison, see also the original, and my kids’ favourite one. There’s plenty of others too.

Team composition and team dynamics are fascinating me right now. We are trying to grow an effective development team, and encourage a process where both the team as a whole and the individuals are doing their best for the company, the product, the customers and them selves. Linda Rising has some thoughts about some of the things that can perturb this effort.

InfoQ: Interview: Linda Rising: Prejudices Can Alter Team Work

We have some pretty lively iteration retrospectives as part of our process, but so far they have almost exclusively focussed on the issues affecting the process. Sarah Taraporewalla wonders if such discussions should also cover thoughts about the code base and the changes made to it.

Sarah Taraporewalla’s Technical Ramblings » Retrospectives for the code base

Looking at the the way collaborative, creative, teams work in fields other than software development can be very useful. There’s an argument that creating a software release is a lot like creating a music album, for example. Chris Johnston has followed this train of thought and asks if we need an analogue to the “producer”.

Chris Johnston » Who plays the role of technial producer on Agile teams?

I’d not seen this, and if you haven’t either then it’s well worth a read. Tim Ottinger gives a detailed list of the ways that Test-Driven Development (TDD) changes code.

Testing Will Challenge Your Conventions

In my work as a software developer I mostly encounter stories in the form of “user stories” – a way of communicating about a change or new feature by describing it as a story. In the wider world, and in the other things I do with my life, stories play a much larger part. Stories form the basis of a large part of human communication and empower everything from watercooler gossip, through books and newspapers, to TV and games.

Stories seem a large part of our lives, and of the lives of everyone.

elearnspace: The Secrets of Storytelling: Why We Love a Good Yarn

The Secrets of Storytelling: Why We Love a Good Yarn: Scientific American

REST is a very powerful way of thinking about application design, but it can be very hard to really understand. Several people including the originator of REST, Roy Fielding, have written on the subject, and InfoQ has attempted to summarise some of these articles.

InfoQ: What Makes Good REST?

Up until recently, I was using a WordPress plugin called “WP-codebox” to format my code samples in this blog, but today it broke my RSS feed, so I have abandoned it. I liked the “geshi” syntax highlighting engine used inside it, but unfortunately it generated flaky HTML, and was not smart enough to disable JavaScript in feeds.

I have now replaced it with WP-syntax. using it is very similar, just wrap code examples in (for example)

<pre lang="Java">
class Whatever {
  public void thing() {
    if(thisHappens()) {
      doThis();
      doThat();
    }
  }
}
</pre>

gives

class Whatever {
  public void thing() {
    if(thisHappens()) {
      doThis();
      doThat();
    }
  }
}

Please let me know if you have any problems with the appearance of code samples, especially if you use a feed reader or aggegator.

To make software development work, everyone involved needs a good working knowledge of the product, the domain, the solution and so on. Communicating enough, but not too much, information can be tricky, especially in an agile environment where anything can change at any time.

Tarek Abdelmaguid has an interesting list of ways to communicate while working on a project. I have done almost all of them with various clients and employers, and also made a lot of use of instant messaging. There is definitely scope for using more structured communications such as mailing lists, internal blogs and forums, though.

On Programming and Applications Development: Project Development Knowledge: Sharing and Enduring

How’s this for an extreme opinion:

If you use an ‘if’ you deserve to suffer at Mark Needham

I actually agree with some of his conclusions, although I am surprised that he does not suggest the simple approach of extracting a method to eliminate the potential confusion of multiple statements depending on an “if”. For example, replace:

class Whatever {
  public void thing() {
    if(thisHappens()) {
      doThis();
      doThat();
    }
  }
}

with

class Whatever {
  private void thisAndThat() {
      doThis();
      doThat();
  }
 
  public void thing() {
    if (thisHappens()) thisAndThat();
  }
}

Seems less trouble than a forced polymorphism approach, at least until polymorphism and the strategy pattern is actually needed.

Planning what order to do stuff in is a vital, yet very difficult, part of software development. Agile wisdom usually stresses the need to do things in order of “business value”, but this can sometimes be extremely tricky to evaluate.

Another approach is to do things in an order intended to decrease risk, such as starting with the most worrying, or least understood parts of a system.

InfoQ: Iterating To Acquire Knowledge, Not Just ‘Business Value’

While this sounds plausible, I am not sure I entirely agree the reasoning. Behind the idea of ordering things by risk seems to be the idea that there is some bounded set of requirements, known early enough in the process to be able to determine their “worry”. To me the whole point of agile development is to step outside that assumption into the real world where work needs to start before full scope is defined, where priorities change on a whim, and development on a product may be cancelled at any time.

Starting work by spending a few iterations investigating complex/risky parts of some imagined future product instead of delivering usable but minimal versions from the first iteration seems somewhat irresponsible.

It’s a long-running argument. Are modern, dynamic, languages such as Ruby and Python really much more concise than more mainstream languages such as Java? What constitutes “big” in each case? Stephan tries to flush out some hard facts, and gets a lot of comments.

Comparing Java and Python – is Java 10x more verbose than Python (LOC)? A modest empiric approach at Stephans Blog

I started the day cursing the stupid jar format, but ended the day smiling. I needed to build a stand-alone executable jar which accessed a MySQL database but kept getting all sorts of build-time and run-time errors.

Normally to make an executable jar I use the excellent pack ant task which build a minimal jar file by trawling all the class dependencies in yout code and including just the required classes from the classpath. For example:

<target name="jar">
   <taskdef name="pack" 
      classname="org.sadun.util.ant.Pack" classpath="lib/pack.jar" />
   <pack 
       classes = "${root.classes}"
       targetJar = "dist/${project.name}.jar"
       manifestMainClass = "${main.class}"
       excludePkg = "java,sun"
       includePkg = "com,org,javax"
   >
      <classpath refid="classpath"/>
   </pack>
</target>

Just set up the main.class property to the main class of the application, and set the root.classes property to a class which refers to everything you need (in many cases this will be the same main class) and you get a lovely small executable jar.

However, trying this approach with MySQL cause a bunch of problems. First I needed to add a load of (apparently dynamically loaded) MySQL class names such as com.mysql.jdbc.Driver.

I thought I was doing well when I had resolved that issue, but then I hit an even harder problem:

java.lang.RuntimeException: Can't load resource bundle due to underlying exception
java.util.MissingResourceException: Can't find bundle for base name com.mysql.jdbc.LocalizedErrorMessages, locale en_GB

Despite spending an hour or so searching and trying, I could not convince pack to find it. So I had to look elsewhere.

The solution I eventually chose was “one-jar” – a trick which subverts the normal jar execution process and creates a custom classloader to resolve jars within a jar. I still use pack to minimise the amount of other classes included in my project, but explicitly include the whole MySQL driver jar.

My ant target now looks like:

<target name="jar">
   <taskdef name="pack" 
      classname="org.sadun.util.ant.Pack"
      classpath="lib/pack.jar" />
   <pack 
       classes = "${root.classes}"
       targetJar = "tmp/main.jar"
       manifestMainClass = "${main.class}"
       excludePkg = "java,sun"
       includePkg = "com,org,javax"
   >
      <classpath refid="classpath"/>
   </pack>
   <taskdef name="one-jar" 
      classname="com.simontuffs.onejar.ant.OneJarTask"
      classpath="lib/one-jar-ant-task-0.96.jar" onerror="report"/>
   <one-jar destfile="dist/${project.name}.jar" manifest="src/main/files/${project.name}.mf">
      <main jar="tmp/main.jar"/>
      <lib>
         <fileset file="lib/mysql-connector-java-5.0.4-bin.jar"/>
      </lib>
   </one-jar> 
</target>

This builds all of my application except the MySQL bits using pack into a temporary “main.jar”, then uses one-jar to build another jar which refers to both main.jar and the MySQL driver jar.

I did need to create an explicit manifest file this way (usually, pack is smart enough to make it for me)

Main-Class: com.simontuffs.onejar.Boot
One-Jar-Main-Class: org.example.project.Main

I now have a working stand-alone executable jar which can access a MySQL database. Cool.

Read more at Deliver Your Java Application in One-JAR™ ! and Java: Using ONE-JAR

A new Firefox plugin, “Geode”, supports location-based services, and an initial application recommends restaurants and other places to eat based on where you are and the type of food you would like to eat.

Firefox Geode: Web sites know where you are | Webware : Cool Web apps for everyone – CNET