Does it make sense to build your own workflow engine?

A “workflow engine” is becoming the new must-have for enterprise system development. In days gone by it might have been an automatic choice to go for an expert system, Enterprise Service Bus, messaging infrastructure or big-ticket database, but those now seem a little bit passé.

There are several commercial workflow engines available, and a whole bunch of open source ones. Here’s a list of workflow engines written in Java, for example.

Boris Lublinsky has written an article at InfoQ expressing the strong opinion that writing your own workflow engine should not be an option.

I’m not sure I completely agree. For me the most telling part of the article appears near the end:

People today rarely implement their own database or O/R mapper or application server. Why is it often that people think that they should write their own workflow engine?

As it happens, I have implemented my own database, O/R mapper and application server, and found the experience invaluable in understanding the challenges and important features of such software. Writing a workflow engine would presumably be a similarly valuable lesson.

Stringtree/Mojasef for Java 1.4 are now proper branches

A few months ago I made the decision to move the Stringtree and Mojasef code-bases on from their requirement to support older Java versions. I tagged a particular version of the code as “1.4 final” and proceeded to work through the trunk code in the repository to bring it in line with key Java 5 features such as generics, varargs and the enhanced for loop.

Since then I have progressively realized that I may have been a bit optimistic. A few odd bugs have been found in the 1.4 code, and a few key improvements have been made to the Java 5 code which by rights should also be implemented for 1.4. So I have decided to try and fix the situation.

As of today, the tag “1.4-final” in both Stringtree and Mojasef should be considered as deprecated, and any code which used to use that tagged version should instead use the branches (“Stringtree for Java 1.4″ and “Mojasef for Java 1.4″) I have just created. Initially, the code is identical, but the plan is to go through and apply known bug fixes and fully-backward-compatible internal improvements to the branched versions.

JSON Schema Proposal

What a great idea! a schema definition for JSON, in JSON. I’m really tempted to build a validator for this format using my Stringtree JSON code.

JSON Schema Proposal

A kind of answer to “Is it time for Java 5?”

A week or two ago I wrote asking Is it time for Java 5?. I got a few answers, but nothing definitive. Today, however, I stumbled upon a blog post from Alex Miller which helped me decide.

Apparently, Java 1.4 will officially reach its end of service life (EOSL) at the end of October 2008. That of itself does not imply that nobody will continue to use it, but it does help third-party providers such as myself decide when to stop active support.

So, this is my decision. Some time after the start of November I will make a branch of the final Java 1.4-compatible version of all my public projects. Any future changes will no longer guarantee Java 1.4 support.

This does not imply, of course, that all my public code will suddenly be rewritten; dripping with annotations and generics and so on. The process will be incremental and progressive. The main point is that I will no longer be ensuring Java 1.4 compatibility in future releases.

As an aside – if you are still using Java 1.4 and would like to suggest a change to any of my software, please ensure that you get in touch in time for me to include it before November!

Java 6 breaks JDBC

I’m cross. Very cross. Cross with Sun for releasing a new version of Java which shatters both backward- and forward- compatibility. Cross enough that I cannot see any sensible way of moving my software to Java 6 in the near future.

It all started with an innocuous question in a comment on my Punchbarrel blog. I had posted asking for opinions on a move to Java 5, potentially abandoning Java 1.4, for the core Stringtree codebase. The question in the comment was about skipping Java 5 and moving directly to Java 6. This would normally be too big a leap, but I replied that I would endeavour to continue my policy of ensuring my code works with as wide a range of Java versions as possible.

Then I actually tried to do it, and that’s what made me cross. The more I attempted to produce code which would compile and run on Java 1.4, Java 5 and Java 6, the more impossible it began to seem.

I was already aware of the first hurdle. Sun have added new methods to a lot of key JDBC interfaces by tagging them with the new javax.sql.Wrapper interface. This is actually relatively easy to fix in a compatible way. Just add two new methods to each class which implements one of the affected JDBC interfaces:

    public boolean isWrapperFor(Class clz) {
        return false;
    }
 
    public Object unwrap(Class clz) throws SQLException {
        throw new SQLException("Not a Wrapper for " + clz);
    }

The actual method signatures in the Java 6 Wrapper interface are phrased in terms of Generics, but the above stripped version compiles fine using Java 1.4, Java 5 and Java 6 compiler and libraries.

However, even after adding these arguably pointless methods to all my concrete implementations of affected JDBC interfaces, I still had a bunch of compilation errors when using Java 6 libraries. And this is where Sun have really screwed up.

Several other key JDBC interfaces have also been extended. But this time it has not been done by anything as simple as tagging with a new interface. These interfaces have all gained extra method themselves. This should not be a deal-breaker. It should be feasible to just add implementations of these methods to the existing Java 1.4-compatible code. After all, any class is free to define any methods it likes, not just those from an interface.

Nope.

It is simply impossible to add these new methods to a class and have that class still compile in a Java 1.4 or Java 5 environment.

The reason is that these methods are themselves defined in terms of classes and interfaces which do not exist in earlier Java versions. For example, the java.sql.Connection interface gains methods referring to new interfaces NClob and SQLXML

There is no answer to this. Sun have broken backward and forward compatibility of JDBC in Java 6. It is no longer possible to write an implementation of several key JDBC interfaces in a way which compiles under all the most popular Java versions.

Once again, Sun completely misunderstands the real world. Not everyone is free to upgrade every deployment to the very latest Java version immediately it is released. Even within those who do manage to update all their machines in one go, not everyone can immediately drop real work to spend time messing with old code which should still work to bring it into line with a new fashion.

As I wrote at the start of this rant. I’m cross.

Grrr.

Is it time for Java 5?

A major goal of the Stringtree software project has always been to be as compatible as possible with all the software people are using for their Java development. Naturally that also includes whatever Java version is being used.

For a long time I interpreted this goal as implying that all Stringtree code should run on all Java versions from Java 1.2 onwards. Java 1.4, however, introduced some compelling new features including built-in regular-expression handling. For a few years I still tried to ensure that most code was still 1.2-compatible (for example by using Ant to swap in a third-party regular-expression library while building a jar file), while also providing a Java 1.4 version. Eventually, use of Java versions prior to 1.4 declined enough that I felt comfortable removing the complicated pre-1.4 version.

For the last few years I have been very careful to keep all my Stringtree code compatible with all versions of Java from 1.4 upwards. Now, however, the pressure is building again to move over to Java 5. In my day-to-day coding I develop with Java 5 and make increasing use of Java 5 features such as the enhanced for loop, the Iterable interface, enums, generics, autoboxing, varargs and so on. It would be very nice to be able to update the Stringtree codebase to use these features too.

Occasionally a Java 5-specific detail has crept in to a Stringtree library, and I have soon received comments or emails pointing this out. I haven’t noticed this for a while, which might indicate either that I have been especially careful, or that I there are no longer any/many people developing with Stringtree code who are still limited to Java 1.4.

If you are reading this and you still require Java 1.4 support, please let me know. Likewise, if you have thrown off the shackles of 1.4 within the last year or so or are desperately hoping for a Java 5 Stringtree that would be good to know too.

Is it time for Java 5 yet?

Slight Improvement to Stringtree XML Parser

Someone just pointed out that the light-weight XML parser included in Stringtree did not handle explicit CDATA blocks. The version in SVN now has provisional support for this.

If you need a simple and fast parser for textual data, then this should be all you need. For XML documents containing opaque binary data in a CDATA block, this may not be ideal. Currently CDATA blocks are loaded as String objects, and this can lead to incorrect data for bytes which do not represent valid characters in the current character set.

I am currently planning for the next version of the Stringtree XMLReader to offer the option of extracting a CDATA block as an unprocessed byte array.

Stringtree Maven Repository

I have experimented with Maven a few times, but never been particularly impressed.

Recently, however, one of my users has been bugging me to make the Stringtree and Mojasef classes and sources available in a Maven-style repository, for easier integration with Maven (or Maven-like) build tools and workflows. So I have done it. The current versions of key Stringtree and Mojasef files are now available from the Stringtree repository at the following URLs:

The Stringtree and Mojasef build scripts now include a “publish” target which uploads such artefacts to appropriate places in the repository, so future versions should continue to be available.

This is the first time I have done this, so I would welcome comments from any readers who actually use Maven (or buildr, or anything else which supports this repository format). In particular, I am interested in opinions on whether it is OK to simply replace artefacts with the same names and locations for minor tweaks and bug-fixes, or whether even the smallest change should result in the generation and upload of a new version with a new name and location.

A Simple HTTP Client – Part 1 (Overview)

This post is the first of a series describing the use and implementation of the Stringtree HTTP Client.

Recently I have been working with systems which talk to each other using REST/HTTP. Providing services and resources is pretty simple using Mojasef, but accessing such resources and consuming such services (in client code and in tests) has always seemed a bit more clumsy than it should be. I tried Apache HTTPClient and HttpUnit, but both seemed cumbersome for simple tasks, and bring in several hundred KB or more of dependencies, which can really bloat a small client application. I’m sure there are others, but I got bored with looking, and instead wrote my own simple HTTP Client which does the things I need without dragging in tons of extra stuff.

The Stringtree HTTP Client consists of just four classes, with no dependencies other than the standard Java APIs:

The point of these classes is to allow simple construction and calling of all valid HTTP requests, including the ability to set and read headers and cookies, simulate the submission of an HTML form, and support both textual and binary content data.

As a very simple example, consider the following code which issues a GET request to a specified URL:

HTTPClient client = new HTTPClient();
Document response = client.get("http://localhost:8080/?a=b");
System.out.println("response content type=" + response.getHeader("Content-Type"));
System.out.println("response content=" + response.getContentAsString());

The above code example shows basic usage of the Stringtree HTTP Client. In more general terms, usage is as follows:

  1. Create an object of the HTTPClient class.
  2. Set any long-lived settings, such as cookies or a user-agent.
  3. Call one of the request methods with appropriate parameters.
  4. Read returned headers and content as required from the returned Document object.
  5. Repeat from (3) for each new request.

A slightly more complex example using a POST request to submit an HTML form might look like:

HTTPClient client = new HTTPClient("Mozilla/5.0 (example)");
client.setCookie("username", "Frank");
Form form = new Form();
form.put("name", "Widget");
form.put("category", "thing");
Document response = client.post("http://localhost:8080/update", form);
boolean ok = "200".equals(response.getHeader(HTTPClient.HTTP_RESPONSE_CODE));

All the request methods return an org.stringtree.http.Document object. This object represents the structure of an HTTP request or response: a collection of name/value headers (which may contain duplicate names), and a block of “content” which may be considered as text or as a sequence of bytes. The HTTPClient code does make one simplifying concession; as seen in the above POST example the HTTP response code is added as a pseudo-header with the name “http.response.code”.

This should be enough to get started playing with the Stringtree HTTP Client. In the next post I will discuss the possibilities for creating and configuring an HTTPClient object in full detail.

Mojasef goes into beta at last

Finally, I have got to a point where I reckon Mojasef is nearing a proper numbered stable release. To celebrate this I have updated the version number in the Mojasef source code to 2.0.b1 and made a downloadable release (source and jar) available at Sourceforge.

I strongly encourage anyone interested in shaping the direction, contents and/or timing of the official 2.0.1 release of Mojasef to grab this release, have a play with it, and let me know your comments and suggestions.

I know that it is light on documentation, especially as even the examples on the web site are not quite accurate any more, but that is now very high on my list of priorities.

Stringtree JSON 2.0.9

Along with the latest release of Stringtree I have released a new Stringtree JSON.

The major change is the provision of an extra version of JSONReader.read which takes a CharacterIterator to allow reading of JSON from within a larger body of text.

I’ve also had another try at ensuring that it is Java 1.4 compatible. Please let me know of any problems you encounter, particularly Java1.4 incompatibility issues.

Download from Sourceforge as usual, and the source is in with Stringtree 2.0.9.

Have fun.

Stringtree 2.0.9

It’s been a month or so, so it’s time for another release of Stringtree. Changes in this release include:

  • fixed some broken EasyTemplater constructors
  • added switchable prefixing of attributes in XMLReader
  • added switchable forcing all values to lists in XMLReader
  • added storing cdata from mixed elements as “text()” in XMLReader
  • added extra “read” methods to JSONReader to allow calling with a CharacterIterator
  • XMLReader now skips a DOCTYPE without complaining.
  • tests.Hierarchy renamed to tests.tree
  • small tidyup of object creation to allow reference to context items instead of full class+parameter specifications
  • support a naive view of namespaces in XML parser, include colonprefix in the name
  • some slight improvements to class creation stuff in util to better deal with multiple calls to “init”.
  • added a GUID generator for use in REST-style applications

Download available from Sourceforge as usual

Demo Friki currently out of action

I have just been informed by my hosting provider that they have had to disable the Java support on the server running the example Friki. Apparently this is due to too many applications from different users clogging up the stderr logs. I won’t be able to re-enable Friki until I have ensured that it does not log anything to stderr.

In practice I have not touched the Friki code in ages, and the demo installation was filling up with Wiki spam, so I might take this opportunity to bring Friki into line with current Stringtree practice.

Sorry if you were looking for a demo, but I’ll announce here when it is back again.

Read JSON from a CharacterIterator

I just had a pleasant email exchange with someone who is interested in embedding Stringtree JSON in another project. In this particular case, the basic functionality of the JSONReader is fine, but the calling API was not quite aligned with what they need.

So now, as well as reading JSON from a String, you now have the ability to read JSON from a CharacterIterator. This was a neat changes, as Stringtree JSONReader uses a CharacterIterator internally anyway, so the extra method actually does a bit less than the original one.

For “power users”, there is actually an extra method. When JSONReader starts reading from a CharacterIterator it’s not entirely clear whether it should get its first character by calling current() to get a character already read once, or by calling next() to get the next unread character, or by calling first() to get the first character of the sequence. With this in mind I have provided a read(CharacterIterator ci, int start), where start can be JSONReader.START, JSONReader.NEXT, or JSONReader.CURRENT.

This update is available in sourceforge subversion, and will be included in the next release of Stringtree and Stringtree JSON.

A Mojasef improvement prompted by thoughts of Spring

Following on from my post a couple of days ago I’ve now added a small extra feature to Mojasef which should add a whole range of extra power. One of the things it enables is a much closer integration with Spring, but that’s only a small part of the possibilities.

The way this new feature is used is very simple. If you recall from the Mojasef documentation, the single most important Mojasef configuration is the http.application value. This may be specified as a system parameter or defined in a “spec” file, but the end result is the same :- it’s the object which provides the methods which eventually get called for each incoming HTTP request.

The new feature supports a similar (but optional) configuration http.application.context. If you define such a configuration, and the specified object implements org.stringtree.Fetcher or java.util.Map, it will be treated as an integral part of the common “context” available to all application methods.

One upshot of this is that if you define http.application.context to be an org.stringtree.spring.SpringFetcher, then all your Spring objects will be available as “first class” objects in your Mojasef applications. No more need for a “spring.” prefix in templates, or extra application code to fetch, cast, and eventually use a Spring BeanFactory.

The use of this new feature is not limited to Spring, though. You can supply any Map or Fetcher implementation, which means that you can use any static or dynamic name/value mapping. I haven’t written any implementations of such things yet, but there is no reason in principle that this technique could not be used to integrate diverse sources such as JNDI/LDAP repositories, databases queried by SQL, XML documents queried using XPath or even asking the whole web using a search engine such as Google.

And, don’t forget that you can combine any or all of these either using the built-in Stringtree FallbackFetcher mechanism, or by providing your own aggregating implementations of org.stringtree.Fetcher or java.util.Map.

As a final twist, Mojasef is smart enough to try alternatives if you don’t want to manually specify such a context. It was not an accident that http.application.context starts with http.application. If your http.application object makes available a property named context (typically by providing a getContext() accessor, or by exposing a get(String name) method which returns a non-null value for the name “context”) then Mojasef can use that too.

This code is currently in subversion, and will be included in the next stable Mojasef release. Example JUnit tests for the new feature can be read here and here

Integrating a Spring context with Stringtree

I don’t tend to use it myself (preferring the “spec” mechanism built-in to Stringtree), but today someone asked me if a Mojasef application could work with Spring.

After a bit of thought, some downloading, and a little playing, here is the result. My intention was to allow beans configured in a Spring application context to act as first-class citizens within a Stringtree context, so that they can be used seamlessly inside Mojasef application code and templates.

The implementation was actually pretty simple, essentially consisting of creating a class which implements org.stringtree.Fetcher and provides access to any Spring beans within a Spring context.

package org.stringtree.spring;

import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanFactory;
import org.stringtree.Fetcher;

public class SpringFetcher implements Fetcher {

	private BeanFactory factory;

	public SpringFetcher(BeanFactory factory) {
		this.factory = factory;
	}

	public Object getObject(String name) {
		Object ret = null;
		try {
			ret = factory.getBean(name);
		} catch(BeansException e) {
			ret = null;
		}

		return ret;
	}
}

To use this class, you will need to create an instance of it, passing in a Spring BeanFactory or ApplicationContext object to the constructor. Then the SpringFetcher object can be used just as any other Fetcher.

It seems that when using Spring in a Mojasef application, what will likely be needed is a single Mojasef context which provides access to both Mojasef and Spring objects. To achieve this, simply wrap the two Fetchers in a FallbackFetcher:

	Fetcher spring = new SpringFetcher(new XmlBeanFactory(new FileSystemResource("application.xml")));
	Fetcher map = new MapFetcher();
	Fetcher ff = new FallbackFetcher(spring, map);

An example Eclipse project containing the SpringFetcher class, some unit tests, and the jar files to compile and run them is available in sourceforge subversion at https://svn.sourceforge.net/svnroot/stringtree/projects/spring_example/trunk.