First I watched an inspiring session from TechCrunch Nordic which likens achieving an “exit” for a startup company to dating. Fun, and with a strand of truth.

TechCrunch Nordic – Tommy Ahlers from Mike Butcher on Vimeo.

Anyone running a startup, or thinking about it, should watch this one.

Second I watched a presentation ostensibly about Kanban and “single piece flow”, but really about much wider issues in planning and managing software development. I found the approach presented particularly interesting as it correlates very well with where my thoughts are at right now.

This video is best watched at infoq, to see both the presenters and the slides

Anyone working in software development, or managing a software project really needs to watch this one.

Watch live video from Startup Lessons Learned on Justin.tv

If you are at all interested in software and/or startups, this is well worth a viewing.

Via Energized Work.

You have to love a good analogy. Here’s one which takes the notion of a “waterfall” development process literally, and contrasts it with a pleasant day out on a serene pond.

The Opposite of Waterfall is Pond – A Metaphor for Agile | Agile Blog: Scaling Software Agility

My favourite snippet:

Eventually, we find a place that everyone agrees looks nice and we pull the boat up on shore. Our project is complete in a way that we couldn’t have predicted exactly because we’ve never been on this particular pond before. We’re ready to set out again just as soon as we’re done with the picnic. In Pond, you always have a picnic at the end of the project.

All of which makes me smile. Back when I was working on an agile project where releases were named after ducks and other wildfowl, I proposed at one point a component named “pond” (standing for “provider of necessary data”) shared by all the deployed ducks.

Ah nostalgia.

Agile is mostly driven driven by managment and consultants, seldom bottom up by developers.

I found this very odd, and completely contrary to my own experience. From the perspective of someone who first encountered agile approaches in the form of Extreme Programming the problem is almost always how to explain the obvious developer benefits in ways which management will understand.

This phrasing of agile for managers seems to be the purpose of Scrum. I tend to recommend that puzzled managers read up on Scrum, but puzzled developers start with XP and Test Driven Development, which provide some much more tangible developer advantages.

via Code Monkeyism: What Developers Need to Know About Agile.

Now imagine that you have a large chunk of time (months to years) and a team of several people (or perhaps even several teams). The daunting challenge is how to plan the work so that it all gets done and all works together.

Agile Coach and Mentor Chris Pitts of Thirsty Bear has recently written some thoughts on this issue which broadly align with my experience.

It’s extremely tempting to begin by setting separate people or teams to work on separate parts of the system. Even if the team are working together, it’s tempting to take on one part of the architecture diagram at a time and continue until all the blocks are in place.

This risks several common problems, though.

The most obvious is that this heads straight for the nightmare of “system integration” – that theoretically short, but all too commonly deadline-busting, period after all the components are “done” but before the system can be shown to actually all work together. This is the part of the classic design/build/test lifecycle which is often ignored yet can force significant (and rushed) changes to the components when assumptions are shown to be wrong.

A less obvious problem is that the longer the delay before being able to show full end-to-end operation, the longer the wait for feedback from all the stakeholders who only care about what the system does on the outside, not how it does it inside. Building a system based on guessing what the stakeholders want based on theoretical discussions, documents, and the hot features offered by some big-ticket infrastructure is very risky compared with real feedback on a system which does at least one small thing completely.

Chris suggests

Slice the functionality, not the components

Which is good advice as it goes.

I’d add a warning that it is also all too easy to define the tasks to produce a system in terms of parts and modules. A task card such as “install database”, or a story for “web client” risk distracting the team from the real desired outcomes the stakeholders care about. If you see any such stories (or tasks, issues, tickets or whatever you call them) try and find some end-to-end scenarios or use cases which require such things, and describe them instead. When that story is tackled, then the required infrastructure and components can be implemented as part of that story.

However, it has been almost impossible to take that expertise, hard work, and custom code and easily/cheaply turn it into usable web applications for general public use. Low-cost hosting providers have generally shunned Java support. The most reasonable one I have found is LunarPages who actually support both ad-hoc JSP and deployment of custom web applications as war files, even though you have to search pretty deep in their web site to find out, and pay an extra dollar or two a month for the privilege.

So for my own projects I have been looking for an alternative for a long time. A way of developing and testing web apps on my various development boxes (currently running Ubuntu, MacOS X and Windows XP) and easily deploying to a low-cost hosting provider.

By far the most popular web development language is PHP. It’s available pretty much everywhere. It’s so focussed on web app development that the primary unit of coding is the web page. Believe me, I have tried to like PHP, but it’s just so clumsy. After a few paces the application begins to get tangled, development speed drops, and bugs creep in. Its web focus makes it tricky to unit test, and end-to-end testing seems to require a full-fat HTTP server. Not for me.

Beyond the traditional stomping ground of PHP, perl, and other CGI fodder is the new range of “trendy” languages. Ruby, Python, Erlang and even the venerable smalltalk are trying to position themselves as the thinking-person’s web development tool.

Erlang syntax is a bit too odd for me right now, although I may come back to it later. Smalltalk is interesting, but carries with it so much history and is hardly a popular choice for low-cost hosting providers. That leaves a short-list of Python and Ruby.

As languages I like both Python and Ruby. They have broadly similar design goals, both have keen developer communities with plenty of open source resources, and both are commonly found on linux-based web hosts. In order to decide between them I took a look around their frameworks and APIs.

Python has Google on its side. It’s the “native language” of the hugely-scalable Google AppEngine. However, a scan around the web looking for ways of writing web applications led largely to two: Zope and Django. (for balance, there are plenty of others, but these are the standout examples). Zope is old and sprawling. It has some great ideas but is hardly an obvious choice for small, tactical web apps. Django is a bit lighter, but still seems to assume a lot and require a lot of relatively fiddly config.

In the Ruby world the blindingly obvious choice for a web application framework is Rails. Sometimes it seems as if “Ruby” is just another way of saying “Rails”. However Rails, like Django, assumes a lot about the eventual application. Rails also makes a lot of use of code generation, which I simply do not like. I would always prefer that a framework eliminate boilerplate rather than just generate it for me. Looking a little further beyond Rails I came across some really interesting alternatives, and the one which really sparked my interest is Sinatra. For me this one framework made the difference. It’s so streamlined that a basic web app is as simple as:

require ‘sinatra’
get ‘/’ do
‘Hello world!’
end

Best of all, simply running the above file using ruby hello.rb starts up a web server and begins serving pages on port 4567. No extra config or faffing.

Looking around for low cost hosting I found that the provider I already use (Dreamhost) supports Ruby web hosting using Passenger. An ssh to the Dreamhost server for “gem install sinatra” followed by a little bit of FTP and my first Ruby/Sinatra application was live!

A key part of any web application is the pages, and a key part of generating pages is a good template language. As a general-purpose templating language, I still prefer my own Stringtree Templater, part of the Mojasef Java framework. So far I have not found a similar templating solution for any other language. However, for the limited and specific requirement of generating web pages, I am becoming quite fond of Haml. It’s not at all a general templating solution, but it does massively simplify the generation of web pages.

Finally, after years of CVS and Subversion, I have made the jump to distributed version control. Not only is it more trendy, but the ability to work on multiple code branches, on multiple machines, with or without an internet connection, and easily share, identify and merge when necessary has been a key benefit. Occasionally I wonder if one of the other distributed VCS (Mercurial, Bazaar, darcs, etc.) would have been a better choice, but I needed to settle on one and get to grips with it. So I chose git.

There have been a few speed-bumps. Attempting to learn git from manual entries and simplistic tutorials gives very little help on deciding what is worth doing. After about a week of following the techniques in A Git Workflow for Agile Teams, though, I began to get comfortable. Add to that some lovely web deployment tricks from A web-focused Git workflow and a selection of GUI tools ( Git Cola, GitX, and Git GUI ) to get round all that tedious “git add” stuff and simplify infrequent operations, and git becomes a very useful and productive tool.

The upshot is that I currently feel very productive.

My toolbox has a few other things in it, but they are still candidates for change if I find anything better. I am managing my projects and git repositories using Unfuddle but it has a few shortcomings including a ticket system which is a pale and clumsy cousin of bugzilla. For editing I use generic syntax-highlighting text editors (gedit on Linux, TextEdit on Windows and Smultron on Mac) but none of them have the nice autocompletion and refactoring tools I grew addicted to in Eclipse. Suggestions welcome!

Finally, a few spare links relating to the tools mentioned above.

Sinatra: 29 Links and Resources For A Quicker, Easier Way to Build Webapps

HAML reference

Rack

Mini reviews of 19 Ruby template engines

The Forgotten Ruby Web Frameworks

It’s natural enough to assume that problems with implementing agile processes in large organizations stem from some characteristic of the organizations themselves. Amr Elssamadisy, however, has a different suggestion. Perhaps some of the problems stem from the use of external agile “coaches”, and the way that they affect the way both the agile team and the organization operates simply by being there.

Read more at: InfoQ: Opinion: Agile Coaches Frequently a Source of Adoption Problems.

The cost is essentially $50 (to the Scrum Alliance for the certificate) plus a voluntary contribution to cover any room costs etc. Sort out your own food and transport. Even for those of us who regard agile certification as a dubious concept this is pretty tempting.

Reading between the lines, though. It appears that the different agile “camps” may be slipping into a battle for mind-share. If Scrum certification becomes more widely available and understood, then alternative approaches (particularly XP, but also all the other adapted, customized and home-brewed agile working practices) may seem to have less value to employers and clients.

Hmm…

With that in mind we are casting around for any hints on how to map out what we build and in what order. Chris Sims has some suggestions which may be useful, but I’m not sure they address our underlying problem.

InfoQ: Story Mapping Gives Context to User Stories.

Are retrospectives an antipattern? | Steven “Doc” List’s Random Musings.