Frank Carver's Punch Barrel

Monday was a bank holiday, so I found a little time to catch up with some of the web videos in my queue.

First I watched an inspiring session from TechCrunch Nordic which likens achieving an “exit” for a startup company to dating. Fun, and with a strand of truth.

TechCrunch Nordic – Tommy Ahlers from Mike Butcher on Vimeo.

Anyone running a startup, or thinking about it, should watch this one.

Second I watched a presentation ostensibly about Kanban and “single piece flow”, but really about much wider issues in planning and managing software development. I found the approach presented particularly interesting as it correlates very well with where my thoughts are at right now.

This video is best watched at infoq, to see both the presenters and the slides

Anyone working in software development, or managing a software project really needs to watch this one.

Kent Beck, rightly well-known for Extreme programming, Test-Driven Development and jUnit gave a really thought-provoking talk at the Startup Lessons Learned Conference in San Francisco on April 23, 2010.

Watch live video from Startup Lessons Learned on Justin.tv

If you are at all interested in software and/or startups, this is well worth a viewing.

Via Energized Work.

You have to love a good analogy. Here’s one which takes the notion of a “waterfall” development process literally, and contrasts it with a pleasant day out on a serene pond.

The Opposite of Waterfall is Pond – A Metaphor for Agile | Agile Blog: Scaling Software Agility

My favourite snippet:

Eventually, we find a place that everyone agrees looks nice and we pull the boat up on shore. Our project is complete in a way that we couldn’t have predicted exactly because we’ve never been on this particular pond before. We’re ready to set out again just as soon as we’re done with the picnic. In Pond, you always have a picnic at the end of the project.

All of which makes me smile. Back when I was working on an agile project where releases were named after ducks and other wildfowl, I proposed at one point a component named “pond” (standing for “provider of necessary data”) shared by all the deployed ducks.

Ah nostalgia.

Stephan Schmidt writes

Agile is mostly driven driven by managment and consultants, seldom bottom up by developers.

I found this very odd, and completely contrary to my own experience. From the perspective of someone who first encountered agile approaches in the form of Extreme Programming the problem is almost always how to explain the obvious developer benefits in ways which management will understand.

This phrasing of agile for managers seems to be the purpose of Scrum. I tend to recommend that puzzled managers read up on Scrum, but puzzled developers start with XP and Test Driven Development, which provide some much more tangible developer advantages.

via Code Monkeyism: What Developers Need to Know About Agile.

It’s a common software development situation. In front of you is a problem which seems to require a large solution. It has several parts which may be deployed separately, or may just need to be swapped out or independently managed. It has infrastructure bits and business-specific bits, servers, services and clients.

Now imagine that you have a large chunk of time (months to years) and a team of several people (or perhaps even several teams). The daunting challenge is how to plan the work so that it all gets done and all works together.

Agile Coach and Mentor Chris Pitts of Thirsty Bear has recently written some thoughts on this issue which broadly align with my experience.

It’s extremely tempting to begin by setting separate people or teams to work on separate parts of the system. Even if the team are working together, it’s tempting to take on one part of the architecture diagram at a time and continue until all the blocks are in place.

This risks several common problems, though.

The most obvious is that this heads straight for the nightmare of “system integration” – that theoretically short, but all too commonly deadline-busting, period after all the components are “done” but before the system can be shown to actually all work together. This is the part of the classic design/build/test lifecycle which is often ignored yet can force significant (and rushed) changes to the components when assumptions are shown to be wrong.

A less obvious problem is that the longer the delay before being able to show full end-to-end operation, the longer the wait for feedback from all the stakeholders who only care about what the system does on the outside, not how it does it inside. Building a system based on guessing what the stakeholders want based on theoretical discussions, documents, and the hot features offered by some big-ticket infrastructure is very risky compared with real feedback on a system which does at least one small thing completely.

Chris suggests

Slice the functionality, not the components

Which is good advice as it goes.

I’d add a warning that it is also all too easy to define the tasks to produce a system in terms of parts and modules. A task card such as “install database”, or a story for “web client” risk distracting the team from the real desired outcomes the stakeholders care about. If you see any such stories (or tasks, issues, tickets or whatever you call them) try and find some end-to-end scenarios or use cases which require such things, and describe them instead. When that story is tackled, then the required infrastructure and components can be implemented as part of that story.

I have spent many years developing web applications in Java for corporate clients. During that time I have used a wide range of frameworks, APIs and other useful stuff. I have written my own versions (sometimes several) of many of these components, learned the tools well, and become very productive and effective.

However, it has been almost impossible to take that expertise, hard work, and custom code and easily/cheaply turn it into usable web applications for general public use. Low-cost hosting providers have generally shunned Java support. The most reasonable one I have found is LunarPages who actually support both ad-hoc JSP and deployment of custom web applications as war files, even though you have to search pretty deep in their web site to find out, and pay an extra dollar or two a month for the privilege.

So for my own projects I have been looking for an alternative for a long time. A way of developing and testing web apps on my various development boxes (currently running Ubuntu, MacOS X and Windows XP) and easily deploying to a low-cost hosting provider.

By far the most popular web development language is PHP. It’s available pretty much everywhere. It’s so focussed on web app development that the primary unit of coding is the web page. Believe me, I have tried to like PHP, but it’s just so clumsy. After a few paces the application begins to get tangled, development speed drops, and bugs creep in. Its web focus makes it tricky to unit test, and end-to-end testing seems to require a full-fat HTTP server. Not for me.

Beyond the traditional stomping ground of PHP, perl, and other CGI fodder is the new range of “trendy” languages. Ruby, Python, Erlang and even the venerable smalltalk are trying to position themselves as the thinking-person’s web development tool.

Erlang syntax is a bit too odd for me right now, although I may come back to it later. Smalltalk is interesting, but carries with it so much history and is hardly a popular choice for low-cost hosting providers. That leaves a short-list of Python and Ruby.

As languages I like both Python and Ruby. They have broadly similar design goals, both have keen developer communities with plenty of open source resources, and both are commonly found on linux-based web hosts. In order to decide between them I took a look around their frameworks and APIs.

Python has Google on its side. It’s the “native language” of the hugely-scalable Google AppEngine. However, a scan around the web looking for ways of writing web applications led largely to two: Zope and Django. (for balance, there are plenty of others, but these are the standout examples). Zope is old and sprawling. It has some great ideas but is hardly an obvious choice for small, tactical web apps. Django is a bit lighter, but still seems to assume a lot and require a lot of relatively fiddly config.

In the Ruby world the blindingly obvious choice for a web application framework is Rails. Sometimes it seems as if “Ruby” is just another way of saying “Rails”. However Rails, like Django, assumes a lot about the eventual application. Rails also makes a lot of use of code generation, which I simply do not like. I would always prefer that a framework eliminate boilerplate rather than just generate it for me. Looking a little further beyond Rails I came across some really interesting alternatives, and the one which really sparked my interest is Sinatra. For me this one framework made the difference. It’s so streamlined that a basic web app is as simple as:

require ‘sinatra’
get ‘/’ do
‘Hello world!’
end

Best of all, simply running the above file using ruby hello.rb starts up a web server and begins serving pages on port 4567. No extra config or faffing.

Looking around for low cost hosting I found that the provider I already use (Dreamhost) supports Ruby web hosting using Passenger. An ssh to the Dreamhost server for “gem install sinatra” followed by a little bit of FTP and my first Ruby/Sinatra application was live!

A key part of any web application is the pages, and a key part of generating pages is a good template language. As a general-purpose templating language, I still prefer my own Stringtree Templater, part of the Mojasef Java framework. So far I have not found a similar templating solution for any other language. However, for the limited and specific requirement of generating web pages, I am becoming quite fond of Haml. It’s not at all a general templating solution, but it does massively simplify the generation of web pages.

Finally, after years of CVS and Subversion, I have made the jump to distributed version control. Not only is it more trendy, but the ability to work on multiple code branches, on multiple machines, with or without an internet connection, and easily share, identify and merge when necessary has been a key benefit. Occasionally I wonder if one of the other distributed VCS (Mercurial, Bazaar, darcs, etc.) would have been a better choice, but I needed to settle on one and get to grips with it. So I chose git.

There have been a few speed-bumps. Attempting to learn git from manual entries and simplistic tutorials gives very little help on deciding what is worth doing. After about a week of following the techniques in A Git Workflow for Agile Teams, though, I began to get comfortable. Add to that some lovely web deployment tricks from A web-focused Git workflow and a selection of GUI tools ( Git Cola, GitX, and Git GUI ) to get round all that tedious “git add” stuff and simplify infrequent operations, and git becomes a very useful and productive tool.

The upshot is that I currently feel very productive.

My toolbox has a few other things in it, but they are still candidates for change if I find anything better. I am managing my projects and git repositories using Unfuddle but it has a few shortcomings including a ticket system which is a pale and clumsy cousin of bugzilla. For editing I use generic syntax-highlighting text editors (gedit on Linux, TextEdit on Windows and Smultron on Mac) but none of them have the nice autocompletion and refactoring tools I grew addicted to in Eclipse. Suggestions welcome!

Finally, a few spare links relating to the tools mentioned above.

Sinatra: 29 Links and Resources For A Quicker, Easier Way to Build Webapps

HAML reference

Rack

Mini reviews of 19 Ruby template engines

The Forgotten Ruby Web Frameworks

I have seen several attempts to implement an agile approach to software development within large organizations, and in many  (if not all) of those cases the end result has not been very compelling. This is in direct contrast to agile adoption in smaller, typically single-team, companies.

It’s natural enough to assume that problems with implementing agile processes in large organizations stem from some characteristic of the organizations themselves. Amr Elssamadisy, however, has a different suggestion. Perhaps some of the problems stem from the use of external agile “coaches”, and the way that they affect the way both the agile team and the organization operates simply by being there.

Read more at: InfoQ: Opinion: Agile Coaches Frequently a Source of Adoption Problems.

In tough economic times with a lot of people out of work and employers cutting back on training budgets few can afford the kind of big-budget, fancy hotel courses which were the staple of corporate training even just a short while ago. So Tobias Mayer has started a no-frills way to get “certified scrum master” status and spread the agile/scrum way of working. He calls it WelfareCSM.

The cost is essentially $50 (to the Scrum Alliance for the certificate) plus a voluntary contribution to cover any room costs etc. Sort out your own food and transport. Even for those of us who regard agile certification as a dubious concept this is pretty tempting.

Reading between the lines, though. It appears that the different agile “camps” may be slipping into a battle for mind-share. If Scrum certification becomes more widely available and understood, then alternative approaches (particularly XP, but also all the other adapted, customized and home-brewed agile working practices) may seem to have less value to employers and clients.

Hmm…

We are currently trying to come to some conclusions about the “shape” of a new software product, and facing a whole lot of problems. Stakeholders are happy to argue for hours about relative priorities of individual features, but so far these exist in a vacuum without an overall vision of a product.

With that in mind we are casting around for any hints on how to map out what we build and in what order. Chris Sims has some suggestions which may be useful, but I’m not sure they address our underlying problem.

InfoQ: Story Mapping Gives Context to User Stories.

Good, thought-provoking stuff. If agile is all about doing what’s necessary when it’s necessary, and not doing what’s not necessary, why should we wait for a retrospective to fix any problems?

Are retrospectives an antipattern? | Steven “Doc” List’s Random Musings.

An excellent point from Andy Palmer. Code which starts out too neat and ordered can make refactoring too difficult. Throw it all in and let the organization emerge from the functionality.

Andy Palmer » Code should start out messy.

Do you want to get certified in agile software development?

Fastest Agile Certification On The Web!.

The team at Energized Work are very upbeat about their process, and in particular how it helps generate real company income every week.

Much of their recipe will be familiar to most agile practitioners, but there are some good extra suggestions, too.

AGILE IN ACTION: ‘No excuses’ done done.

Wow, is Test-Driven Developement (TDD) really 10 tears old? Gojko Adzic is blogging the QCon London conference, and has a summary of the TDD lessons-learned session.

Gojko Adzic » QCon London 2009: What have we learned from 10 Years of TDD?.

Agile development relies for its speed and productivity on making only that which is really needed. Sometimes it can be easy to forget that performance and scalability can be needed too.

Alex Verkhovsky has an incisive post about how to deal with the notion of “premature optimization”.

Thread.current.to_s: The keyword in “premature optimization is evil” is “premature”

How minimal can an agile process really go? Brad Cross has described his lightweight process and it does seem pretty lightweight

On the Gnarl Side: Lightweight Lean Agile.

We currenty use a wall, covered with brown envelopes, for story and task tracking. It has its advantages but prople, particularly people not based in this office, often ask for something else.  Chris Sims at InfoQ has a useful summary of the pros and cons of high-tech and low-tech “information radiators”

InfoQ: Information Radiators: Is low tech really better?.

YAGNI – it’s a neat term for a valuable technique. Ignoring an unknown future to concentrate on a known present. That does not mean that it’s application is obvious, though. I often find myself in “discussions” with architects and designers who recoil at the idea of building something specific to one customer or situation, when it is “obvious” that a general solution will be better.

Mark Needham has written some of his thoughts on this topic.

YAGNI: Some thoughts at Mark Needham

I know this article was written back in October, but I can’t believe that nobody has pointed out its flaws. No Scott, measuring “acceleration” is not the same as measuring delivery of value!

IBM developerWorks : Blogs : Agility@Scale: Strategies for Scaling Agile Software Development

Scott Ambler, an author who I generally have a lot of time for seems to have completely lost it in an article at IBM developerworks. In this article he goes to some lengths to justify measuring teams on “acceleration” (in effect, measuring the proportional increase in story points processed per iteration) and thereby assessing their value. While this may have some merit, the article insists on contrasting it against other methods of attempting to measure actual productivity.

Unfortunately such a comparison does not make sense.

• A high-accelaration team may be worse over time than a low-acceleration team, if their starting productivity is less.
• A high-accelaration team may be worse over time than a low-acceleration team, if their maximum productivity is less.
• Ambler’s analysis assumes that the entire surrounding process is constant, including task estimation. This is very unlikely

Fundamentally Acceleration and speed are quite different things. I can wait at the lights on my pushbike and burn off nearby cars, because of my higher acceleration, but fifty feet later they will overtake me and carry much more stuff a much further distance.

Jason Yip from Thoughtworks has a nice summary of points to remember when organising or managing stand-up meetings.

You’d think with all my video game experience that I’d be more prepared for this: A quick summary of how I like do stand-ups