We are currently struggling with how to integrate work on refactoring/simplifying/cleaning our product codebase with existing streams of stories and bugs. One of the tricky aspects of this is how to estimate and prioritise the cleanup work: how much is it worth, and how much time should we spend on it this iteration?
Martin Fowler has written about extending the idea of “technical debt” by including the concept of “interest”. Extra time spent on completing any given task compared with the time which would/might have been spent implementing that outcome on a cleaner system is effectively an “interest payment” on the “technical debt”. Noting an estimate of such an amount along with the elapsed time on each piece of work allows a relatively simple calculation of the total “interest payments”, and helps inform any decision to “pay off” the debt by refactoring, simplification, etc.
MF Bliki: EstimatedInterest
Planning what order to do stuff in is a vital, yet very difficult, part of software development. Agile wisdom usually stresses the need to do things in order of “business value”, but this can sometimes be extremely tricky to evaluate.
Another approach is to do things in an order intended to decrease risk, such as starting with the most worrying, or least understood parts of a system.
InfoQ: Iterating To Acquire Knowledge, Not Just ‘Business Value’
While this sounds plausible, I am not sure I entirely agree the reasoning. Behind the idea of ordering things by risk seems to be the idea that there is some bounded set of requirements, known early enough in the process to be able to determine their “worry”. To me the whole point of agile development is to step outside that assumption into the real world where work needs to start before full scope is defined, where priorities change on a whim, and development on a product may be cancelled at any time.
Starting work by spending a few iterations investigating complex/risky parts of some imagined future product instead of delivering usable but minimal versions from the first iteration seems somewhat irresponsible.
I started to idly watch this video presentation, but quickly found that it describes a situation eerily close to our own, so I have decided to set some time aside to watch it in proper detail. I hope it pays off!
InfoQ: Planning with a Large Distributed Team
A lot of writings about agile processes seem to assume that everyone comes with all the skills they need, but in the real world people sometimes need to gain new skills which cannot easily be learned “on the job”. Planning ahead enough to ensure team skills are available when needed, and dealing with the impact on development speed of people spending time on training or independent study rather than productive work seem like tricky problems.
Jooli Atkins has written a bit on this topic for the British Computer Society (BCS) Agile Training : Blogs : BCS
I’d be really interested to hear from anyone how they deal with this!
We don’t use scrum as such, but we still face the issue of team-member absence during fixed iterations. It’s a tricky one to deal with, especially as absence comes in both predictable and unpredictable forms.
InfoQ has a nice summary of some of these issues and some solutions. There is till no “silver bullet” for this, though. Losing staff will almost always mean losing productivity, which in turn means that iteration (”sprint”) commitments will need to be renegotiated, which works best if stakeholders are informed as soon as possible.
InfoQ: Handling Absence in Scrum Teams
Mike Cohn raises an interesting and topical issue about how to indicate what you are working on at a daily stand-up meeting.
In our project we are currently working on an iteration where all the stories overlap and interact. Our usual approach is that each developer grabs a story and works on it until its done, then repeats until either the iteration ends or we run out of scheduled stories. In this case we need something a little smarter - some way of grouping the tasks for the iteration, regardless of which stories they “belong” to, so that developers can work on a chunk of related functionality rather than a story as such.
This is potentially risky, as some of the commenters on the above post point out, but at the moment I can’t see a way around the problem.
Should the Daily Standup Be Person-by-Person or Story-by-Story? | Mike Cohn’s Blog - Succeeding With Agile™
From time to time I toy with the idea of writing a book. One of the things which has put me off is the whole “big project” nature of the task. I am so used to test-driven, small-iteration, early-value working that slogging away for six months or a year on a single task seems particularly daunting.
An interesting possible solution to this is the idea of writing in an agile manner.
InfoQ: Can Authors Use Agile Methods?
This story really resonated with me. We often suffer the same problems described in the article - frequent interruptions and panic priorities, continual flip-flopping of overall direction and dissatisfaction among stakeholders.
The solution we adopted was to choose fixed three-week iterations, with an iteration planning meeting (theoretically including all the stakeholders) just before the development start of each iteration to select a set of tasks totalling a specific number of “jelly beans” (a.k.a ideal engineering days.)
This approach has certainly solved some of the problems, and highlighted others. Interestingly the current iteration is different. Several key members of the team are out of the office this week, so we chose to run a single one-week iteration with a clear goal for a single stakeholder rather than take a productivity hit on the start of a three week one. This iteration has a quite different feel.
perhaps we should also consider reducing iteration length, to see what happens!
InfoQ: Presentation: Extremely Short Iterations as a Catalyst for Effective Prioritization of Work
It must be that end-to-end software production process is on my mind. I keep spotting interesting articles about the topic. Here’s one from Digital Dim Sum which delves into the way that requirements find their way into development, contrasting a “push” of requirements with a “pull” of information..
Don’t push requirements - pull information | Digital Dim Sum
Where I work I think we are using a more “pull”-centred process than many I have seen. Anyone is free to write and contribute “user stories” at any time. From time to time ones which seem valuable are pulled into a planning poker game for estimation. Before each iteration of work, a mixed group of stakeholders gets together to pull a fixed amount of development effort from the pool of available estimated stories into the next iteration schedule. Developers then pull stories from those scheduled for the iteration, work on them and place them in a pool of implemented stories. When the integration test team are ready, they pull a build containing stories implemented so far and test it. Meanwhile the customer deployment team may pull and deploy any tested build at any time.
A short article about planning and organising a project using post-its and/or a spreadsheet to associate tasks with phases. The interesting thing about this article for me is the explicit mention that the process is cyclic and iterative, rather than linear. Tasks are expected to move around the phases in a variety of directions, and special “evaluation” phases are provided as stopping points between other parts of the process.
Planning
This morning our team had our first real go at “planning poker”. We had not got any real planning poker cards, so we used a few regular packs. In general it seemed to work.
The web has various recommendations about what the best selection of cards for each person to hold might be. In our case we went with 0 (King), less than one (Queen), 1 (Ace), 2, 3, 7, and Lots/Inestimable/Split (Jack). This selection was certainly useful, but I’m still not sure whether it was the best choice. There was some discussion of the need for more higher numbers, and a significant proportion of the stories we estimated converged onto a result of 3, which may be a hint about some of the psychological aspects of the game.
One of the biggest benefits was the quick and easy way to vote a story as too large or otherwise in need of splitting. In the end I think roughly 20-25% of the stories we tackled ended up in the “split” pile, with one of them being split and re-estimated during the game and the rest passed back for further discussion with the originating stakeholders.
Which brings me to the major problem we are facing. Our stakeholders are not often available for direct questioning, and getting any of them into a planning session such as this has proved very difficult. For this attempt we took the approach of clarifying as much as we could within the team, but placing aside for more discussion any stories which seemed completely un-estimable. M