Frank Carver’s Punch Barrel

This looks like an interesting project. I’m slightly worried by the way that it seems to embody the one class === one test assumption, but if that doesn’t get in the way of other forms of unit testing it could be useful.

moreUnit - Home

I’m currently working on some software which sends notifications to users (using SMS, email, or whatever) and have faced the inevitable problems with testing it. On balance I’d prefer not to receive a test SMS on my mobile phone every time our continuous integration system runs an end-to-end test.

Gojko Adzic has some thoughts about how to make such a system more straightforward to test.

Gojko Adzic » How to test e-mail notifications properly

I develop almost all my code these days using Test Driven Development (TDD). By taking this approach I never bother about unit test code coverage, but some developers seem very concerned by it.

One thing that I have never considered doing is retroactively adding tests just to increase some code-coverage metric. Dan Manges has written in interesting explanation of why that might be, using some examples in Ruby.

Dan Manges’s Blog - Increasing Code Coverage May Be Harmful

As we attempt to spread agile, continual, iterative, processes throughout the company, there is a growing confusion as to when things are actually ready for release. The old “waterfall” notion of a complex and detailed advance specification, backed up by months of laborious manual testing is no longer applicable.

In most ways this is a very good thing. A whole lot of unnecessary rigidity and busy-work has been removed. However, one aspect we miss is some sort of final “quality gate” to approve a release. We are in danger of ending up either with a queue of potential releases jostling for testing priority, with none of them ever actually being released before they are overtaken by a new candidate, or with untested and potentially broken builds being released to live deployments.

This is obviously also on other people’s minds, too. InfoQ has recently published an article on “The Power of Done” which summarises some attempts to chart an agile course through this dangerous situation.

InfoQ: The Power of Done

As software testing spreads out in scope from the old notion of manual exercising of a system into areas such as developer unit tests and automated acceptance tests, the issue of ownership becomes more important. In this short and pithy post Kristan Vingrys states his opinion.

Vinktank | Who owns automated acceptance tests?

I’ll put my cards on the table. At heart I am a “classic TDD” guy. My first attempt at any new code is to test return values or side-effects rather than immediately jump to setting up mock objects and testing interactions. Although I do use mock objects as a testing technique from time to time I have never liked any of the popular mock frameworks. The nearest I have found to what I would like is probably Mockito, but even so I still use my own recording framework.

thekua.com@work » The world mocks too much

This slightly provocative post examines the role of QA (a.k.a testing) in various view of a project lifecycle and considers how we might react to a QA team which reported no bugs.

Ryan Greenhall: What is The Role of QA?

A fascinating counterpoint to Gojko Adzic’s writings on acceptance testing in an agile process.

thekua.com@work » Automated story-based acceptance tests lead to unmaintainable systems

Update: here’s some more discussion on this topic, and how it is affected by the nature of user stories

User Stories are Just Schedulable Change

An article with some good tips on avoiding a clash of cultures between developers and testers on an agile project. (note, there may be a click-through ad on this link)

Software testing on an agile project: How to get started

We are currently in the middle of trying to improve a whole collection of process issues where I work. We have already put in place a fixed iteration cycle in development, but are having difficulties fitting post-development testing (a.k.a “system testing” or “integration”) into the mix.

The main problem with testing after development is that any fixes to problems identified in the follow-on testing have a tendency to become tangled in upcoming work for the next iteration. This in turn can lead to no iteration ever being “clean” enough to actually release.

One possibility for a solution is to reconsider the development-led nature of an iteration, and instead lead it from a set of acceptance tests which are estimated, developed and executed alongside the scheduling and implementation of code changes.

Gojko Adzic, one of my current favourite bloggers on the subject of testing, has attempted to map out an iteration structure which incorporates this approach:

Gojko Adzic » Fitting agile acceptance testing into the development process

This looks interesting. The role of testing (at least, testing beyond unit-level regression testing and TDD) is the subject of much discussion where I work at the moment. Maybe I should actually attend this talk at Skills Matter in London on 18 September.

In The Brain of Gojko Adzic: Agile Acceptance Testing

When I first saw this it looked great: a ruby REST wrapper which supports a lot of useful test and integration possibilities. However, the deeper I looked, the more disappointed I became. I’m now saddened to believe that this is based on yet another misunderstanding of what REST is.

As far as I can tell, the ActiveResource concept, on which this approach is based, is merely an attempt to impose a constrained CRUD data-model over HTTP. There is no concept of content negotiation - all data is XML according to pre-assumed schemas. There is no support for automatic discovery and use of hrefs between resources. There is even the suggestion that it’s a good idea to casually extend the HTTP protocol with extra custom methods.

All of these are typical problems found in projects which use the name REST without the key bits which make it really work. The end result is just another fragile, application-specific RPC protocol. Sigh.

nutrun » Blog Archive » Testing web services with ActiveResource

I just found this link in my drafts. Still worth looking at if you are interested in a holistic view of agile development.

InfoQ: Integrating Testers on to the Agile Team