Frank Carver's Punch Barrel

Just when I thought private project hosting choices had settled down in favour of Unfuddle, along comes a potentially game-changing announcement from Assembla.

My big problem with Assembla has been that they equated one project (“space”) with one source repository, and charged extra for extra spaces. Each extra space gives extra issue tracking, documentation storage and so on. This might make sense in a traditional centralised subversion model, but it’s completely alien to git, where a project is commonly represented by a network of repositories.

As of 14 April, Assembla now offers unlimited repositories with each space, allowing a single project to share issue tracking, documentation, activity notifications etc. across multiple repositories. If you don’t need the rest of the features, they also now offer basic private source repositories for free, presumably with the idea of up-selling the other services.

Assembla seem to have slightly better tools than Unfuddle, but its hard to tell as they still don’t offer a real, private, non-expiring, “try before you buy” like Unfuddle does. With Assembla you get a month’s usage before you have to start paying, and their cheapest plan is $24 per month, considerably more than Unfuddle’s “micro” plan of $9 per month.

Both services are free for open-source projects, but then so are plenty of others (sourceforge, github, and so on.)

For the moment, I am still happy with Unfuddle, but I’m keeping an eye on Assembla.

I have spent many years developing web applications in Java for corporate clients. During that time I have used a wide range of frameworks, APIs and other useful stuff. I have written my own versions (sometimes several) of many of these components, learned the tools well, and become very productive and effective.

However, it has been almost impossible to take that expertise, hard work, and custom code and easily/cheaply turn it into usable web applications for general public use. Low-cost hosting providers have generally shunned Java support. The most reasonable one I have found is LunarPages who actually support both ad-hoc JSP and deployment of custom web applications as war files, even though you have to search pretty deep in their web site to find out, and pay an extra dollar or two a month for the privilege.

So for my own projects I have been looking for an alternative for a long time. A way of developing and testing web apps on my various development boxes (currently running Ubuntu, MacOS X and Windows XP) and easily deploying to a low-cost hosting provider.

By far the most popular web development language is PHP. It’s available pretty much everywhere. It’s so focussed on web app development that the primary unit of coding is the web page. Believe me, I have tried to like PHP, but it’s just so clumsy. After a few paces the application begins to get tangled, development speed drops, and bugs creep in. Its web focus makes it tricky to unit test, and end-to-end testing seems to require a full-fat HTTP server. Not for me.

Beyond the traditional stomping ground of PHP, perl, and other CGI fodder is the new range of “trendy” languages. Ruby, Python, Erlang and even the venerable smalltalk are trying to position themselves as the thinking-person’s web development tool.

Erlang syntax is a bit too odd for me right now, although I may come back to it later. Smalltalk is interesting, but carries with it so much history and is hardly a popular choice for low-cost hosting providers. That leaves a short-list of Python and Ruby.

As languages I like both Python and Ruby. They have broadly similar design goals, both have keen developer communities with plenty of open source resources, and both are commonly found on linux-based web hosts. In order to decide between them I took a look around their frameworks and APIs.

Python has Google on its side. It’s the “native language” of the hugely-scalable Google AppEngine. However, a scan around the web looking for ways of writing web applications led largely to two: Zope and Django. (for balance, there are plenty of others, but these are the standout examples). Zope is old and sprawling. It has some great ideas but is hardly an obvious choice for small, tactical web apps. Django is a bit lighter, but still seems to assume a lot and require a lot of relatively fiddly config.

In the Ruby world the blindingly obvious choice for a web application framework is Rails. Sometimes it seems as if “Ruby” is just another way of saying “Rails”. However Rails, like Django, assumes a lot about the eventual application. Rails also makes a lot of use of code generation, which I simply do not like. I would always prefer that a framework eliminate boilerplate rather than just generate it for me. Looking a little further beyond Rails I came across some really interesting alternatives, and the one which really sparked my interest is Sinatra. For me this one framework made the difference. It’s so streamlined that a basic web app is as simple as:

require ‘sinatra’
get ‘/’ do
‘Hello world!’
end

Best of all, simply running the above file using ruby hello.rb starts up a web server and begins serving pages on port 4567. No extra config or faffing.

Looking around for low cost hosting I found that the provider I already use (Dreamhost) supports Ruby web hosting using Passenger. An ssh to the Dreamhost server for “gem install sinatra” followed by a little bit of FTP and my first Ruby/Sinatra application was live!

A key part of any web application is the pages, and a key part of generating pages is a good template language. As a general-purpose templating language, I still prefer my own Stringtree Templater, part of the Mojasef Java framework. So far I have not found a similar templating solution for any other language. However, for the limited and specific requirement of generating web pages, I am becoming quite fond of Haml. It’s not at all a general templating solution, but it does massively simplify the generation of web pages.

Finally, after years of CVS and Subversion, I have made the jump to distributed version control. Not only is it more trendy, but the ability to work on multiple code branches, on multiple machines, with or without an internet connection, and easily share, identify and merge when necessary has been a key benefit. Occasionally I wonder if one of the other distributed VCS (Mercurial, Bazaar, darcs, etc.) would have been a better choice, but I needed to settle on one and get to grips with it. So I chose git.

There have been a few speed-bumps. Attempting to learn git from manual entries and simplistic tutorials gives very little help on deciding what is worth doing. After about a week of following the techniques in A Git Workflow for Agile Teams, though, I began to get comfortable. Add to that some lovely web deployment tricks from A web-focused Git workflow and a selection of GUI tools ( Git Cola, GitX, and Git GUI ) to get round all that tedious “git add” stuff and simplify infrequent operations, and git becomes a very useful and productive tool.

The upshot is that I currently feel very productive.

My toolbox has a few other things in it, but they are still candidates for change if I find anything better. I am managing my projects and git repositories using Unfuddle but it has a few shortcomings including a ticket system which is a pale and clumsy cousin of bugzilla. For editing I use generic syntax-highlighting text editors (gedit on Linux, TextEdit on Windows and Smultron on Mac) but none of them have the nice autocompletion and refactoring tools I grew addicted to in Eclipse. Suggestions welcome!

Finally, a few spare links relating to the tools mentioned above.

Sinatra: 29 Links and Resources For A Quicker, Easier Way to Build Webapps

HAML reference

Rack

Mini reviews of 19 Ruby template engines

The Forgotten Ruby Web Frameworks

I’m currently spending a fair amount of time on evaluating distributed version control systems (DVCS) such as git, bazaar and mercurial. Some things which were easy using a centralized version control system such as subversion or cvs seem more complicated, but I am now starting to find ways of using DVCS to do things which the old systems simply could not manage.

It took me a whole evening of fiddling, but I have now set up git to allow me to make chanches to a local version of one of my web sites and commit changes as I feel like it, then when I am happy, send the changes to the live web site with a single git push. This has been working well for a few days, so I plan to roll out a similar structure to the rest of my web sites soon.

Another interesting use for DVCS was pointed out by Francisco Trindade. Distributed version control can help to enable “set based design“. This is a technique where a team samples a wide range of possibilities to converge on a “sweet spot” where market need, business value and manufacturing efficiency are balanced.

As you may recall from a previous post, I am looking for some decent project hosting both for my open source software and for some business ideas. I want a single point of contact which offers as a minimum: version control, wiki, bug/ticket tracking, calendar and tasks/todo. Nice to haves include time-tracking, collaborative planning and continuous integration.

Eventually I have found two likely candidates: unfuddle and assembla. Both offer most of what I want, but with a few differences.

Unfuddle is probably the simpler of the two. It offers subversion and git hosting, a kind of wiki called “Notebook”, a kind of low-rent bulletin board called “Messages” for discussions, deadline management using simple text milestones, and bugs/tickets using Trac. It also provides RSS and iCal feeds of workspace events and upcoming milestones. With increased monthly payment you can also get time tracking and file attachment storage for messages.

Assembla offers subversion, mercurial and git hosting (and can integrate with remote subversion and github repositories), a wiki with extensions to directly reference tickets, bugs, software versions etc., similar “messages” to unfuddle but with file attachment storage included, bugs/tickets using trac or a proprietary alternative, similar milestones to unfuddle (but I can’t find any iCal feeds which is a shame), time recording, scrum-style progress reporting, and a few other options such as a “chat” facility and a specialist repository for storing and annotating images. It can provide updates of events using twitter or by HTTP call-outs, which seems pretty flexible.

Neither one seems to have a very sophisticated calendar, so no arranging meetings etc. Neither one has any significant collaborative planning in the vein of Mingle or Silver Catalyst. Neither supports recording of anything other than time (money spent on each task would be very useful, for example.) And neither supports continuous integration as such (although it could probably be hacked together using Assembla’s HTTP call-outs.)

Pricing seems roughly similar, although calculated differently.

Unfuddle offers five price bands (free for open source, $9, $24, $49, and $99 per month). Each band offers larger quantities of storage, numbers of participants and projects etc. The $9/month plan offers 512MB of storage for 10 people on 4 projects, so for an example small private team of three developers the cost would be $9/month

Assembla determines pricing per user per “space” (a space seems roughly to equate to a single project). Free for public spaces. For private ones, each user/space is $2/month, and $3 per gigabyte of storage per month. So for the same small private team of three developers the cost would also be $9/month but for more features and more storage. The down side is that adding any new team member, even one who only needs occasional access, costs extra.

I have registered for free accounts on both systems and have started to try out everything I can. I’ll report back soon on my findings.

In the meanwhile I’d love to hear from anyone with any other suggestions for project hosting services along the lines of these two.

I’m still searching for decent project hosting. I now have several projects on the go, and several others bumping around in my head, and the fuss and bother of tying together all the various bits of a distributed software project development is making my head hurt.

All the bits I need are available separately, but so far I have not been able to find any single provider (free or paid for) which offers the combination of features I need. Essentially these are:

• Version control. Ideally git, but at a pinch one of the other distributed VCS tools or even subversion would probably do if everything else was in place. GitHub seems good for this.
• A project wiki. Using any other system for project docs just seems so clumsy. There are plenty of these; I use WikiDot for one project.
• Sensible bug/feature tracking. This is a bit more tricky – there is plenty of bug-tracker software, but not much that works equally well for managing unimplemented feature stories and associated tasks. Ideally this should link in with the version control, allowing code and change metadata to be updated in one go. Trac seems a possibility for this.
• Calendar management. For recording and communicating meetings, deadlines etc.. Something which works well with calendar syndication, so that anyone working on the project can see project events in with the rest of their appointments. Plenty of these: Google calendar, 30 boxes, etc. They all have their quirks, though.
• Task (todo) management. I find it amazing that task management is so poor in on-line calendars. There are standalone task tools such as Remember The Milk, but it is integration which is needed.

There are also a few other features which are definitely in the “useful to have” category, but I’m practical enough to use manual or off-line tools if necessary.

• Effort recording, tracking and reporting. For velocity tracking, process improvement, and even billing.
• Collaborative planning and prioritisation. Mingle tries to simulate a task wall, but is somewhat clumsy and irritatingly expensive; I have heard of on-line tools to run “Planning Poker” sessions, but as usual, not integrated with anything else.
• Continuous Integration. I’m not aware of any really smart tools to make use of distributed version control for this, yet. Our Cruise Control installation just stops and complains when something breaks, for example, but it should be possible to just “park” the failing patch and continue building with others in a real dvcs-based approach.

If anyone has any suggestions – or wants to build a product which does all this stuff – please let me know!

For interest, here are a few associated links.

Cuberick: Distribute Your Software Just Like Ubuntu With Launchpad

Comparison of open source software hosting facilities: Wikipedia

I’m tired, and I’m cross. Last night I was up too late, and this morning I have wasted another few hours. All trying to do something which should be simple – evaluate some products from the well-known software consultancy “Thoughtworks”. And so far I still have not succeeded.

It all started so simple. I was looking around for a bundled software solution for running a new software project and found “Buildix 2.1“. This looked to be a large part of what I need, so I investigated further. Subversion for storing source code, Cruise Control for continuous integration, Trac for change tracking and wiki collaboration. All familiar, useful, stuff. And it also offers “Mingle“, a tool I have not used before, for agile project planning. A quick check shows buildix is offered under an Apache licence. Best of all, they offer it as a pre-built virtual machine image, so I should be able to just download it and run it alongside my existing VMs and give it a good evaluation.

And that’s where it all began to go wrong.

I downloaded it and extracted the files from the archive, started my VMware server console and gave it the details of the VM image. First problem – the image somehow requires a later version of VMware, forcing me to upgrade. I have installed a bunch of different vm images before and never had to do this, so why can’t Thoughtworks manage to build a compatible image?

So I went through all the pain of an upgrade to VMware server 2 (complete with the sudden appearance of a login box which was never needed on the old vm manager, and required me to create a new Windows admin user!) but eventually I got to a stage where the new vm image would at least load into the hypervisor.

Then I hit the bit which astonishes me the most. The supplied VM image is broken and won’t even start. Worst of all, this is a known problem. Did they not even test this most basic of steps?

So, I followed the instructions and logged in to issue a “fsck” command to get it working a bit more. Which in turn required me to go to the web again to find the default root password (if it helps anyone else, it is also “root”).

After a while it seemed to have stopped initialising, so I found the dhcp address which had been assigned to the vm, and tried an HTTP request to port 80. Timeout. Using my new found root login skillz, I hit Alt-F2 on the VM console, logged in as root and entered “ifconfig”, the response did not include an ethernet connection, only a loopback. My knee-jerk response in this situation is to enter “ifup eth0″, and in this case it seemed to work. It is probably not a long-term solution, though, and may not survive a reboot. For that I may need to mess with some config files.

After all this effort, I just about have a running system. Connecting to the same ip address with a web browser now gives a redirect to https, which in turn causes Firefox to complain about a self-signed certificate, so I plod past that. Eventually I see a page which asks me to configure Mingle, so I enter a bunch of smtp and user details and suggests that I enter the details of the new user in the buildix config (with a handy link). Unfortunately, when I click that link I find that it requires a login first. I try the details from the user I have just created, with no luck. Back to the internet I go.

It turns out that the username and password I need are “buildix” and “buildix” respectively. After a bit of bouncing around pre-populated login fields, I get to the config page and enter the details.

Now I have run out of what little instruction there was. On the Buildix web site the navigation link labelled “Documentation” takes you to a breathtakingly dismissive page which just states:

Documentation

The forums can be found here.

No sign of even the simplest tutorial or walk-through to give a hint of what you can do with the system. No links to documentation for any of the component applications. No FAQ for any of the show-stopping problems which I have tripped over so far. Unbelievable.

Left to my own devices, I try clicking some of the links, starting with “create project” but all I get is a cryptic error message. The suggested solution from the forums is to find and edit yet another config file, which allows me to create a project, but leaves me without Mingle. Somewhere in the application I find some text suggesting I need to register Mingle, with a handy link to Thoughtworks Studios. When I click the link I see nothing about registering, so I wander around the site for a while, and eventually find a page where I can buy licences. I’m not interested in paying USD 318.60 per user, especially when the software has needed so much TLC so far to even get it running. Luckily, the page states

Remember that the first five users are free. Thus, if your team has 15 people, you should key in only 10 in the “Qty.” field below.

I don’t need more than 5 users, so I enter 0 in the quantity field and click “proceed to checkout”. The purchase system then informs me that “You do not have any items in your cart.”. Another needless failure. Fed up with trying to get a licence I return to the Buildix application and poke around a bit more. Re-reading the config page I see an un-checked checkbox under Mingle Options labelled “Enabled”. So I enable it and continue. Magically, a Mingle Icon appears next to the others for my test project. Filled with excitement I click it, only to see

We apologize for the inconvenience.

You should be able to continue work by returning to the projects page or you may prefer to sign out and sign in again.

So I sign out and in, using the admin user I created so long ago, which eventually logs me in to an unregistered Mingle, but with no projects. So I create one with the same name as the one I created using the “a little bit of our own ThoughtWorks magic” which was obviously not quite as magic as it should be.

Eventually it seems that I have achieved a running Buildix 2.1 Virtual machine. But I’m exhausted, and sick of all the nonsense I had to go through to get here, so I’m hardly enthusiastic about evaluating it and telling all my colleagues how wonderful it’s components (and by extension ThoughtWorks’ coding skills) are.

Seriously ThoughtWorks. Do you have any idea how poorly this sort of thing reflects on your products and your company?

Plenty of people seem to be able to put together working systems of this complexity in their spare time, and to get it right. If this is really the best you can manage, why would anyone consider hiring your team for anything important?

We all love source control, right?

Source Control Mastery – The Daily WTF (recommended by Ernest Friedman-Hill)

I couldn’t live without it any more. So much so that the “gateway drugs” of CVS and Subversion are not enough any more. I keep being tempted to move on to stronger stuff, like Git and Bazaar …

I’m intrigued by this. A hosted subversion service, which seems to be positioning itself alongside similar services for other protocols (e.g. GitHub). Right now I’m having trouble working out whether there are any significant extra features beyond what I get already with my account at Dreamhost .

Beanstalk — Version Control with a Human Face

Like many other developers I was keen to try out the recent Eclipse Ganymede release. In particular I wanted to try the new subversion integration (“subversive”) to see if it is any better than the old (“subclipse”) plugin I used to use with Europa.

Getting the new subversion working was surprisingly tricky. Obvious choices simply do not work, and the error messages do not give quite enough information to solve the problem. Add to this that some of the names of locations and updates can easily be misconstrued and you have a recipe for confusion.

For example, the error message presented if you try to use the subversive plugin without installing any third-party connectors is the only-partially-helpful:

I think the Eclipse folks have potentially missed something important here – checking out a project or three from subversion is almost always the way I start when I want to evaluate an IDE.

Anyway, here are a set of steps which should result in a working Eclipse Ganymede with the new subversion:

Get the basic subversion integration plugin from the eclipse download site

1. Select Help::Software Updates from the menu bar
2. Unfold Ganymede::Collaboration Tools
3. Select SVN Team Provider (Incubation)
4. Click install… and follow the instructions, including a restart of Eclipse

Resist the temptation to try and use the plugin at this point. It won’t work and you will get the above confusing message about connectors.

Get the third party SVN connectors from a different update site

1. Select Help::Software Updates from the menu bar
2. Click Add Site…
3. Enter http://www.polarion.org/projects/subversive/download/eclipse/2.0/update-site/ and Click OK
4. Unfold the new site :: Subversive SVN Connectors
5. Select appropriate connectors. In my case I selected Subversive SVN Connectors, SVNKit 1.1.7, and SVNKit 1.2.0
6. Click install… and follow the instructions, including a restart of Eclipse

You should now be able to either Window::Show View::Other…::SVN Repositories or Window::Open Perspective::Other…::SVN Repository Exploring and add repositories to your hearts’s content.

Please let me know if your process differs from this, or if you have any corrections or suggestions.

Regular readers (are there any?) should have guessed by now that I am investigating distributed version control (specifically git). I also have a significant investment of code and history in subversion repositories. A way to migrate from svn to git, keeping all the commit histories sounds great…

Simplistic Complexity » Cleanly Migrate Your Subversion Repository To a GIT Repository

Also An introduction to git-svn for Subversion/SVK users and deserters or Git Guides – Import from Subversion

Recently I decided that it was time to stop just reading about distributed version control and give it a go. I had already run through a variety of git tutorials, but all of them concentrated on use of local repositories and/or starting with a clone of an existing remote repository.

My main need is something slightly different, though. I want to set up my own remote repositories for my large and rambling collection of projects, so that I can access them wherever and whenever I am working.

I was tempted to use Github but decided that the learning process of doing it myself would be valuable. Eventually, I think it was, but I still might go to Github for some of my projects.

My first step was to install Git on my main development machines. On the Ubuntu box it was easy using the Synaptic package manager. On a Windows box it’s not quite so straightforward, involving a choice between a Git based on Cygwin and a Git based on direct use of Windows libraries. I eventually chose the second option, but I might change my mind later if it doesn’t work out. For the moment it does the job.

My next step was to set up the remote repository. I decided to host my repositories on some space I rent from Dreamhost, as they seem generally pretty supportive of new stuff, and already include subversion hosting.

I started by following the steps in the DreamHost Wiki entry on Git, but could not make that work. Something went wrong at the step of pushing a local repository over WebDAV. Determined to continue, I instead tried the (somewhat more complex) steps at Autopragmatic » Blog Archive » Hosting a git repository on dreamhost. Even though this involved rebuilding a slightly non-standard version of Git on the server this now works very well.

I now have a small green-field project in a remote repository at Dreamhost. My next plan is to investigate using git-svn or some other form of git/subversion bridge.

Nice summary of the new features in Subversion 1.5.

Mike Mason » Subversion 1.5 is Ready

.. and it’s much more useful than the simplistic and argumentative coverage of the same launch by InfoQ