Experimenting with VMware CloudFoundry

Yesterday evening I went along to the Ipswich Ruby User Group, where Dan Higham gave an enthusiastic presentation about VMware CloudFoundry. The product looked interesting enough (and appropriate enough to my current project) that I decided to spend a few hours evaluating it. On the whole I’m impressed.

After poking around the web site a bit I decided to download the “micro cloud foundry” version, which does not need dedicated hardware, as it runs as a virtual machine on a development box. Once I had passed the first hurdle of requiring registration before even starting a download, then waited for about 40 minutes for the VM image to download, I had a go at running it up.

Now, I must say that I went a bit “off road” at this point. Naturally enough the web site recommends only VMWare virtual containers for the task, but the last time I used VMWare player (admittedly a few years ago) I found it clumsy and intrusive, and did not want to clutter my relatively clean development box. I already have virtualbox installed, and use it every day, so I thought I’d see how well the image runs in this container. Starting the VM was just a matter of telling virtualbox about the disk image, allocating some memory (I have a fair amount on this machine, so I gave it 4G to start with) and kicking it off.

Initially, everything seemed great. The VM started OK, and presented a series of menus pretty much as described in the “getting started guide”. I had been warned that setting it up might take a while, so I was not worried that it twiddled around for 15 minutes or so before telling me that it was ready. The next step according to the guide was to go to the system where the application source code is developed and enter vmc target http://api.YOURMICROCLOUDNAME.cloudfoundry.me (the micro cloud foundry VM has no command line, it’s all remotely administered) to connect the management client to the VM. This was a bit of a gotcha, as the “vmc” command needs a separate installation, found elsewhere on the site. Essentially “vmc” is a ruby tool, installed as a gem. In this case I already had ruby installed (it’s a ruby project I’m working on), so it was just a matter of sudo gem install vmc. Once I had installed vmc, I tried to use it to set the target as suggested, but the request just got rejected a somewhat confusing error message. On the surface it did not appear to be a network issue – I could happily “ping” the pseudo-domain, but vmc would not connect. After some looking around, both on the web, and digging in the “advanced” menus of the micro cloud foundry VM, I eventually realized that the error message in question was not actually coming from the micro cloud foundry at all but from a server running on my development system!

To make sense of why, I need to describe a bit about my development set up. The basic hardware is a generic Dell dektop with its supplied Windows 7 64-bit OS. I don’t particularly like using Windows for development (and did not want to wipe the machine, because I also need to use Windows-only software for tasks such as video and audio production), so I do all my development on one of a selection of virtual Ubuntu machines running on virtualbox. This is great in so many ways. I have VM images optimised for different work profiles, and run them from a SSD for speed. Best of all, I can save the virtual machine state (all my running servers, open windows and whatnot) when I stop work, and even eject the SSD and take it to another machine to carry on if needs be.

So, the problem I was seeing was due an interaction between the “clever” way that micro cloud foundry sets up a global dynamic dns for the VM, and the default virtualbox network settings. To cut a long story short, both my development VM and the micro cloud foundry VM were running in the same virtualbox, and both using the default “NAT” setting for the network adapter. Somewhat oddly, virtualbox gives all its VM images the same IP-address, and all the incoming packets were going to the development VM. More poking around the web, and I found that a solution is to set up two network adapters in virtualbox for the micro cloud foundry. Set the first one to “bridge” mode, so it gets a sensible IP address and can receive its own incoming packets, and set the second one to NAT, so it can make requests out to the internet. I left the development VM with just a “NAT” connector, and it seems happy to connect to both the web and to the micro cloud foundry VM via the dynamic dns lookup.

Of course, it was not all plain sailing from there, though. The first issue was that I kepy getting VCAP ROUTER: 404 - DESTINATION NOT FOUND as a reponse. A message that was obviously coming from somewhere in cloud foundry, but gave no obvious hint what was wrong. After a lot of trying stuff and searching VMware support, FAQ and Stack Overflow, I came to the conclusion that this is largely an intermittent problem. After a while things just seemed to work better. My guess is that when the micro cloud foundry VM first starts it tries to load dependencies and apply updates in the background. This is probably a quick process inside VMware’s own network, but out here at the end of a wet bit of string, things take a while to download. Eventually, though, things settled down and I was able to deploy some of the simple examples. Hooray! I have subsequently found that the micro cloud foundry VM needs a few tens of minutes to settle down in a similar way every time it is started from cold. Good job I can pause the virtual machine in virtualbox.

The process for deploying (and re-deploying) applications which use supported languages and frameworks is largely smooth and pleasant. It does not use version control (like Heroku, for example) but a specific set of tools which deploy from a checked-out workspace. If you want to deploy direct from VCS, it’s easy enough to attach a a little deploy script to a hook, though.

Once I got past paying with the the examples, I tried to deploy one of my own apps. It’s written in Ruby, uses Sinatra as a web framework and Bundler for dependency management, so it should be supported. But it does not work at all on cloud foundry. It works fine when I run “rackup” on my development box, and it works fine when I deploy it to Dreamhost, but on cloud foundry – nothing. Now, I can understand that there may be all sorts of reasons why it might not work (the apparent lack of a file system on the cloud foundry deployment, for one), but my big problem is that I have so far not discovered any way of finding what is actually wrong. An HTTP request just gives “not found” (no specific errors, stack traces or anything useful). Typing vmc logs MYAPP correctly shows the bundled gems being loaded, and the incoming HTTP requests reaching WEBrick, but no errors or other diagnostic output. I can only assume that the auto-configuration for a Sinatra app has not worked for my app, but there seems to be no way of finding out why.

To me, this lack of debuggability is the single biggest problem with cloud foundry. I hope it is just that I have not found out how to do it. If there is really no way at all of finding out what is going on on the virtual server we are back to “suck it and see” guesswork, which is so bad as to be unusable. I am simply not willing to spend hours (days? weeks?) changing random bits of my code and re-deploying to see if anything works.

If anyone reading this knows a way to find out what cloud foundry expects from a Sinatra app, and how to get it to tell me what is going on, please let me know. If not, I may have to abandon using cloud foundry for this project, and that would be a real shame.

Comments (4) left to “Experimenting with VMware CloudFoundry”

  1. Andy Piper wrote:

    Hi Frank – so you’re able to run your Sinatra app locally using “ruby app.rb” (or similar?) – does vmc tell you that it has detected a Sinatra app when you are pushing to your cloudfoundry instance? Out of interest, have you tried pushing to cloudfoundry.com, or only to the local micro CF instance?

    Our usual support route at the moment is via Stack Overflow, FWIW – I found this post via a web search. I’ll point Dan over here as well, since you were inspired to check out CF by his talk! Really hope we can help you to move forward and apologies that you’ve hit this issue.

  2. Frank wrote:

    Thanks Andy.

    I’ve done a small amount more experimentation, and I am now thinking that the problem might be that CF has a slightly different view of what it means to be a Sinatra app than I do. I usually run my app locally using “rackup” or “shotgun”, and I have tried it on one of my Dreamhost sites which uses phusion passenger. The key difference seems to be that I have a bunch of stuff in config.ru (preloading common gems, attaching middleware, mapping url prefixes to Sinatra sub-apps and so on)

    CF seems to think that the way to start a Sinatra app is to pick the first ruby file it finds which declares that it uses Sinatra and just run that file. From that viewpoint my app is probably better characterised as a Rack app rather than a “pure” Sinatra app. In this case, however, the MCF auto-detection spots a reference to Sinatra and assumes its own model without bothering to look for a config.ru file. I seem to recall that one of the systems Dan demonstrated did support starting of rack apps, but I’m not yet sure if that feature is available (even if hidden) in the one I have.

    All is not lost, though, as I’m going to try some sort of work-round replacing config.ru with a ruby file that does get recognized by CF. Maybe my app is odd in using Sinatra under the control of config.ru (and Rack), but I have found plenty of examples on the web which are odd in the same way ;)

    It is a bit of a shame that CF is not quite smart enough to even recognize the problem, though. Do you guys often run rack apps using CF?

  3. Monica Wilkinson wrote:

    Hi Frank
    Good catch. This has been fixed on cloud foundry but we have not shipped a new micro cloud foundry for a few months.
    May I suggest you not use micro cloud foundry and instead use cloudfoundry.com itself ?

    Then when you push an app with a config.ru you should see:

    Would you like to deploy from the current directory? [Yn]: Y
    Application Name: test23232
    Detected a Rack Application, is this correct? [Yn]:

    On Cloud Foundry.com you can pick from these:

    1: WSGI
    2: Node
    3: Spring
    4: Rack
    5: Django
    6: Standalone
    7: JavaWeb
    8: Sinatra
    9: Grails
    10: Erlang/OTP Rebar
    11: PHP
    12: Lift
    13: Rails

  4. Dan Higham wrote:

    Hi Frank,

    If you want some guidance on this, I am more than happy to help out. I found you on Skype and have sent a request. Or just send me an email and I will endeavour to help and work through any issues you might have.

    Thanks

    Dan

Post a Comment

*Required
*Required (Never published)