Archive: May, 2010

Hudson and VMBuilder

May 31st, 2010 · Soren · Code, Ubuntu 2 comments - Tags: ,

Unhappy with the current state of VMBuilder, I recently decided to take a look at Hudson, hoping it can help improve quality going forward. Hudson is a “continuous integration” tool. This means that it’s a tool you use to apply quality control continuously rather than only either when you’re feeling bored or when a release is imminent.

I’ve set up Hudson with a number of jobs:

  • One monitors the the VMBuilder trunk bzr branch. Whenever something changes there, it downloads it, runs pylint on it, runs the unit tests (pylint and unit tests setup with help from a blog post by Joe Heck), and rolls a tarball. Finally it triggers the next job..
  • ..which builds an Ubuntu source package out of it, and triggers the next job..
  • ..which signs and uploads it to the VMBuilder PPA that I recently blogged about..
  • Last, but certainly not least, I’ve set up the very first completely automated, end to end VMBuilder test. It grabs the freshest tarball from Hudson, copies it to a reasonably beefy server, builds a VM, boots it up and upon succesful boot, it reports back that it all worked, and Hudson is happy. It doesn’t exercise all the various plugins of VMBuilder (not even close), but it’s a start!

VMBuilder in Lucid == lots of fail

May 27th, 2010 · Soren · Code, Ubuntu 7 comments - Tags: ,

Let it be no secret that I’m unhappy with the state of VMBuilder in Lucid (and in general for that matter). Way too many regressions crept in and I didn’t have time to fix them all. I still expect to do an SRU for all of this, but every time I try to attack the bureaucracy involved in this, I fail. I need to find a few consecutive hours to throw at this very soon.

Anyways, in an effort to make testing easier, I’ve set up a PPA for VMBuilder.

I’ve set up a cloud server that monitors the VMBuilder trunk bzr branch. If there’s been a new commit, it rolls a tarball, builds a source package out of it, and uploads it to that ppa. That way, adventurous users can grab packages from there and test things out before they go into an SRU. To do this, you simply run this command:

sudo add-apt-repository ppa:vmbuilder/daily

I’m also working on a setup that will automatically test these packages. The idea is to fire up another cloud server, make it install a fresh VMBuilder from that ppa, perform a bunch of tests and report back. To do this, I’m injecting an upstart job into the instance that

  1. adds the ppa,
  2. installs vmbuilder,
  3. builds a VM, which (using the firstboot option) will call back into the host when it has booted succesfully,
  4. sets up a listener waiting for this callback,
  5. waits for set amount of time for this callback.

If I get a response in a timely manner, I assume all is well. If not, it’ll notify me somehow.

The idea is to make it run a whole bunch of builds to attempt to exercise as much of the code base as possible.

I’ll try to make a habit of blogging about the progress on this as I know a lot of people are aggravated by the current state of affairs and this way, they can see that something is happening.

Cloud computing – Same old song?

May 3rd, 2010 · Soren · Cloud computing 6 comments - Tags:

I recently ended up in a conversation with a guy who turned out also to work in IT. When I mentioned I worked on cloud computing, he started talking about how it was just the same old song. Before I had a chance to reply, we were interrupted, but I haven’t really been able to push this aside, and I’d like to address this point of view, as it’s probably held by others as well.

He said that he found cloud computing to be “old wine in new bottles”. His arguments were almost exclusively about how outsourcing is a bad idea. The rest of the time he spent pointing out that for all the time he’d had an Amazon S3 account (I think he said 2-3 years) he hadn’t noticed a price reduction in spite of the price of self-hosted storage is ever decreasing.

Cloud computing certainly shares some characteristics with outsourcing. You are running services on someone else’s hardware, in their infrastucture, leaving a big chunk of responsibility with this provider. This is also true for cloud computing. It’s also true that you’re paying a premium for the hardware compared to what it would have cost if you had it in your own data center. The difference between CAPEX and OPEX seemed to be lost on him, along with the fact that you’re also freeing human ressources to work on more interesting things, but none of this is really the point.

Apart from sharing the benefits (and drawbacks!) of outsourcing, cloud computing offers a new level and type of dynamism and availability. If you’re just going to take your Exchange server (his example) or whatnot and put it on a statically allocated cloud server, then yes, it’s the same old outsourcing song. If you, however, design your service so that it can scale horizontally, the dynamism of cloud computing will let you scale both up and down to address changes in demand. This way you save money when your service is idling, yet you can scale up quickly to respond to rising demand. More ressources are (supposedly) always available and right at your fingertips. They’re a simple API call away. Leveraged properly, it’s very likely that you could not only save money running the same service in the cloud, but also be able to deal with fluctuations in service demand much better than you could in your own data center or in an old school outsourcing scenario.

As for his other point, about the prices never decreasing in spite of the cost of hosting these things yourself decreases over time.. That’s a good point. He thought that that was how the these providers were really expecting to make money. I wouldn’t go that far at all, though. What makes cloud computing a viable business is by and large the economy of scale. Hosting lots and lots and lots of virtual servers or petabyte upon petabyte of data is lots cheaper /per unit/ than hosting a few servers and a few terabytes of data, but I have to agree that it does seem that the price per GB of stored data should be decreasing over time in response to the decreasing cost of storage on the market.

Linux2Go is Digg proof thanks to caching by WP Super Cache