18 Oct 2011 mattl   » (Journeyer)

Spotlight: Merlin Cloud

First, let's dispel this myth about cloud computing. What is it, and why is it compatible with free software ethics?

"Cloud computing," unfortunately, is one of the most misunderstood terms in the recent history of computing. Literally, it has a million different definitions to a million different people. I like to think of cloud computing simply as an abstraction: a system that provides methods for requesting and utilizing computing resources without having to know anything about the underlying systems providing them.

Primarily, cloud computing is about (a) aggregating server, network, and storage resources into a seemingly contiguous system ("the cloud"), (b) providing some kind of interface for the user to request or release these resources, and (c) making these resources network or location agnostic, so that the resources are accessible from anywhere, even in the face of system or network failures.

I agree that there are many ethical considerations with "the cloud," especially in its current invocation by larger corporations. However, their cloud need not be ours; it is becoming more and more possible to use free software to build clouds that respect free software ethics and don't require a user to compromise on their freedoms or privacy.

For instance, one could imagine a GNU cloud which only ran free software, and had strong privacy and data protection safeguards, but provided users the same type of experience they might expect from Amazon's EC2. This kind of vision is entirely possible.

Okay, so what's Merlin? Give me the 30-second overview...

Primarily, Merlin is a Ruby on Rails application that I created as a way of interacting and controlling EC2 API-compatible clouds. This can mean Amazon AWS, which has many significant ethical concerns, but it can also mean something like Eucalyptus/UEC (the GPLv3-licensed version), or other free software which provides similar services.

Merlin allows you to quickly request the creation of new virtual machines and storage volumes, and can also assist in provisioning them (getting them ready to do real work). It handles a lot of details in the background (such as Puppet certificate signing, BIND dynamic DNS updates, etc.) and aims to make day-to-day cloud operations as easy as a couple of clicks. It also aims to provide a free software option versus the proprietary software or closed web services that are used to interface and interact with clouds today.

What technologies are used to make Merlin work?

As I said, Merlin is a Ruby on Rails project. I actually used the project as an excuse to learn Ruby/Rails, as I had only used Django or Zope in the past. Underneath the hood, it uses ActiveMQ to pass messages between the front end and the back end, and of course a whole basket of Ruby gems. Merlin also assumes that you are using Puppet to provision your systems.

What can developers who are interested in Merlin do to help it succeed?

Merlin, primarily, needs more attention. It works for me, in my environment; it would be helpful if others would try using it. It really needs better packaging and documentation: it isn't as simple as installing it and clicking a few things. It also needs quite a bit of work on its UI, and I am hoping to get it to a beta release where a lot of the basic functionality will be a bit more implemented than it is now. In the future, Merlin will support spawning entire groups of machines, or entire self-contained systems, within the cloud. I would also like Merlin to be API-agnostic, and support a wide variety of cloud software. All that being said, the software is available today, so anyone who wants to hack on it can do so.

We've seen some rather gloomy-looking options from big companies who are talking up the cloud as the solution to all our problems. What can Merlin do to give people some flexibility without requiring them to give up their control?

Primarily, the real issue with the pervasiveness of clouds with ethical issues for users is simply that they are there, and they are the only real option unless you are looking very carefully. The average user isn't yet aware of the traps and pitfalls involved in using cloud services, and is very much attracted to the ease of use and simplicity that the cloud promises -- especially the idea of their files or data being available "everywhere." Unfortunately, this simplicity comes with a price -- it is inherently linked to an expropriation of freedom and privacy which becomes harder and harder to reverse the more one relies on the corporate clouds.

To my knowledge, no one has attempted to build a large, scalable free software cloud that inherently respects the freedom and privacy of its users. I sincerely hope Merlin can be helpful in this regard, even if it is only to encourage communities of users to build their own small private clouds instead of always using the big companies.

How do you use Merlin?

We use it internally to provision new virtual machines within our private cloud. Merlin will then provision these instances and even register them automatically in DNS. It makes the rapid creation of large numbers of provisioned virtual machines particularly quick and easy, which I would say is a general, if unwritten, expectation of cloud computing.

Syndicated 2011-10-18 19:31:39 from fsf

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!