Open Source and Other Things
Last week we (the dev team of Memset) released a new version of different open source projects which objective is to provide a convenient file system-alike interface for OpenStack Object Storage (codename Swift).
As you may know, Swift is a object storage and not a file system or real-time data storage system. To explain it short: you can store, retrieve and delete objects over a RESTful HTTP API, but not all the operations that you would expect on a real file system are available.
Swift works very well as a long-term storage or when you need to store static data, and its API is compatible with Rackspace Cloud Files, so you have several applications that can communicate with it out of the box (or even with explicit support, for example with Cyberduck and OpenStack Object Storage protocol).
When we started working in a product based on Swift (to be released soon, stay tuned!), we found that the pseudo-hierarchical directories support on Swift has lots of possibilities, and… wouldn’t it be great to use any of your favourite file transfer applications directly on Swift?
Nick Craig-Wood (Memset’s Technical Director) started looking for any open source doing that, and we found the project of Chmouel Boudjnah: ftp-cloudfs. We were more interested in a SFTP interface, but Chomuel approach was very nice (MIT licensed), so we decided to put some work on his code, writing a fake file system interface layer to make some specific applications work over FTP.
After some love (other people call this QA), ftp-cloudfs has reached version 0.9, and we believe it’s in good shape to be used in a real-world™ environment.
At the same time we started to work in a SFTP implementation of the same idea, using Paramiko to deal with the protocol details, and thanks to reusing some code from ftp-cloudfs, we promptly released sftp-cloudfs as open source.
We’re planning to refactorize the code and move out the file system layer to a different project (right now sftp-cloudfs depends on ftp-cloudfs, which is not optimal), but as today I think we have reached an important milestone in both projects.
With this release we also added the packages to PyPi to make as easy as possible to start playing with the daemons, including our most crazy project to provide a full file system interface through FUSE: pycloudfuse.
So if you want to give them a go you can download the code, or try with pip:
$ pip install ftp-cloudfs
and for the SFTP code:
$ pip install sftp-cloudfs
Note: pycloudfuse mostly works, but it’s not in the same good shape than the other two projects and it needs more love (again, talking about QA here). Any help is welcome!
Tomorrow I’ll be attending OggCamp 11, in Farnham. See you there!
Update: look out! I’ve uploaded photos!
MemSet 1 Year Birthday OSCON from OpenStack on Vimeo.
It could be better, but it could be worse too! Anyway I think we made a couple of interesting points, didn’t we?
Today at the OSCON11 the Open Cloud Initiative (OCI) was launched as a non-porofit organization to advocate standards-based Open Cloud Computing.
From the official announcement:
The Open Cloud Initiative (OCI) has launched its official website at http://www.opencloudinitiative.org/ and commenced a 30-day final comment period on the Open Cloud Principles (OCP), which are designed to ensure user freedoms without impeding the ability of providers to do business. They are focused on interoperability, avoiding barriers to entry or exit, ensuring technological neutrality and forbidding discrimination. They define the specific requirements for Open Standards and mandate their use for formats and interfaces, calling for “multiple full, faithful and interoperable implementations”, at least one of which being Open Source.
This is a big step forward in the direction that the Franklin Street Statement pointed out three years ago. Check current status of the Open Cloud Principles (OCP), and then look for similarities with the Franklin Street Statement.
This is really interesting and I’m looking forward to the development of the initiative.
