Nearly at the top of that first hill

I’ve been thinking about the past week at the OpenStack Design Summit (Bexar) solidly from last night (flying home from San Antonio, TX) through the various errands I’ve been running today. This morning Rick Clark tweeted “A question about OpenStack”. As I think about it, this shouldn’t be about what is going right and wrong, but where the project is and what will provide the most benefit by improving it.

I’m saying all this after a week with the OpenStack guys – both in design sessions and just chillin’ out. Focused, intelligent, demanding conversations scattered through the week with an amazing “no-ego” attitude presenting itself. Not that there weren’t some good ole technical “best way to do it” or “which is better” fights, but given the breadth of this project and the open nature with vendors lurking all around the corners – well, frankly I expected a lot more “special interest” to be clearly showing itself. Everyone at that conference was interested in making OpenStack better at every turn.

250 people, 12 countries, 90 companies/organizations – all that after 3 months from being publicly announced. And they’re going it without any prior structure – building up an OpenStack foundation, doing all the legal and community building, right from scratch. And yeah – that’s showing right now.

The first thing I see that will provide the biggest gains:

• “How do we all work together?”

Some of the best sessions were around “What does the status X of bugs mean” and talking through the development and release process. At this point I’m convinced the core folks are reasonably comfortable with LaunchPad (the platform the system is hosted on) – and being at the conference really taught me a great deal about how OpenStack is effectively using it. Prior, it wasn’t comfortable or familiar to me at all. The object store and compute (swift and nova, respectively) core groups are really quite separate teams, all trying to figure out how to get some common ground in re-using code, libraries, and even setting up documentation.

The second:

• “Show me it’s workin’, again and again”

OpenStack is quickly heading to be the kernel or core of a platform. You could see it in the twinkle of Eucalyptus’ eye when they talked about Swift (the object store), or chatting with the folks from Scalr or RightScale. The whole system is being built with API in mind from the ground up, and while there is some pretty good unit testing in play and continuous integration, it was clear that installing this sucker was a PITA – and the documentation to really pull that all together starting coming together in the documentation sprint and install fest at the summit. One of the “blueprints” of the design summit (i.e. “Things we want to do, and how we want to do it for the next release”) is to get some fully automated integration testing as well as track the metrics on how the system is operating. There were a lot of folks that have some cross over into the Drizzle project, and the ideas of running and tracking benchmark data on every revision is darned power.

Add to that the benefits of a constant flow of functional testing against a couple of pre-defined clusters of both compute and object store, and you have a powerful engine to make sure trouble is spotted early and can be resolved quickly.

The third:

• “How’s this thing tick?”

One of the admitted weak points is that some small, damned effective core teams have done most of the work – and if you want to understand the system, well… you’ve just got to read the code. That is a huge investment – and frankly a barrier to entry into the project that can be avoided with some effort towards docs and discussion. Again, great progress was made there (I learned what the “project” concept was in Nova at the summit) – but the interactions between components, what the components are responsible for, and what they’re *not* responsible for, are all kind of tricky to learn right now.

This extends down into digging into the code, where docstrings could be better (and are getting better!) so that if you wanted to go help with something specific, you didn’t have to grok a broad codebase to get a handle on what the impacts are of the changes you’ll need to make.

And the last thing I’ll throw in here:

• “What OpenStack isn’t, or won’t do…”

The project is still in a lot of flux. There were some great components that were shown off at the summit that ride over the top of the infrastructure, or work with it through APIs. Should those be a part of OpenStack, or on the side? Some service providers were very interested in more platform-kind of elements – a common logging infrastructure, a common authentication, ID, and authorization infrastructure. Should that be a part, or on the side? How tightly or loosely do we want to couple some of these elements? The philosophy is there and forming up, but the real truth of it all will be over the next 6-12 months of the project when decisions are made, reviewed, and a core forms out of it. There have been a few architectural decisions made early: “Don’t mandate anything in the client”, “If a feature would restrict scale, it MUST be optional”, etc. that I absolutely applaud. I think it will form up more as projects apply to join the OpenStack umbrella and either make it or don’t. It will become clear what’s common, and what isn’t, pretty darn quickly.

I’m pumped about this project, the people, and it’s future. The core openstacker’s have clearly been driving up a steep hill to get to where they can level out a bit and move into more of a marathon mode. Really, it feels like we’re nearly at the top of that first hill.