6 months later… Essex

Easter Sunday and I’ve some time to sit back, relax, and spend the weekend doing a little cleaning and catching up. I’ve got to admit to some embarrassment that the last post here was the wrap-up of the OpenStack Diablo design summit.

To say it’s been a busy six months since the last post is really quite an understatement. What we released in the past month, and the work that went in to getting us there, is nothing short of phenomenal.

The OpenStack Essex release is out, and I’m still sort of catching my breath from that. Just six withs prior to the release I was elected as the PTL for Keystone. I had no idea how much additional work it was to wrangle the bugs, approvals, and the other sundry details that go into getting the release out the door. I rather wish there was a “new PTL user’s manual” for the process. It was a lot of learning, most of it last-minute and ad-hoc around our project management, release process and associated the mechanics. I spent a lot of time trying to figure out how to wrangle Launchpad into some semblance of useful tracking, etc. (To be fair, it was mostly a matter of learning what Launchpad could and couldn’t do, and figuring out the conventions that were already in place).

The past month was terrifically busy with getting OpenStack deployed and tested – running in small environments (devstack), and large. I’ve been working from the Ubuntu distributions myself, and the whole gang that has been packaging OpenStack (Chuck, Adam, Kiall, and others – inside Canonical and out) with the downstream distributions has done a tremendous job wrapping together the changes and making it into a deployable release from packages.

A much better release:

Where I was, er, more than disappointed with the Diablo, I’m significantly happier with the Essex release. Keystone (the project for which I am now the PTL) didn’t advance this release cycle so much as retrench and prepare for advancement this next release cycle. Horizon made huge leaps and bounds, which I think is fantastic as the face of OpenStack to many users. Nova and Swift advanced, and the integration work and definition going into making Quantum (and Melange) a reality has been terrific.

I think if I were to pull out a star of the show for the Essex release, then I’ve really got to point to the combined work for the crew formulating and keeping DevStack solid, and the CI team that integrated it into our development and review process so that we had a minimum of guaranteed interoperability. Where I was screaming into the wind during the last milestone of Diablo for continued breakages between the components, the integration of devstack has demanded that changes be rolled in with though to an overall use case and interop. With 200+ developers all kicking this ball down the field, the guaranteed CI has been the piece that has done us the most good.

A little about Keystone:

Keystone, as we retrenched and simplified the codebase, also has a significant advancement in integration testing. We replaced the entire codebase based on a series of integration tests that verified and guaranteed API compatibility with the Diablo and trunk releases as well as the client while we made that switch. The underlying code base is now significantly simplified, and the internal architecture of that service now wrapped around some core “internal service” concepts to allow us to have drivers that cleanly back-end into external systems to support identity and authorization.

I’ve received a number of questions about “why the API still sucks” six months later after Diablo. There’s still no obvious means for a user to “log out” (invalidate their own token), or change their own password (assuming the back-end were to support it). In switching out the underlying code base and architecture we needed to keep the API stable so that we could make sure we had the internals correct. With the internals switched out, it is now time to revisit the API and take the lessons we’ve learned from 6+ months of using the v2.0 API and improve upon it.

Looking towards the summit:

While I’m much, much happier with the Essex release, there are still plenty of places for improvement – many that I remember from the Diablo summit, and some new things that I’d love to see some focus on for the upcoming six months.

While Horizon has given us a dramatically improved user experience for OpenStack, there’s much more that we could do there – both with a web based UI, and with our command line interfaces to OpenStack. One thing that could use some explicit attention is the proliferation of “clients” needed to interact with OpenStack. As projects are splintering off Nova into their own domains, we have a number of new command line clients (nova, keystone, glance, quantum) – and they don’t all act the same. There is some great work on driving them to consistency, but I wonder if we shouldn’t bag all the individual clients and roll them together into one “OpenStack” client that is consistent in how it handles command line options, what the “commands” look like, and generally how you interact with them.

I hate to admit this, but even as the PTL of Keystone, I ran into a brick wall of old docs referring to using the command of “nova-manage project list” to see the projects, when I knew darn well and good they were all in the keystone system, and to see a list of them I should use the command “keystone tenant-list“. Not so hot, huh? Quantum and the nova-network components are going to really come into their own in this release, so we’ll have more shattering of the user experience from a deployer’s point of view unless we do something to be specific about tying it together.

There are more changes to be made in Keystone that I’m paying a lot of attention to – some of which are honestly going to require buy-in from the entire community to make happen. Where we place the boundaries of role based access, and how we deal with trust and information sharing about identity between the projects is probably going to need some changes that will ripple all the way down into the API’s of nova, swift, glance, quantum, melange, etc. Getting the brainstorming around those concepts and desired features is what I primarily want to accomplish at the Design Summit, with follow on in the milestone timeframe thereafter to bring the new features into existence and integrate them across OpenStack. My ideal goal is to get all the heavy features implemented early in the Folsom release cycle so that they’re solid and nailed down for the later milestones, and we can tweak and fix what may be broken long before any release points.

About that CloudStack thing:

Seems like in the last week, everybody (or at least the pundits) got their knickers in a twist about Citrix finally being more open about their dual-interest in CloudStack and OpenStack. Regardless of Gartner pundit sensationalistic babbling, the release of CloudStack as an Apache licensed project is nothing but goodness for the whole community. The Apache license makes me feel more comfortable with reviewing their code and looking at how they attacked the same issues and problems OpenStack has, and I expect they’ll look closely at how OpenStack has done the same. I applaud any company trying to build an open-source community, and I’m looking forward to seeing what Citrix does to actually do that community building. If it gains ground and folks contribute to CloudStack, we’re all better off – it’s more ideas hitting the street and becoming reality.

Finally, for anyone that didn’t think that the user-facing EC2 API was the defacto API, wake the F* up. It is reasonable, solid, and nobody is going to be turning it off any time soon. It also doesn’t allow/enable some things that a lot of cloud administrators would like to do – so it shouldn’t surprise anyone that CloudStack has their “api” and OpenStack has their “api”. The APIs are another place where we can learn from each other, as we add some value beyond what the bookseller down street might want to impose and expose.

If you want a little pundit action from this source, watch the API’s of both CloudStack and OpenStack (not just the EC2 compatibility layer) over the next six months. I think you’ll be seeing some very interesting steps forward.