Infrastructure (python)

I’m taking some time today to read up on the events and talks at PyCon 2009 happening this past week in Chicago. I’ve been watching a few folks on twitter who are there (John DeRosa and Ted Leung). It’s been interesting to see what’s happening through their comments and links. There’s the inevitable talks that I wished I’d attended, hopefully slide decks will appear for a good number of those and I can catch up even after the conference content. PyCon is one of the best there – they go out of their way to make the talk content available as quickly as possible with the right set of controls for content (the speaker’s are the ones responsible for uploading their content).

Anyway, all this led me around to the conversations that have been happening around packaging with python. It was a topic of some conversation (and has been needed for a while now, in my opinion) at the Python Language summit (notes from Ted about it). Tarek Ziade did a survey recently of what folks were using, and posted his results, which were interesting. zc.buildout and virtualenv got some attention from multiple folks. I think the most crucial was the summit talks themselves, which Tarek summarized on the mailing list. The gist – there is a high-level plan at improving the packaging world around Python:

• standardize more metadata, especially including dependencies, and APIs for accessing the metadata and dependencies.
• there should be an API to get metadata for a package without actually executing any of the package’s installation script.
• this will go into the stdlib for Python 2.7 and 3.1
• it will be provided as separately downloadable backports for prior versions (maybe using a bootstrap not unlike ez_install)
• keep distutils, but start deprecating certain higher-level functionality in it (e.g. bdist_rpm)
• don’t try to provide higher-level functionality in the stdlib, but instead let third party tools built on top of these core APIs compete

When I was working more extensively with Django, that was the biggest headache – getting a packaged, ready to roll quickly environment. I’ve trailed off in the past while with the Django project (not actively using it at the moment, although I think that’ll likely switch around again) – but the lessons about packaging have remained with me.