I took some time this afternoon to benchmark out the work we’ve been doing with the Django Queue Service. We haven’t done anything to optimize it, and we’re starting to push in some code to make it fully support REST verbs (as opposed to its current REST/RPC mixed style). Seemed like a good time to get some numbers down.
The results are all from my laptop – 2GHz Intel Core Duo, 2GB RAM. Running python 2.5 and the current trunk of Django (0.97-pre-SVN-6668).
The test ran through the timing for adding messages into the queue and getting/deleting messages from the queue, scaling the queue size from 0 to 25,000 items. The client that ran these tests was put together with httplib2, release 0.40 – running in python on the same machine. Yeah – so pretty much factor out network latency on this one…
The adding stayed a fairly consistent 8 ms, deviating around a bit – but mostly there.
The combination of getting a message and deleting it (in our API, the functions are separate – so it means two calls back to the service) rose linearly with the size of the database. At a base level, it was running about 20 ms and by the time the database size was at 25,000 items the “get and delete” operations were taking roughly 165 ms.
For SQLite-on-disk database backend, I’m pretty pleased with those results.
Rhonabwy 
This looks great! Thanks. I’d been hoping someone would do something like this. Just FYI…your link to the Google Code site is presently broken…it looks to be trying to generate a relative rather than absolute URL. I just hacked off the unneeded portions and found the site fine, though.
Looking forward to trying this out.
LikeLike
Thanks Ken! You called the link error exactly – I missed the ole http:// stuff at the front of it. Link updated.
LikeLike
Have you not published any code yet? I was actually hunting for it a couple of weeks ago and still can’t find anything. I believe I can make use of this.
LikeLike
The code is all out there – just not as a downloadable ZIP or tarball. You’ll need to use subversion to check it out from Google. The quick and dirty to get the application onto your dev instance would be the following command:
svn checkout http://django-queue-service.googlecode.com/svn/trunk/qs/ qs
If you’re really feeling brave, you can dig into our unstable branch and see what we’re doing with the pure REST verbs. The benchmarking above was against the unstable branch from this weekend.
LikeLike
Found it soon after asking of course. Thanks.
LikeLike