A subtle django performance hit

Wow – I’m really glad I learned about this performance hit on the Django list today.

Turns out that in the process of attempting to give back from really nice error messages from the default AddManipulator’s in Django 0.95, it has a HUGE performance impact that is documented, but man – I would’a easily missed it. The default value for “raw_id_admin” on a ForeignKey model in Django is False. Turns out the implications of that are that when you invoke an AddManipulator for an object with a ForeignKey relationship somewhere, it attempts to load up all potential values into memory and check against them… Yuck!

I really couldn’t believe this, so I went home and using the models from that mail thread, I created a test app and ran some time. Here’s what I got:

Yeah – that’s something like near linear growth of the time it takes to add a new object with the number of potential relationships.

Update: I posted the test case I created as bug #2638

6 thoughts on “A subtle django performance hit

  1. Just to clarify: the values aren’t loaded “to check against”, they’re loaded because the default behavior in the generated form is to display a select box listing the possible values. Which, of course, means you have to load all those values into memory. I agree that the docs could be better in pointing that out, but I’m not sure about changing the default value of raw_id_admin to True.

    Like

  2. I love the functionality (even if I didn’t quick grok it’s implications) – but I really think it should be “opt-in” since it has such profound performance implications, not “opt-out” to get performance.

    Like

  3. This post is a bit misleading to someone who doesn’t know Django already. You don’t have use the AddManipulator to create new objects – it is a shortcut for manipulating data posted to a form. If your form has a drop down box for the relationship, then it will need to load all these objects to display the list. If you don’t want a drop down box, then use a custom manipulator.

    Like

  4. Thanks for the comment Luke – absolutely correct, but I disagree. The documentation and tutorials all lead someone new to Django to use the AddManipulators (and ChangeManipulators) – and rightfully so. They’re great tools that do a HUGE amount of work on your behalf. I love using them, and I think that the default behavior should be different than it is.

    You can also create objects directly – and in those cases you’re back to doing all the form validation and verification type activities yourself. You can loose quite a bit when doing it manually – especially when that data is coming from a web form.

    Like

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s