A subtle django performance hit

Posted byheckjPosted indjango, Geekstuff

Wow – I’m really glad I learned about this performance hit on the Django list today.

Turns out that in the process of attempting to give back from really nice error messages from the default AddManipulator’s in Django 0.95, it has a HUGE performance impact that is documented, but man – I would’a easily missed it. The default value for “raw_id_admin” on a ForeignKey model in Django is False. Turns out the implications of that are that when you invoke an AddManipulator for an object with a ForeignKey relationship somewhere, it attempts to load up all potential values into memory and check against them… Yuck!

I really couldn’t believe this, so I went home and using the models from that mail thread, I created a test app and ran some time. Here’s what I got:

Yeah – that’s something like near linear growth of the time it takes to add a new object with the number of potential relationships.

Update: I posted the test case I created as bug #2638

Published by heckj

Developer, author, and life-long student. Writes online at https://rhonabwy.com/.

6 thoughts on “A subtle django performance hit”

  1. Just to clarify: the values aren’t loaded “to check against”, they’re loaded because the default behavior in the generated form is to display a select box listing the possible values. Which, of course, means you have to load all those values into memory. I agree that the docs could be better in pointing that out, but I’m not sure about changing the default value of raw_id_admin to True.

    Like

  2. I love the functionality (even if I didn’t quick grok it’s implications) – but I really think it should be “opt-in” since it has such profound performance implications, not “opt-out” to get performance.

    Like

  3. This post is a bit misleading to someone who doesn’t know Django already. You don’t have use the AddManipulator to create new objects – it is a shortcut for manipulating data posted to a form. If your form has a drop down box for the relationship, then it will need to load all these objects to display the list. If you don’t want a drop down box, then use a custom manipulator.

    Like

  4. Thanks for the comment Luke – absolutely correct, but I disagree. The documentation and tutorials all lead someone new to Django to use the AddManipulators (and ChangeManipulators) – and rightfully so. They’re great tools that do a HUGE amount of work on your behalf. I love using them, and I think that the default behavior should be different than it is.

    You can also create objects directly – and in those cases you’re back to doing all the form validation and verification type activities yourself. You can loose quite a bit when doing it manually – especially when that data is coming from a web form.

    Like

  5. A dropdown is much more friendly than a text box in the default case. When you need the performance, you can disable it. If you have thousands of records, chances are you are not going to want a select box anyway. Another solution might be to use AJAX:
    http://code.djangoproject.com/wiki/AJAXWidgetComboBox

    BTW, you should ask to have your django feed added to the Django community page.

    Like

Comments are closed.

%d bloggers like this: