Django and the headaround fail…

Posted by Elf Sternberg as programming, python

Last year I briefly flirted with Ruby.  I once infamously described Python as “Perl for grown-ups,” and if that’s true, then Ruby is the hot 20-something firecracker chick of your programming mid-life crisis.  You’ve got nothing in common but damn the interfacing is great.  Maturity is getting over it and coming back to the stable reliability and maintainability of Python once more

But if Python is the language of the mature developer, than Django is that stage of midlife where life becomes way too complicated.  (Studies indicate that 44 is “the worst year of life” for most people: kids, family, career, finances, professional, social and romantic obligations all pile on hard around this time, leaving most people too little time to think or plan for happiness.)

I’ve been spending the day trying to port narrator, the program suite that runs my story site, from Ruby to Python.  Ought to be easy, right?  I decided to make it harder by porting it to Django instead.  I have the data models, and they’re great for Python.  Easy to port.  But I had to make it harder on myself, by trying to create containerized subsystems for my stuff: stories belong to authors, but authors might have series, series might have novels as well as stories, but there might be standalone novels, and you might have series so short and standalone that they go in your short story collection with an ident or something.  This immediately suggested to me a database trick I know called modified preorder tree traversal, a technique in which allows you to store hierarchical information in a meaningful manner.  There’s even a pre-build MPTT script for Django unmysteriously named django-mptt.

And that’s where I wandered off into the weeds.  I tried to understand how the mptt worked and how to incorporate it into the models I was building, and eventually my head started to explode.  It involved something called “generics,” which are a contributed library for Django’s ORM that uses metaprogramming to create one or more model classes with a many-to-many relationships to many objects in many different models.  It’s very cool.  It’s very esoteric.  It’s very hard to understand.  The layers between implementation and concrete realization are many and intertwined.

One of the differences between Rails and Django is that Django is “just” a bunch of Python libraries loosely assembled.  But Python encourages reading the source; that’s the idea, it’s supposed to be a readable, self-documenting language.  And sure enough, it is, if the problem domain is small enough.  It’s when you start to layer domain on top of domain, solution on top of solution, that the system become unwieldy.  The headaround for some Django applications is beyond the average programmer (and believe me, I am an average programmer).  Assaf Arkin encapsulated this idea perfectly in a recent post about object-oriented programming, quoting Travis Jensen:

“My point is that the architectural complexity of these applications inhibit a person’s understanding of the code. … Without actually doing anything, applications are becoming too complex to understand, build, and maintain.” Every layer makes perfect sense in isolation. The cracks start showing when you pile them up into a mega-architecture.

This seems to be the problem even with small Django applications. By presenting the four components of Django– the ORM, the Templating language, the Route/View library, and the Administrative envelope– as four separate components— The Django Project has greatly increased the cognitive load on the programmer.  Django becomes harder to learn than Rails because of the extra mental effort needed to grasp all the intricacies.

There is a perfect irony here: Django is loosely coupled enough that you can do whatever you want with it and it will probably work.  But understanding what crosses that loose coupling is difficult, and when you’ve got all the layers, plus contribs and plugins going, you need  to pull out pencil and paper for even the smallest of efforts.  Rails, in contrast, is a hodge-podge of different technologies all thrown together into a pot, but because the average web developer is actively discouraged from being curious about more than what he needs to get the job done, Rails is easier to grasp from the very start.

It’s probably worth it.  The payoff is a Deep Understanding of the joys of Python metaprogramming.  Done right, I’ll probably have an even better grasp of the abstractive power of it all.  But it’s a long, slow slog.

5 Responses to Django and the headaround fail…

Mike Altarriba

February 23rd, 2009 at 1:00 pm

Given what you’ve seen of Perl6, do you think you’ll change your position concerning the relative benefits of Perl vs. Python?

I’ve done a lot of Perl coding, and have more than once flirted with the idea of switching over. When I look around my office, I see about 20 books dedicated to Perl. When I look online, I see CPAN. The coding I do includes basic maintenance scripts and utilities, as well as exploring AI algorithms and writing code that interacts with the web and web pages.

What issues did you consider when you switched (assuming it’s fair to say you switched from Perl to Python). What issues should I consider?

In short, will the benefits of switching over outweigh what I lose?

Elf Sternberg

February 23rd, 2009 at 2:30 pm

I guess that’s a good question. I really have yet to see any details of Perl6 that make me think it’s going to surpass python in python’s key winning point: maintainability. (I’m actually a little concerned about Python 3.0, since it’s such a change that it might present a maintainability challenge.) I wrote a lot of Perl4 and Perl5 for my last long contract, and while I’m grateful that it helped put food on my table it wasn’t a language I learned to love.

Really, the question about Perl vs. Python is simply one of long-term maintainability. You can write maintainable code in both, but Perl doesn’t make it easy, while Python does I’ve seen people refer to Perl as a “write only language” and I generally think that’s true. (That said, I do have some perl scripts I wrote that I still use every day; I have seen no reason to port them to Python.)

The same functionality available through CPAN are available through the easy_install facility of Python (and, arguably, with less stress than firing up Perl -MCPAN -eshell to get them). Whether this is a “win” or not is a matter of taste. I mean, there’s a reason why Catalyst has neither popularity nor passion.

Rock Howard

February 23rd, 2009 at 2:31 pm

The django-mptt app hails from an early incarnation of Django before model inheritance. Try django-treebeard for a more straightforward implementation of the same concept (and it is faster as well.)

As for untangling Django, a simple diagram of the flow of control for an individual request winding through the process of being handled is sufficient to untangle almost all of the initial confusion about the major Django components. The remaining parts of the system involve deeper abstractions that most developers rarely encounter in day to day work but can learn about either by reading the source code and docs possibly supplemented by grinding through Marty Alchin’s excellent Pro Django book.

Andrew Ingram

February 23rd, 2009 at 5:22 pm

The rails ecosystem has a similar thing to generic relations in the form of the various acts_as_foo plugins, acts_as_taggable (for example) handles all the magical generic tagging voodoo in the background much generic relationships are handled in Django. I’m undecided on whether generic relations are worth enough to justify the fact that they’re effectively a black box of magic – but I’m happy that if I do have to use them there is a standardised way to do it that comes as part of Django.

Also, the generic relation functionality is part of contrib rather than core which suggests similar reservations amongst the Django developers as to whether it should be considered a key framework feature.

Elf Sternberg

February 23rd, 2009 at 7:03 pm

Rock: That may be, but I’m used to that deeper understanding, given that writing Python and Javascript all day long is what I do for $DAYJOB. But it’s an older framework (Webware) and I want to learn something new. Django is new and sexy.

Still, I am trying to wrap my head around everything. Thanks for the suggestion for treebeard.

Comment Form

Subscribe to Feed



February 2009
« Jan   Apr »