12Nov

Continual Integration Testing for Web Applications: A Sample Application in Django

Posted by Elf Sternberg as django, programming, python, web development

This is Part 2 of a series.

Before I demonstrate how to do continual integration testing, I need a demonstration application. I’ve chosen a simple Django application, your basic echo program, with no styling or media at all. This ought to be more than enough to demonstrate base functionality.

A New Django Project

Start by building an application. Well, start by installing Django (easy_install django), but after that, start a new Django project:

django-admin.py startproject echo

This will build a complete skeleton of a Django project for you. There’s only one thing you have to change to make it work, too. Go into the “echo” directory that now exists, and edit settings.py. You’re looking to change two lines.

DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'echo.db'

Now, if you start Django,

./manage.py runserver

… you’ll get a “Welcome to Django!” message if you browse to http://localhost:8000.

A Test-Driven Development

Django comes with a pretty nice testing harness, and for the first pass that’s what I’m going to use. I’m going to enhance that harness by using a pair of external Python tools, Beautiful Soup and Coverage. Beautiful Soup is a very forgiving HTML parser, and it’s helpful for making sure that web application views have the content I expect. Coverage is a testing tool that tells you what lines of code were or were not exercised by a test suite.

My intent is to create a simple echo server: you type something in, and it collects what you type and spews it back out at you, one line at a time, recording every line you’ve input. To make things “fun,” I’ve decided to uncouple the object that records the message from the one that receives it (you’ll see what I mean later), mostly to demonstrate that Coverage works just fine with Django signals. For this, I need two applications in the project, the sender and the receiver:

./manage.py startapp sender
./manage.py startapp receiver

Django tests rebuild the database for every test, not for every test class or even test run. For every test. Keep that in mind, because it’s important to our test rationale.

Basically, I want to put in some content, and have it echoed back to me. I want to be able to put in multiple lines, and see multiple lines returned. Where I want to test this is in the more complex of the two applications, so the file to edit now is receiver/tests.py under my project directory. My test ends up looking like this:

from django.test import TestCase
from BeautifulSoup import BeautifulSoup as Soup
import re

class TestMessenger(TestCase):

    def test_02_multilple_okay(self):
        r = self.client.post('/',
                             {'message': "Mmmm. Doughnuts!"})
        self.assertRedirects(r, '/')

        r = self.client.post('/',
                             {'message': "Mmmm. Tacos!"})
        self.assertRedirects(r, '/')

        r = self.client.get('/')
        assert re.search('Doughnuts', str(r.content))
        assert re.search('Tacos', str(r.content))
        assert not re.search('Bacon', str(r.content))

Here, I enter two strings, and then assert that they exist in the test, and assert likewise that a third string I did not enter likewise does not exist.  This is a common enough test, but it is not the whole of the story.  If I were writing a more comprehensive application, I would have other tests about string correctness and security and so forth.

Run this test:

./manage.py test receiver

And it fails.

It should fail.  You haven’t written any code yet!  So, let’s write some code.

The Sender Application.

(In Django, “Applications” are libraries of code that do one thing and do them well; a suite of models, views, and templates for managing one existing components of a Django project. They are rarely standalone programs by themselves– those are “projects”. The idea is that you glue a lot of little applications together into your project, and after you’ve done this or collected enough other people’s applications, your projects is nothing but a few mini-apps that aggregate and combine other apps into one big functional web application, and you can turn over template design to designers. This language confusion confuses a lot of Django newbies, and you just have to learn to live with it.)

First, the “Sender.”  This is a bad name, perhaps, and TDD would have shown it to be a bad name, but I’m sticking with it.  It doesn’t reflect it’s real purpose (to record user input), but instead my intended purpose as a developer: it “sends” messages to the database manager, which the receiver app picks up, washes, and stores.   Really, sender and receiver ought to be something like “RawRecord” and “WashedRecord,” respectively.

In the sender directory, open models.py; and put this in:

from django.db import models
class SentMessage(models.Model):
    message = models.CharField(max_length = 255)

The Recevier Application

And in the reciever directory, open models.py and put this in:

from django.db.models import signals, get_model
from django.db import models

class ReceivedMessage(models.Model):
    message = models.CharField(max_length = 255)

def record_new_message(sender, **kwargs):
    if not kwargs.get('created', False):
        return
    instance = kwargs.get('instance')
    me, cr = ReceivedMessage.objects.get_or_create(message = instance.message)
    me.save()

signals.post_save.connect(record_new_message,
                          sender = get_model('sender', 'sentmessage'))

Now, this is where things get a little funny. You see, ReceivedMessage, which is more or less a direct copy of SentMessage, is never directly manipulated by user input. Instead, I have set up a listener in the last line there to pay attention to the database, and when a message from SentMessage is stored in the database, this file indicates that it is “interested” in that message, picks up a copy of it, and saves a new ReceivedMessage instance with a copy of the message.

The View

A view is what Django uses to interact with the user. The view for this thing is straightforward, but let’s take it apart carefully. First, here’s the text of the script which goes into receiver/views.py (we want to put it there to keep sender pure: sender shouldn’t know anything about receivers or views or any of that; it is receiver that has sender as a dependency, not the other way around):

from django.shortcuts import render_to_response
from django.template import RequestContext
from django.forms import ModelForm

from django.http import HttpResponseRedirect
from django.core.urlresolvers import reverse
from sender.models import SentMessage
from models import ReceivedMessage

class SenderForm(ModelForm):
    class Meta:
        model = SentMessage

def home(request):
    if request.method == 'POST':
        form = SenderForm(request.POST)
        if form.is_valid():
            form.save()
            return HttpResponseRedirect(reverse('home'))
    else:
        form = SenderForm()

    return render_to_response('home.html',
                              dict(form = form,
                                   messages = ReceivedMessage.objects.all()),
                              context_instance = RequestContext(request))

I’ve created a form that uses SentMessage as its model, so it now “knows” that when summoned it’s to show an HTML <input type="text"> object. In the method “home,” I specify that if input is coming in from a form via POST, we’re to process that input with the form’s handler, then save it. Otherwise, we create a blank form and render the template, which contains two things: a list of all messages in ReceivedMessage (which is picking up things we send to SentMessage, remember), and the form.

There are two things still to do: The template home.html specified in render_to_response, and set up the URL dispatcher.

The URL Dispatcher

First, the dispatcher. In the root directory of your project, open urls.py and substitute the existing urlpatterns entry with this:

urlpatterns = patterns('',
    url(r'^', 'receiver.views.home', name='home'),
)

This now tells our program that anything sent to the root directory of our server, i.e. http://localhost:8000/, will be handled by the view receiver.views.home.

The Template

The template goes in a file named home.html in the directory receiver/templates. You’ll have to make the directory yourself:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html lang="en" xmlns:fb="http://www.facebook.com/2008/fbml">
<head>
<title>Sender/Receiver Test Demo</title>
</head>
<body>
<h1>Messages:</h1>
<ul>
{% for message in messages %}
<li>{{ message.message }}</li>
{% endfor %}
</ul>
<hr>
<form action="." method="post">
{{ form.as_table }}
<input type="submit" value="Send">
</form>
</body>

If you look back at the view, you’ll see we sent two variables to the template renderer: messages and form. We know messages is a collection, so we can process it with a for-loop. The Form handlers have special renderers, and here we invoke the as_table renderer. Forms do not supply their own actions, so I have to supply my own <form> tags, along with all the usual action and method attributes. Here, the action is set to “the current URL”, and method is set to POST, which in our view tells the view to go through form processing.

I hope to hell I haven’t forgotten anything. I apparently did miss something (thanks for finding this, WolfWings!):

Go back into settings.py and add sender and receiver to your list of INSTALLED_APPS. In fact, because we’re not using anything else, they may as well be the only apps you install:

INSTALLED_APPS = (
    'sender',
    'receiver'
)

Test against live code

Okay, now, run this as a test:

./manage.py test receiver

And you should see:

Creating test database...
Creating table sender_sentmessage
Creating table receiver_receivedmessage
.
----------------------------------------------------------------------
Ran 1 test in 0.042s

OK
Destroying test database...

Good. Now run it as a server:

./manage.py runserver

And browse to http://localhost:8000/, and you should be able to play with it.

Test with coverage

As a final test, let’s run this through coverage:

coverage run ./manage.py test receiver

The set ought to run fine. Now we review the results:

 coverage report -m

And this spits out:

Name                Stmts   Exec  Cover   Missing
-------------------------------------------------
__init__                1      1   100%
manage                  9      5    55%   5-8
receiver/__init__       1      1   100%
receiver/models        11     10    90%   12
receiver/tests         13     13   100%
receiver/views         18     18   100%
sender/__init__         1      1   100%
sender/models           3      3   100%
settings               23     23   100%
urls                    2      2   100%
-------------------------------------------------
TOTAL                  82     77    93%

We can’t do much about manage.py, and reviewing line twelve of receiver.models shows that it’s a boilerplate check on whether or not the sender instance was being edited or created. I can live with that for now, but it is a warning flag: do I really mean to not save editings? Are editings even possible? Things to consider when writing more than a toy.

In a future post, I’ll show how to save this into a repository, and how to set up Hudson to watch the repository, check out your changes, and test them.

Code Availability

As always, the source code for the echo demonstration program is available at elfsternberg.com

2 Responses to Continual Integration Testing for Web Applications: A Sample Application in Django

WolfWings

November 16th, 2009 at 7:23 am

Two minor notes for being pedantic:

1) You don’t explicitly say it’s the receiver/views.py file being modified instead of sender/views.py file.

2) I had to manually add echo.receiver and echo.sender to settings.py running Python 2.6.2 w/ Django 1.1.1 installed via easy_install under Gentoo.

Elf Sternberg

November 16th, 2009 at 9:26 am

That’s not pedantic. Those are excellent points. I’ll fix that.

Comment Form

Subscribe to Feed

Categories

Calendar

November 2009
M T W T F S S
« Oct   Dec »
 1
2345678
9101112131415
16171819202122
23242526272829
30