This is part 4 of a series.

So far, we've written a basic Django application, written some tests for it, checked everything into a central repository, and then integrated those tests with the Hudson continual integration server.

But Django's tests run in a kind of pseudo-server mode, with both the tests and the application running inside the same process. A more realistic test is to the run the server, and then actually hit the process from the outside with real HTTP requests and review the responses that comes back.

This is what WATIR (Web Application Testing In Ruby) does very well. Even better, you can run this with Celerity, a headless browser (that is, a browser that doesn't have a display), which means that your tests can run on an integration server like Hudson in a stand-alone mode, and you can feel somewhat reassured that your program is doing what you expect it to do.

I'm going to go one step further, however, and integrate WATIR with Cucumber, a "behavior driven development" tool. In the Cucumber language, you describe the behaviors you expect a typical persona to perform as they wander through your site. Cucumber's language is highly management-friendly, allowing you to describe your scenarios in plain English.

Now, this is Ruby-heavy. And you'll need jRuby installed for Celerity. Install Celerity, WATIR, RSpec, and Cucumber.

Features

Cucumber documents "features," which are things your user can do with your system. In the echodemo, we have one feature: users can talk to themselves. You're going to need a typical Cucumber environment first, so move to the root directory for the project and

mkdir -p features/support
features/step_definitions

Now, under features/support you're going to edit a file, env.rb:

$: << File.expand_path(File.dirname(__FILE__) + "/../support")

require 'spec'
require 'celerity'

HOST = '127.0.0.1'
PORT = 10003
ADDRESS = "http://#{HOST}:#{PORT}"

Before do
  @browser = Celerity::Browser.new
end

Here, we tell the cucumber environment that we're going to use Celerity and RSpec, and before anything else gets run we're going to instantiate a new browser connection to a Celerity browser.

Ruby's pretty magical to me (in a bad way), but this is easy enough for me to follow.

The next thing we're going to do is create a run wrapper for one test. It's a bash script, and it goes into the features directory. Here's run.sh:

#!/bin/bash

HOST='127.0.0.1'
PORT=10003

# Initialize the environment.
rm -f echodemo_test.db
DJANGO_ENV=TEST python ./manage.py syncdb --noinput

# Start the server.  This becomes JOB_1
DJANGO_ENV=TEST python ./manage.py runserver "$HOST:$PORT" > /tmp/django_testrun.txt 2>&1 &

# Run the tests
jruby -S cucumber $*

# Kill all jobs associated with the test server
JOBS=`pgrep -d' ' -f '/usr/bin/python.* ./manage.py runserver .*[: ]*10003'`
kill -HUP $JOBS 2> /dev/null

This is pretty self-documented, and the HOST and PORT variables line up with the ones in the environment. Basically, we're cleaning out our test environment, re-initializing the test database, running the server in the background, then running the browser to bother the server and run the tests. When we're done, we kill all the jobs associated with the server run for this test.

The environment variable DJANGO_ENV=TEST tells Django not to use the usual database.  To make this stick, we have to modify settings.py:

import os
ENVIRONMENT = os.getenv('DJANGO_ENV', 'development')
DATABASE_NAME = 'echodemo.db' # Or path to database file if using sqlite3.
if ENVIRONMENT == 'TEST':
    DATABASE_NAME = 'echodemo_test.db'

Now, believe it or not, we're ready to write a test.

Your first feature

A feature uses the Cucumber programming language, which looks like English. The main keywords are "Given", "When", "Then", and "And." There are others, of course, but you'll get the flavor of it from my feature for the echo server, which goes into the file features/echodemo.feature:

Feature: Talk to myself

In order to ensure my insanity, when I post a message,
I want to have it echoed back to me.

    Scenario: Post a message
        Given I am at the home page
        When I post the message "I'm Home!"
        Then I should see "I'm Home!"

Under the "Scenario" header, I have a "Given", a "When", and a "Then". You can have multiples of all of these, using "And", which makes Cucumber use the same token as previously seen-- an "And" following a "When" is treated as another "When", and so on.

Now, we're going to back out to the root of our project and call features/run.sh. The results are spectacular:

You can implement step definitions for undefined steps with these snippets:

Given /^I am at the home page$/ do
 pending
end

When /^I post the message "([^\"]*)"$/ do |arg1|
 pending
end

Then /^I should see "([^\"]*)" on the page$/ do |arg1|
 pending
end

What this means is that none of these steps have actually been implemented and WATIR doesn't know how to run them. Therefore, we take to the next step: writing them in Ruby. These steps go into a ruby file under the directory step_definitions, so let's call it echodemo_steps.rb:

Given /^I am at the home page$/ do
  @browser.goto ADDRESS
end

When /^I post the message "([^\"]*)"$/ do |arg1|
  @browser.text_field(:name, 'message').set(arg1)
  @browser.button(:value, "Send").click
end

Then /I should see "([^\"]*)"$/ do |text|
  @browser.text.should include(text)
end

In many ways, this echoes the tests we did under Django, but now the server is running independent of the tests, and the browser is looking at assets in the HTML document, rather than just whatever the Django view happens to return internally. If this were a bigger test, we could test edge server includes and other features that aren't a part of Django normally, but are provided by ESI servers like Varnish.   Each of these tests uses the @browser variable that Cucumber instantiates when it first starts up and finds its env.rb file.

Now that we've created the steps, let's run it again:

Feature: Talk to myself

In order to ensure my insanity, when I post a message,
I want to have it echoed back to me.

  Scenario: Post a message              # features/echodemo.feature:6
    <span style="color: green">Given I am at the home page         # features/step_definitions/echodemo_steps.rb:1</span>
    <span style="color: green">When I post the message "I'm Home!" # features/step_definitions/echodemo_steps.rb:5</span>
    <span style="color: green">Then I should see "I'm Home!"       # features/step_definitions/echodemo_steps.rb:10</span>

1 scenario (<span style="color: green">1 passed</span>)
3 steps (<span style="color: green">3 passed</span>)
0m2.054s

If your console is capable of colors, you'll see it just like above, and green is good. Congratulations, that's your first test of the server running independent of the tests being run on it, and the Celerity browser says it works as advertised.

Celerity works with some Javascript. I won't go into that here, but it claims to be Rhino-compatible and W3C compliant. I haven't tried it out; all my Javascript testing with WATIR is usually done in a real browser, like Firefox, but maybe I'll get to demonstrating that as well.

Next, I'm going to try and get this test pass working under Hudson.

Source code for this version of echodemo is available.