26 Mar 2006 titus   » (Journeyer)

Using 'supervisor' to keep your Web sites (and other processes) running

I've been using supervisor to manage some of my Web stuff -- especially my Trac/SCGI sites. I wrote up an intro article on setting it up here. Comments welcome.

Using twill for more than Web testing

Grig recently pointed me towards systir, a DSL for doing system-level testing. It looks oddly like twill code (which in turn looks very similar to PBP, which looks like WWW::Mechanize::Shell).

Anyway, based on this similarity to systir, Grig suggested that twill could be used as an all-around acceptance testing framework, much like systir. (See my last post for a specific example of extending twill with arbitrary python code.) Interesting idea... I'd been thinking about writing some extensions to deal with DNS monitoring (e.g. "does this DNS server report that there is an A entry for this host pointing to this IP?"), and so I took the bit in my teeth, so to speak, and went ahead with it.

Ta-da! A mere 30 minutes later, the 'dns_check' extension module for twill was born.

Below is the output from a test script for this module. 'dns_check' uses the fantastic dnspython Python library to allow simple assertions about DNS entries; stuff like 'make sure these two names resolve to the same address' is pretty easy with it. (Note that I turned on command debugging output so that commands could be seen as they were executed; normally these commands are silent.)

% ./twill-sh test_dns.twill

>> EXECUTING FILE test_dns.twill twill: executing cmd 'extend_with dns_check' Imported extension module 'dns_check'.


Extension functions to help query/assert name service information.


* dns_resolves -- assert that a host resolves to a specific IP address. * dns_a -- assert that a host directly resolves to a specific IP address * dns_cname -- assert that a host is an alias for another hostname. * dnx_mx -- assert that a given host is a mail exchanger for the given name. * dns_ns -- assert that a given hostname is a name server for the given name.

twill: executing cmd 'dns_resolves amazon.com # amazon.com ==> any of 3' twill: executing cmd 'dns_resolves amazon.com' twill: executing cmd 'dns_resolves amazon.com' twill: executing cmd 'dns_resolves idyll.org.' twill: executing cmd 'dns_resolves idyll.org. joiry.net. # same IP addr?'

twill: executing cmd 'dns_mx idyll.org. mail.idyll.org. # '.'s are handled' twill: executing cmd 'dns_mx idyll.org mail.idyll.org.' twill: executing cmd 'dns_mx idyll.org. mail.idyll.org'

twill: executing cmd 'dns_a amazon.com # explicit 'A' records.' twill: executing cmd 'dns_a amazon.com' twill: executing cmd 'dns_a amazon.com'

twill: executing cmd 'dns_cname www.idyll.org. idyll.org. # explicit 'CNAME' records.'

twill: executing cmd 'dns_ns idyll.org nsa.idyll.org' -- 1 of 1 files SUCCEEDED.

The script itself is pretty simple, of course:

debug commands 1 extend_with dns_check

dns_resolves amazon.com # amazon.com ==> any of 3 dns_resolves amazon.com dns_resolves amazon.com

dns_resolves idyll.org. dns_resolves idyll.org. joiry.net. # same IP addr?

dns_mx idyll.org. mail.idyll.org. # '.'s are handled dns_mx idyll.org mail.idyll.org. dns_mx idyll.org. mail.idyll.org

dns_a amazon.com # explicit 'A' records. dns_a amazon.com dns_a amazon.com

dns_cname www.idyll.org. idyll.org. # explicit 'CNAME' records.

dns_ns idyll.org nsa.idyll.org

All in all a very short & satisfying bit of hacking. I'm going to send the underlying code to Bob Halley, the author of dnspython, to see what I did wrong ;). If anyone has further suggestions re the functionality, please drop me a line. You can grab the very latest twill version for yourself here if you want to play; the 'dns_check' module is in twill/extensions/dns_check.py.

Incidentally, the very latest twill code also contains some nice additions to the shell for dealing with extension modules. Specifically, TAB completion now works for extension module commands, as does the 'help' command. Huzzah!


Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!