How BioWare Makes Game Communities Work

An Interview with Scott Bakker

The Lone Software Developer's Methodology

Entertaining and interesting rant (with explanation).

When unit testers fail...

At some point over the last week, my nose unit tests for twill started flailing. Yes, not just failing, but flailing. There was so much output that it was tough to figure out exactly what was going on. I spent a few minutes across a few days trying to figure out what had changed; I'm pretty sure it was due to a nose version upgrade, now, but I could never actually figure out what version of nose recognized all my tests *and* still worked.

Whatever the proximal cause, it turns out I designed my unit tests incorrectly. The tests are in modules (separated in individual files), with 'setup', 'test', and 'teardown' functions in each module. The latest version(s) of nose was discovering the 'setup' and 'test' functions and running (as near as I can tell) 'setup' twice, 'test' once, and 'teardown' zero times.

I'm still not sure if this is a bug; I couldn't figure out what the behavior of nose should be, because it's not explicitly documented on the nose page. (Maybe it's on the py.test page?)

Finally, I discovered 'setup_module' and 'teardown_module' and renamed setup and teardown in all my tests. That solved my problems.

I also learned (from the nose documentation) that you could specify a test collector in setup.py when using setuptools. So now 'python setup.py test' will run all the tests.

The shocker for me in all of this was how uncomfortable I felt even thinking about modifying twill without the unit test framework. Even though I could execute the tests individually, I knew that I wouldn't run them nearly as frequently as I do when 'nose' is working. Automated tests are a really important security blanket now...

What users want, and what they'll get.

So, it appears that an increasingly common use case for twill is Web browsing.

When I started developing twill, I intended to develop a replacement for PBP. PBP was aimed at Web testing, and that's what I needed myself. Roughly, this meant:

• a simple scripting language, with an interactive shell;
• Functional cookies/formfilling/etc, so that interacting with a Web site actually worked;
• a fair amount of assert stuff, like "code" (to make assertions about return codes) and "find" (to make assertions about presence of specific text/regexps).

Then I made a few design choices: to use python and python's Cmd for the interactive shell, and to use a metaclass to automagically import all twill.commands functions into the interactive shell. This meant that all of the functions accessible from twill were also directly accessible via Python, and the twill language docs functioned equally well as Python language docs.

Thus, twill could be used from Python code to do most Web browsing tasks that didn't involve JavaScript.

I was happy with this result, but it was largely unintended. I mostly use twill via the scripting language.

However, the early press on twill was from people like Grig and Michele, who talked glowingly about the ability to use twill from Python. This has led to people wanting more functionality: especially, sensible return values from twill commands. This, in turn, has led to a bit of caviling on my part about this direction, because I haven't really thought it through.

Anyway, the upshot is that I have to rethink my priorities for twill a bit. I was going to focus on recording functionality and extension management for the next release, but it seems like I should also work on simplifying and documenting the browser interaction code. Given the one-to-one mapping between twill.commands and the scripting language, I don't want to add things like return values and Python-specific documentation to the commands; perhaps I can satisfy people with a browser-object interface...

The big surprise for me -- and it really shouldn't have been a surprise -- is that people really seem to want to do Web browsing via a trivially simple interface: go, follow, submit, and get_page contain 90% of the desired functionality. mechanize and mechanoid are serious overkill for this level of Web browsing, and the fact that twill puts a high priority on "just working" (albeit at the expense of customizability) probably helps contribute to users' interest.

The simplest route for me to go is probably to work on two applications I've been planning: twill-crawler, and mailman-api. (Presumably the names give away the function ;).) Then I'll have some idea of what people need for Web browsing; right now I'm feeling a bit underplanned, so to speak.

--titus