Older blog entries for titus (starting at number 176)

Martin Fowler on Ruby

Evaluating Ruby says nothing hugely new, but does give the whole RoR combo a thumbs up from his ThoughtWorks-y perspective.

Malone

Yesterday, I wrote about tracking bugs across bugzilla. Stephen Thorne pointed me towards Malone, a Canonical system for doing just that.

fight!

fight fightfight! ;)

...and someone else weighs in.

In other advogato news, bi calls advogato "elitist". Uhh... yeah, sure, if by "elitist" you mean "we use an explicitly simplistic ranking system to keep spam out". We have incredibly low levels of spam on advogato, and I have to say I don't find the ranking system all that elitist. More exclusionary, if anything.

--titus

For future reference

The slamd distributed load generation engine.

mocks considered harmful?

robertc

asks if mock objects lead to interface skew.

I could probably go off on a long polemic here, but let's see if I can keep it short... Basically, if all you're doing is testing with mocks, then the answer is probably that you're going to screw yourself up.

But let's generalize the question, to see if perhaps there's a deeper truth:

Dear Mr. Touchstone,

I'm using technique X for testing, and I'm only doing unit testing (...or functional testing, or acceptance testing, or integration testing, or regression testing, or UI testing, or smoke testing, or fizzlebar testing...) I'm worried that this is going to cause problems in the future. What should I do?

-- Anxious in Albuquerque

...and my answer:

Dear Anxious,

I think you should consider testing at additional levels. Generally unit testing is not sufficient, because it purposely tests only small, largely independent units of the code. Functional and smoke tests (as in "push button -- does smoke appear?") should be de rigeur for any project; IMO they're as or more important than unit tests.

So that's my answer: there are bigger problems lying in wait! Mock objects are a specific solution to a fairly general problem with unit tests: you can mimic external interfaces that need to interact with your code in fairly stereotyped ways. That stereotyping is a specific weakness of the approach, as well as being the strength that led you there in the first place.

More generally, any time you find yourself relying completely on your unit tests, you're going to be in trouble. Unit tests are a great programmer tool, and they are very useful, but they are not everything.

Someone interviewed me

Pythonthreads just posted a lengthy interview with me. It's been a month or so since I answered the questions, and I'm still fairly happy with my answers!

Bugs

I read an interesting post a few days ago that discussed the dismal situation with bugs in Linux packages: basically, there's been a proliferation of bugzillas, and you can often no longer have the conversation necessary to fix a bug in any one of the bugzilla sites.

It occurs to me that one possible solution is to build a test case that demonstrates the bug; that's sort of the maximally portable bit of problem documentation necessary, no? Then just post that to the most upstream bugzilla...

--titus

GEICO

I went to GEICO's Web site today to pay our car insurance, and apparently hit a bad URL (retrieved from old e-mail). The error page came up with a Tolkien quote: "Not all who wander are lost." How cool is that?

Mango Sauce

The latest in Google's AdSense misadventures: Mango Sauce banned from adsense. Going by this page, what we have here is an arbitrary decision being made by a lower-level functionary at Google. Read the page & send in your protest...

The mango sauce page itself is pretty interesting. Very racy, but quite entertaining.

Webstemmer

This looks cool. Someone should integrate it with twill.

Deleting spam sent to moderatedSourceForge mailing lists

Problem: Sourceforge runs an old version of mailman that doesn't have the nifty new "discard all" button.

Solution: a twill script to automate the process of logging in & clicking "discard" for each message.

(I've mentioned this before, I think, but it's gotten easier with some of the latest additions to twill.)

so:

...
The PyWX-discuss@lists.sourceforge.net mailing list has 863 request(s)
waiting for your consideration at:

https://lists.sourceforge.net/lists/admindb/pywx-discuss

Please attend to this at your earliest convenience. This notice of pending requests, if any, will be sent out daily. ...

thus:

% ./twill-sh examples/discard-sf-mailman-msgs -- pywx-discuss
>> EXECUTING FILE examples/discard-sf-mailman-msgs
==> at https://lists.sourceforge.net/lists/admindb/pywx-discuss
Enter list password:
Note: submit is using submit button: name="request_login", value="Let me in..."
-- matches 220
set 220 values total
Note: submit is using submit button: name="submit", value="Submit All Data"
--
1 of 1 files SUCCEEDED.

(wipes hands together in a "done" gesture)

I don't grok HTTP very good

I spent a lot of time this weekend hacking together two WSGI middleware apps: first, a recorder/playback system, which lets me both record and play back all traffic into and out of a given WSGI application; and second, a WSGI-based transparent proxy app. The ultimate goal, of course, is to enable recording of all Web traffic much like TCPWatch, but in a nice modular WSGI way.

So, the recorder works: I can now place a middleware wrapper around any app, save & pickle everything that goes into and out of the app, and then play it back. That's pretty neat.

The proxy doesn't work. Or, rather, it sort of works: I can browse Google Search and go to many (but not all) Web sites. There is, however, a bug somewhere: none of my Trac sites work, and AJAX stuff seems broken, too. I'm guessing it's in my broken dealing with HTTP 1.1 features; that, or WSGI and proxying just don't go together. (Also plausible, 'cause I'm breaking a number of PEP rules here.)

Depressing. But I'm sure I'm almost there.

(People interested in wading through some really cruddy code can e-mail me to get a copy. But bring your knee-high waders, 'cause it's bad.)

--titus

Two new articles

I've put up two new articles -- What is WSGI? An Introduction and Testing WSGI applications with twill -- a (very) brief intro. Both were written for today's SoCal PIGgies meeting.

In the article on testing WSGI applications with twill, I have a doctest example that pretty stubbornly won't work. I hereby publicly appeal to Grig to get it to work ;).

Legal issues with "releasing software into the wild"

A useful reference.

And the moral of the story is...

In this very interesting piece on a company moving to Linux, the lesson was: test. The company had the confidence to switch deployment platforms -- and not just once but twice -- largely because they had a complete testing setup. Or at least that's what I took away from it ;).

--titus

Various miscellany...

MIT swipes Caltech cannon; then hits below the belt!

http://web.mit.edu/ec/www/cannoncoeds/:

"""
"I say take 'em back with the cannon.  In fact, forget the cannon,"
remarked senior Jeff Phillips.  Those who have been here for a shorter
period of time reacted differently.  Those quotes are not printable.
"""

Heh.

Fluxus

My sister pointed out this "fluxus" thing. Check it out [pdf]. Not sure what to make of it. To quote,

A piano is lifted by means of a windlass to the height of 2 meters and then
dropped.  This is repeated until the piano or the floor is destroyed.

I think it's a technique that can be used to shatter preconceptions about art and the role of the audience.

Whooooooooooaaaaaaaaaaaaa.

agile-testing

I've been enjoying the agile testing mailing list; here's an interesting post where Michael Bolton (the famous tester, not the famous singer) discusses the so-called 'flat cost curve of change' due to agile methods.

avriettea = ennui+belligerent

avriettea, if I never entered the business because I knew in advance that I wouldn't be happy, does that mean that I'm still a defeatist? (That's why I'm in academics, frankly; it keeps me happier, long term, than programming. Bear in mind that science is as or more difficult than most computer jobs, we're just not paid so well.)

Anyway, it's an interesting idea, but I don't understand why sticking with something that you hate simply because leaving would be "weak" is a good path. (Not having anywhere else to go -- that's a different story.)

And yes, it worries me that you write posts like this and talk about playing with large guns ;).

--titus

Michigan State University, Software Engineering, and Python

Just got back from MSU, where I gave two talks, one on my computational work and one on my biology research. (I'm applying for a joint CS/biology position there.)

One of the topics that came up frequently was whether or not I was interested in (or capable of ;) teaching CS undergrad courses, given that I have little formal CS training on my resume. My recent activities in agile testing sparked some interest, as did the notion of teaching a software engineering course based on agile methodologies. I also mentioned Greg Wilson's Software Carpentry course as a possible cross-over course for computational scientists.

I also proposed using the vast base of available OSS software as a starting point for an advanced software engineering course. The idea would be to demonstrate problems and solutions on an already-available hunk o' code; things like setting up (or extending) testing, stabilizing APIs, etc. It could actually be combined with a survey course on different languages. Hmm.

Interestingly, people in the department were already investigating the idea of switching to a scripting language -- Python was explicitly named -- for part of an intro-level programming course. (One professor mentioned ALICE, too.) Needless to say I'd be pretty enthusiastic about the opportunity to introduce Python at that level.

I also spent some time proselytizing about agile development techniques to various friends. Yep, I drunk the cool-aid, it seems.

--titus

From lesscode:

" The only architecture that matters is the simplest one you can get to solve the problem at hand. "

Amen.

DNS & mailman, oh my

Spent several hours today wrestling with DNS and mailman. The goal was to consolidate my DNS onto one machine which would then serve to my hosting company's name servers; this has been years in coming, and I finally had all my pins lined up. All went well, except for a weird glitch in the public-facing name servers which ended up disagreeing on some domains. Boiiiinnggg went some mail...

Then it was mailman's turn. I decided to use Debian's mailman install, which worked fine except for some of the standardized yet esoteric places they place config files. /etc/mailman/mm_cfg.py, anyone?

Next up: exim4 and virtual domain hosting.

I'm getting too old for this sh*t.

rsync.net

John, of JohnCompanies.com (my hosting provider), is starting a new service; I betcha can guess what it does from the name ;). Go check it out.

--titus

More buildbot

In addition to our buildbot automation hacks, we spent a fair amount of time twiddling our buildbot configuration for the PyCon project. I wrote up some of the configuration file stuff last night. If you're interested in a private "force build" status page, locking master & slave resources, driving builds from svn checkin, or using the @reboot crontab extension to start buildbot on boot, you might be interested in reading it.

More dnspython

I sent the little dns_check module on to Bob Halley, the author of dnspython, and he sent me back a couple of patches for the code. Good stuff. All checked-in and documented now.

Sysadmin tools

This slashdot article infuriated me more than the usual half-troll slashdot article. An MP3 player and a terminal program are sysadmin tools? Bah.

Here are a few of my favorites.

  1. screen -- run multiple programs in a single terminal window, flip between the output, detach and re-attach. Life is good. My all-time favorite; I've been using it ever since Mark Galassi introduced me to it back in the late 80s/early 90s.

  2. VNC -- like screen, but for X11 programs. Bonus: VNC to Flash recorder for screencasting, although that's not really a sysadmin application.

  3. bash 'for' loops at the command line. Silly, I know, but combined with 'cut', 'sort', 'uniq', 'tail', and 'head', I can do tons of things in one long complicated line. I don't actually know how to program in bash beyond this -- I use Python for anything more complicated.

  4. 'find'. Its command line options have gotta be nearly Turing complete. Powerful beyond belief.

  5. supervisor. It's hard to explain how happy I was when I found this Python-based system for starting and restarting persistent processes...

  6. twill. Really. Having a command-line tool to script Web site aliveness tests & (now) DNS checks is pretty handy. I'll probably add some ping-is-the-machine-alive code, too.

I'm sure there's more that I'll remember as soon as I post this.

--titus

26 Mar 2006 (updated 26 Mar 2006 at 20:14 UTC) »
Using 'supervisor' to keep your Web sites (and other processes) running

I've been using supervisor to manage some of my Web stuff -- especially my Trac/SCGI sites. I wrote up an intro article on setting it up here. Comments welcome.

Using twill for more than Web testing

Grig recently pointed me towards systir, a DSL for doing system-level testing. It looks oddly like twill code (which in turn looks very similar to PBP, which looks like WWW::Mechanize::Shell).

Anyway, based on this similarity to systir, Grig suggested that twill could be used as an all-around acceptance testing framework, much like systir. (See my last post for a specific example of extending twill with arbitrary python code.) Interesting idea... I'd been thinking about writing some extensions to deal with DNS monitoring (e.g. "does this DNS server report that there is an A entry for this host pointing to this IP?"), and so I took the bit in my teeth, so to speak, and went ahead with it.

Ta-da! A mere 30 minutes later, the 'dns_check' extension module for twill was born.

Below is the output from a test script for this module. 'dns_check' uses the fantastic dnspython Python library to allow simple assertions about DNS entries; stuff like 'make sure these two names resolve to the same address' is pretty easy with it. (Note that I turned on command debugging output so that commands could be seen as they were executed; normally these commands are silent.)

% ./twill-sh test_dns.twill

>> EXECUTING FILE test_dns.twill twill: executing cmd 'extend_with dns_check' Imported extension module 'dns_check'.

Description:

Extension functions to help query/assert name service information.

Functions:

* dns_resolves -- assert that a host resolves to a specific IP address. * dns_a -- assert that a host directly resolves to a specific IP address * dns_cname -- assert that a host is an alias for another hostname. * dnx_mx -- assert that a given host is a mail exchanger for the given name. * dns_ns -- assert that a given hostname is a name server for the given name.

twill: executing cmd 'dns_resolves amazon.com 207.171.175.29 # amazon.com ==> any of 3' twill: executing cmd 'dns_resolves amazon.com 207.171.166.102' twill: executing cmd 'dns_resolves amazon.com 72.21.206.5' twill: executing cmd 'dns_resolves idyll.org. 131.215.34.116' twill: executing cmd 'dns_resolves idyll.org. joiry.net. # same IP addr?'

twill: executing cmd 'dns_mx idyll.org. mail.idyll.org. # '.'s are handled' twill: executing cmd 'dns_mx idyll.org mail.idyll.org.' twill: executing cmd 'dns_mx idyll.org. mail.idyll.org'

twill: executing cmd 'dns_a amazon.com 207.171.175.29 # explicit 'A' records.' twill: executing cmd 'dns_a amazon.com 207.171.166.102' twill: executing cmd 'dns_a amazon.com 72.21.206.5'

twill: executing cmd 'dns_cname www.idyll.org. idyll.org. # explicit 'CNAME' records.'

twill: executing cmd 'dns_ns idyll.org nsa.idyll.org' -- 1 of 1 files SUCCEEDED.

The script itself is pretty simple, of course:


debug commands 1 extend_with dns_check

dns_resolves amazon.com 207.171.175.29 # amazon.com ==> any of 3 dns_resolves amazon.com 207.171.166.102 dns_resolves amazon.com 72.21.206.5

dns_resolves idyll.org. 131.215.34.116 dns_resolves idyll.org. joiry.net. # same IP addr?

dns_mx idyll.org. mail.idyll.org. # '.'s are handled dns_mx idyll.org mail.idyll.org. dns_mx idyll.org. mail.idyll.org

dns_a amazon.com 207.171.175.29 # explicit 'A' records. dns_a amazon.com 207.171.166.102 dns_a amazon.com 72.21.206.5

dns_cname www.idyll.org. idyll.org. # explicit 'CNAME' records.

dns_ns idyll.org nsa.idyll.org

All in all a very short & satisfying bit of hacking. I'm going to send the underlying code to Bob Halley, the author of dnspython, to see what I did wrong ;). If anyone has further suggestions re the functionality, please drop me a line. You can grab the very latest twill version for yourself here if you want to play; the 'dns_check' module is in twill/extensions/dns_check.py.

Incidentally, the very latest twill code also contains some nice additions to the shell for dealing with extension modules. Specifically, TAB completion now works for extension module commands, as does the 'help' command. Huzzah!

--titus

LAMP

Linux; Apache; Most of our cool scripting languages start with P; and PostgreSQL. Yep, LAMP ;). (ref)

Miscellaneous twill

The next release of twill is taking a bit longer than I'd planned. I think this is mainly because of the increase in the number of people using it -- they keep finding bugs, damn them! However, bug fixing is proceeding apace, and there have been one or two amusing incidents along the way. At this rate I think it's fair to say that most of the minor burrs in twill will be ironed out by the time that the beta, 0.9, arrives. I've come to the realization that a "1.0" release should be predicated on as much actual

use of the software as possible -- that way you really do get not only a well-tested piece of software, but one that genuinely meets a variety of needs.

On that front, Grig pointed me towards a challenge on the agile-testing list. Brian Mairick presented a Watir solution, and it seemed incumbent upon me to immediately drop all other work and whip out an example using twill. A solution in twill form is here:

extend_with table_picker
go http://issola.caltech.edu/~t/transfer/jjj.html

showforms check_cells "name1" 0 showforms

Of course, this merely punts the solution into the 'table_picker' extension module, which follows at the bottom.

A few quick notes about this solution:

  • You'll need the latest version of twill, 0.8.4a11, either in egg version from my alpha dist directory or from the .tar.gz resting home.

  • With that, Python, and setuptools, you're set; no other software is needed.

  • Both the Watir solution and the twill solution rely on using the Real Programming Language (be it Ruby or Python) to do the logic processing necessary to find the right cell(s).

  • My solution could be shorter, but it works fine as-is ;).

  • The real advantage to twill in this particular situation is that it's completely and trivially automatable. Of course, it also doesn't handle JavaScript...

--titus

table_picker.py:

import re
from BeautifulSoup import BeautifulSoup

def check_cells(label_regexp, label_column): """ >> check_cells <label_regexp> <label_column>

Check all checkboxes in rows where column #'label_column' matches 'label_regexp'. """ # deal with input parameters label_regexp = re.compile(label_regexp) label_column = int(label_column)

# first, get the browser obj from twill import get_browser, commands b = get_browser()

# grab the HTML from the current page html = b.get_html()

# parse the HTML. soup = BeautifulSoup() soup.feed(html)

# grab the table table = soup.first('table')

# grab the rows & iterate rows = table('tr')

names = [] for i, row in enumerate(rows): cols = row('td') # get the label column col = cols[label_column]

# if it's a match, run through & record all checkboxes if label_regexp.search(str(col)): for col in cols: for element in col('input'): if element.get('type') == 'checkbox': names.append(element['name'])

for name in names: name = str(name) commands.formvalue('1', name, '1')

167 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!