Older blog entries for titus (starting at number 109)

Here. Cute ;).

Sourpuss & unfair gripe of the day

It's hard to take the xerces-c project seriously with build instructions like this.

I particularly like the way they make you set an environment variable, although the "runConfigure" script is pretty astounding, too.

(Not that I like the default autoconf/configure much, but it's pretty standard; I have to re-read the instructions for xerces-c everytime I build it, this way.)

WSGI support for Trac

Took the existing patch for adding WSGI support to Trac and added a simple SCGI server and a dumb CGI server to wrap the WSGI app.

Here. Works for me, so far...

trac, incidentally, is pretty cool. Small, neat, to the point. And pretty.

DoS attack

Finally got the chance to investigate some intermittent Web server downtime. Turns out that it was neither name service nor Apache config problems, but instead a DoS SYN flood. This guide was useful; I got the site back up and running in no time flat. I'm still getting lots of connections, though. If it escalates I'll start using iptables to block.

I'm not sure which site they're after. Perhaps wrightflyer.org?

twill additions

Ed Rahn sent me a nice patch adding run and runfiles functionality to twill. These enable the execution of straight Python statements and other twill scripts from within twill. He also submitted something enable $variable substitution, which I'm still a bit iffy on; people tend to expect "this $variable" to be substituted, which doesn't work yet. Adding that could be a bit of a mess.

Ed's patch catalyzed a namespace reorganization and the addition of setglobal and setlocal commands, as well.

All in all, it sounds like a busy weekend -- but it wasn't ;). Lotsa sleep. hooray!

--titus

Gave a presentation on darcs, the distributed revision control system, at the SoCal Python Interest Group meeting yesterday. The presentation is available here.

twill news

Lambda the Ultimate, the Programming Languages Weblog, had a post about twill. I am not worthy. It was cool to see it there, though!

twill is in a nice, happy place. Most of the work I plan to do on it has to do with increasing its applicability to e.g. thread and load testing. Very little functionality needs to be added; if anything, I need to help out with mechanize. twill is, after all, just a thin wrapper around mechanize...

Speaking of mechanize, what's up with mechanoid? It took me about 5 minutes of browsing to figure out that this page was where I wanted to go, not the PyPI-referenced page. I still haven't figured out why the guy forked mechanize, either.

back to the crib,
--titus

Released twill 0.7.2. May it live long and prosper.

It's mainly a bugfix+docs release, with one or two minor new features.

Trac

Installed Trac for the aforementioned Caltech Software Carpentry class. Pretty cool stuff. A fairly simple, straightforward project management system. It's also pwetty.

Naturally I had to get frustrated with something, so I spent half an hour hacking in SCGI support (it comes with FCGI and CGI support only). I'm happy to send out patches, but I'm not going to submit 'em to the Trac project until I have some more time to make a clean set. (Or somebody asks.) I found a note that Trac is going to add SCGI support with WSGI support sometime around 2.0; right now it's at 0.8.4. Hmm. That might be a worthwhile hour or two of my time...

I also spent an hour or so installing a hacked version of Trac that contains support for a darcs backend instead of subversion. Works really well so far! The tricky bit was noticing that trac actually contains provisions for running directly from its source directory. This is good because it has a number of moving parts which would be difficult to otherwise configure without simply installing it from scratch in some new location.

chillin',
--titus

Spent much of Tuesday trying to get the lirc_streamzap module working. No luck; it segfaults. The situation is better than it was -- LIRC didn't even nominally support the StreamZap until relatively recently -- but it sucks balls that I can't get it working. Ah well, less whining, more hacking!

WinMyth rocks, even if it's still a bit slow & buggy. Good job! (I've been using MythOnMacOsx for a while now; I'm trying to get my wife hooked into our Myth setup with her WinXP box.)

Updated twill, my Web-scripting-for-testing tool, to fix a few problems. The biggest problem was that the extend_with command didn't work to import commands into the shell, although it worked fine for script execution.

And, in other news, working on a Trac install for the Caltech Software Carpentry class. More on that once it's all set up...

All of this is occurring in a general background of sadness for the victims of Katrina. I wasn't nearly this sad about 9/11, for some reason; perhaps because I think the misery was more extended in this case. Not sure. But when I read about 100 people who drowned while waiting for emergency evacuation... I can't help but get pretty upset. sigh. My wife may end up going out there to help the Red Cross with shelter ops, so we're holding off on donations 'til we see if we have to buy a plane trip. I'm feeling moderately useless myself ;(.

peace out,
--titus

Cartwheel passes data to and from 3rd-party binaries. Generally, that data is in the form of sequence. So, when talking about security, I felt it necessary to add this proviso:

...if someone can encode an attack in valid IUPAC DNA or protein sequence, then it may get through...

I wonder if it's possible to write a buffer overflow for BLAST in DNA sequence? That'd be pretty neat. I'm tempted to offer a bounty, but I don't have any money to offer. Hmm. Perhaps I could offer to sequence part of someone's genome? ;)

Anyway, a first stab at docs is complete. Let's hope they're not too rudimentary to be of use.

twill

OK, I'm going to start working on this soon. There's at least one outstanding bug that Michele Simionato has been waiting for me to fix...

--titus

Over the last year or two, I've become a convert to the (rather obviously good) practice of documenting my packages. I'm not talking about the source code itself, which is more-or-less documented depending on the when I wrote it and the language I wrote it in -- e.g. Python requires fewer docs, C++ more. I'm talking about documenting APIs, developer setup, compilation tricks, and installation procedures. Writing tests is also a form of documentation, and I've been doing that a lot more, too.

Now, when I say "become a convert" I mean that I'm actually doing it now. Given the time involved, this isn't trivial: even writing sketchy documentation is a time-suck. It's also way less fun than hacking on new features. The flip side is that people are more likely to actually interact with your software if you document it. And, for me, that's one of the main points behind writing open source software...

There are some less obvious benefits, though, that add even more value than use. The main one is that writing installation instructions for other people gives me time to reflect upon just how crufty the install actually is for some of my packages. Cruft is usually easy to fix, once identified -- but I tend to work in my own little cocoon of already-installed programs, and it's hard to identify the cruft without leaving that cocoon. Another benefit is that I get to document some of the stupidities that are on my TODO list, and after thinking about them I often reprioritize my development to fix the worst ones first.

I'm sure there are other benefits as well, but those are the two that I'm really noticing.

I bring this up because I spent a few hours writing some Cartwheel install instructions. A guy down at UT San Antonio, Michael Edwards, has spent a week trying to get everything working. His e-mails inspired me to spend some time documenting, and as a bonus he's also found a few out-and-out bugs in the process. The documentation is still a work in progress, but I'm kind of amazed at just how much software I had to get working in the first place... here's a temporary link, just for posterity's sake.

Anyway, back to documenting...

--titus

cxtest popped up in a /. interview with someone from Crossover Office a while back, and I duly recorded it as something to try. I just checked it out from CVS, built it, and tried it out. Their README instructions worked perfectly, despite their dire prognostications, and I got a recorded script of me testing out xmessage.

Very cool.

I wonder how well it scales? I will have to try it out with my (so far untestable) GUI... Combined with Web testing a la twill and C++ and Python code coverage analysis, I may finally have a soup-to-nuts testing setup available for my application suite.

--titus

I spent most of June and July (6 weeks total) at Embryology Boot camp in Woods Hole, MA; came back to California after stopping off in NY and Minnesota to visit parents & relatives; and then went to Antibes, on the Cote d'Azur / French Riviera for a week with my in-laws. I've been back just over a week, and I spent most of that week dealing with long-neglected tasks and restarting my research work.

Embryology Boot Camp

The Embryology course was really amazing. It was an intensive 6 wk intro to animal development; we covered worms, flies & other arthropods, four vertebrates (chick, mouse, Xenopus, and zebrafish), ascidians, sea urchins, several annelids, ctenophores (comb jellies), cnidarians, clams, other lophotrochozoans, and more clams. The format was 2-4 hours of lecture a day, interspersed with 6-12 hours of lab work, 6 days a week; each day usually stretched from 9:30am 'til 2am. Drinking -- occasionally heavy drinking -- was part of the course (par for the course? ;), as was late night swimming and almost weekly parties.

The instructors were all excellent. Each lecture was typically given by a different professor, and the professors were among the best in their field. We usually had an hour-long Q&A session after each lecture, so we could really get into their research. And in general the lectures were coordinated with the labs, so immediately after hearing about e.g. chick development, we could go into the labs and work with chick eggs.

The lab part of the course was stunning: each group of teaching assistants brought reagents and technique handouts, and were available throughout the lab period to answer questions and help with techniques. I've never had so many expert biologists waiting hand and foot on me ;).

Zeiss brought several million dollars worth of microscopes to the course, too, so I could play with scopes that I simply don't have daily access to even here at Caltech. They did keep on breaking their own scopes by swapping out hardware, but that just means that I'm now somewhat expert at troubleshooting their microscopes!

The best part of the course was the other students. There were 25 students total, and I got to know the other 24 students really well. It was definitely a boot-camp atmosphere, with the attendant intimacy and group dynamics. Good stuff. I expect to be seeing 'em all again over the next few years.

It's hard to really explain how great this course was, but that's enough gushing for now ;). I'll post some pretty pictures from there over the next few weeks.

Antibes and the Cote d'Azur

If you've been there, you understand. If you haven't, go. The Nice/Antibes/Cannes area is fantastic. We stayed at the Three Palms, which I highly recommend.

For-work & for-fun programming

I'm slowly catching up on stuff. There are some twill issues that need to be dealt with. I'm also working on updating the Cartwheel documentation to fix the problems that an installer has been reporting. And sloooowly but surely I'm picking up the pieces of my research from before the summer: PhD or bust!

--titus

Obviously I've broken planetpython.org (or, more precisely, my RSS feed from advogato has). I've asked the Planet Python maintainers to remove my feed until I figure out something else; I may just post via the socal-piggies blog for the summer.

How frustrating.

In real news...

twill 0.7.1

Coooooome and get it! Announcement, docs, download.

This is basically a bug-fix version with about 15 different minor fixes. Several users requested things like HTTP basic auth handling and file uploads, and I patched several bugs in the underlying packages (urllib etc.) Internal ugliness was fixed or augmented, depending on whether or not I was dealing with forms, which is inherently ugly code.

Once I've got all the desired features in there, I can start cleaning up the internal code; right now I'm focused more on testing what's there (which will make cleanup easier!) I've also gotta figure out a way to do meaningful unit tests. That's for v0.8.

For the summer, I will be gone. I predict I will not be working on twill until I get back in late August...

--titus

dear advogato diary, please stop sending out empty messages from me with recent modification times. this is spamming people listening to my RSS feed.

thank you, --titus