Older blog entries for titus (starting at number 61)

Selenium

Grig Gheorghiu pointed out Selenium, a Web testing framework. Does anyone have any experience with this? Please let me know... - thanks!

Gormenghast

Ankh, I'm not sure what to make of your comment about turning on heels... ;) I've read parts of the Gormenghast trilogy, and I love the atmosphere that Mervyn Peake creates. What can I say, titus is my real name!

As far as whither XML... My Cartwheel/FamilyRelations software can communicate between client & server using an XMLish data format, and I recently extended it to save/load this format from disk. This way people can load analyses offline. The only real problem I ran into was that the files can contain large analyses & I ended up doing a Bad Thing and encapsulating the analyses as blobs within the XML data. Not sure what I else I could have done; using XML properly would have made the files 50 times larger.

I spent some time writing Java, Python, and C++ parsers for the format, but then realized that until I made the system more generally useful no one was going to care but me. So, I adhered to Rule Y and just built the libraries without writing a detailed spec. So no DTD, no spec, just an internal feature format that could be regularized were anyone interested. Which they're not ;).

I've spent more time on the RPC route in recent years, and am now stuck on the fence about the future. My next set of features could depend on XML-RPC to communicate data, but then the data can't be accessed without a server connection. This seems like a bad idea, but the overhead of writing the XML I/O functions also seems unnecessary at the moment.

But I'm sure that's more than you wanted to know...

--titus

Random Miscellany

Undefeatable spam. (Well, at least without real AI.)

Public posting of genome shotgun data can lead to the discovery of new species, sequenced unintentionally. Heh.

CatchUp! Record the refactoring of your API, then "play back" the refactoring on dependent applications. Very cool!

WaterBot. I don't get it.

Corollaries and Laws

My first rule of thumb (a.k.a. "law that I will force into every technical discussion, irrespective of relevance") is this:

Rule Z: A marshalling library isn't complete unless A = load(marshal(A))

A corollary is this:

Corollary Z_1: An analysis program isn't usable by other people unless it is possible to build a complete marshalling library for the output.

This is one of my bet bioinformatics peeves: people (or large institutions) who build otherwise useful programs that have an utterly nonregular output format, rendering these programs largely useless for pipelines.

Corollary Z_2: A list of tab-delimited lists is not a real file format.

'nuff said there. One day I hope to be inspired enough to write a rant about this; my title is already planned out: "You wouldn't use Excel as your database, so why are you using GFF?" (GFF is a simple tab-delimited format that everyone uses in bioinformatics.)

Another rule of thumb:

Rule Y: Premature standardization is the root of much evil.

Writing data export specs before you've talked about use cases with at least three other groups is one flagrant contravention. Solidifying APIs without actually developing a real application that uses them is another.

Prevayling Stochasticity

Via Max Ischenko, the Pyrasun rant about Prevayler is pretty wild. Mike Spille seems to have some serious objections to Prevayler, and after reading the interview with Klaus Wuestefeld I get a bad feeling about it, too. It's not like this guy Klaus needs any more publicity -- even (or especially?) bad publicity must feed his ego -- but there are some choice quotes in the interview:

Q. "Who are you?"
A. [ ... ] "That is, I'm an nonconformist. [sic]"

and

Q. "What would you say to those people who own this kind of application [SQL app], and want to migrate to OO using Prevayler?
A. " Initially, they will feel like Neo, floating on that water container, completely atrophic, for living their entire life inside a database bubble."

and

Q. "Have you ever been confined in an asylum?"
A. "In reality, I'm just a prophet."

Wow. Well, if attitude and arrogance equalled credibility, this guy would be even more credible than me.

OK, I lied, I didn't enjoy it that much ;). insom points out that Market Forces wasn't really very good. The memory has dimmed, but I do recall thinking about 50 pages into it that I shouldn't continue. (Rare, for me.) I finished the book anyway. Things perked up a bit towards the end, but I did feel pretty icky about most of the book. I think I was averaging over my memories of the Takeshi Kovacs books & Market Forces when I said I enjoyed it...

Any other British authors you can recommend, insom? We only get the pretty decent ones over here, and (as you can tell) I'm not really that discriminating in my reading ;).

But back to programming...

Patching wwwsearch/ClientForm to do my bidding: clear(). One more old task from December bites the dust.

ClientForm is pretty nifty, in a functionally hacky kind of way that I find typical of code that has to deal with the Web (prolly due to Postel's Law). It'd be nice to be able to do a round-trip, tho, where you can parse a form into an object and then spit the form back out. That gives you a pretty easy way to test the code, too...

I think I need to make this a "law":

a parsing library isn't complete unless the parse function f and the write function g can be chained indefinitely with no loss of information, e.g. g(f(A)) = g(f(g(f(A))))

Or, more broadly,

a marshalling library isn't complete unless A = load(marshal(A))

Not very pithy, but then I'm not a pithy kind of guy, unfortunately.

--titus

Richard Morgan's books

avrietta, I love Richard Morgan's books... Market Forces, the latest (due out in the US sometime soon) is not as good as the Takeshi Kovacs books, but I enjoyed it nonetheless. You might also be interested in Marc Matz's "Nocturne for a Dangerous Man", or at least the free association part of my brain thinks so.

I picked up the Market Forces book in Canada on a trip up to Vancouver. Vancouver is one of my favorite places to visit, because of the fantastic scenery, the great cuisine -- I've eaten at a number of places in the city area, and have always had a great meal -- and the British publishing scene. Everytime I go I come back with about $100 (CAN ;) of books by people like Jon Courtney Grimwood, Iain Banks, and Richard Morgan. I usually can't get these books in the US for at least a year after their publication in England, presumably for publisher-related reasons. The swine. Now that I've added three new Brit/Can fantasy authors to my "buy on sight" list -- R. Scott Bakker, Steve Erikson, and China Mieville -- I'm pretty upset about this situation. Hey, any Brit/Can people want to get involved in a book exchange program?

--titus

(This diary entry courtesy of synchronicity between avrietta's post on Richard Morgan, Ted Leung's post about the Guu restaurant in Vancouver, and my own late-night completion of "The Warrior-Prophet", R. Scott Bakker's latest.)

p.s. There's a pretty darn good Native American restaurant in Vancouver that my then-girlfriend/now-wife and I visited when we were up in the area for ALIFE 8. Anyone know the name? The ambience was fantastic!

A Generic Python Blog Entry

Using decorators to make Python free-threading through bypassing the global interpreter lock doesn't require static typing, so why does Guido like the '@' notation?

Net, the game

Ohayou points out Net, a moderately addictive Flash game. It seems like there are a set of simple heuristics you can use to guarantee a minimum-move solution, though. This is because the map is constructed so that the correct solution is parsimonious. Not sure what to make of Ohayou's "best - 1" solution, although I guess I can imagine one way that might happen.

--titus

"When the going gets weird, the weird turn pro"

Hunter S. Thompson, the original Gonzo Journalist -- 1937 - 2005.

RIP.

I respected his writing immensely & thought he was one of the few people capable of making sense of the crazy politics of the late 20th century. His masterwork is "Fear and Loathing on the Campaign Trail '72", and two other highly recommended books are "Hell's Angels" and "Fear and Loathing in Las Vegas". His collected letters are also very much worth reading.

Inveterate Doonesbury readers may know that "Duke" is modeled on HST.

I was planning on holding a projector party to show "Fear and Loathing in Las Vegas" and "Breakfast with Hunter" next weekend; looks like it'll have to be a memorial party instead. Sigh.

--titus

p.s. The Wikipedia is really on the ball, I have to say!

p.p.s. Wonder if Doonesbury will do a memorial cartoon?

Groupware

"""
Dude. It is possible to get some [sic] laid by creating a really really really really really really useful software product. Take single sign-on for example.
"""
(ref)

Being married, this is less of an incentive for me, but I can see how it could work. ;)

That jwz article is full of good quotes. I also like:

"""

Then the next thing you want is an invitation manager like Evite but that doesn't suck. Evite sucks because they're spammers, and because it's more important to them to put advertising in front of your eyeballs than to be useful, so the mail they send out doesn't actually include any information, in a lame-assed attempt to drive hits to their web site. So what you want next is a free replacement for Evite -- but more to the point, one that doesn't require any kind of server running anywhere.
"""

I'm not smart enough to figure out how to do this, but it does nicely sum up my feelings about Evite.

Reputation systems, certs, etc.

A few days ago I begged for links and comments on certification and reputation systems. I only got one reply, from Steven Rainwater. He mentioned robots.net as a more up-to-date site running mod_virgule. He also said that he thought the user interface for cert systems that he's seen is a large part of the problem, and that users frequently don't seem to understand what the certs are or what they mean.

Here's a handy link to his Web page on mod_virgule.

I also found this O'Reilly Network article called "Annotating Everything", which reports on Marc Smith's work on Aura.

Finally, on the subject of annotating everything -- CleverCS had a cute article on acquiring linguistic/world knowledge from volunteers. Looks interesting, although clearly I'm not smart enuff to understand their prototype app. But maybe you are...

Again, if you have links to effective annotation/reputation/certification/contribution systems, please send them to me! Thanks!

Shotgun Golf

In response to no hockey season, HST introduces Shotgun Golf:

"""

The game consists of one golfer, one shooter and a field judge. The purpose of the game is to shoot your opponent's high-flying golf ball out of the air with a finely-tuned 12-gauge shotgun, thus preventing him (your opponent) from lofting a 9-iron approach shot onto a distant "green" and making a "hole in one." Points are scored by blasting your opponent's shiny new Titleist out of the air and causing his shot to fail miserably. That earns you two points.
"""

(ref)

--titus

[ sorry, dup entry ]

Google knows Everything

My wife found this RISKS digest with a post from me in 1995. Well, I still agree with the old me that having a command-line interface is a good thing ;).

socal-piggies

Grig is organizing a Southern California Python Interest Group, and, as usual, I'm sticking my nose into it. We've set up a mailman list.

Artificial Life

The Discover article I mentioned is online at the author's Web site. (It was also on /.) I do believe that they're working on a scripting interface for Python, partly due to my proselytizing, but I can't find any mention of it on the Web sites; I'll ask.

Interview with Hunter

This one is from before Fear & Loathing in Las Vegas came out.

Beta, beta, beta

zdnet has an article about how beta software "releases" are becoming increasingly prevalent. Google, of course, would be the case in point. Of course, this is part of google's release strategy: "underpromise and overdeliver". Web apps really can get away with this, because it requires no user installation to upgrade to the latest version.

More WSGI

Implemented wsgiMemcached and wsgiPullAdvogato. So, you can now do this:

app = wsgiPullAdvogato.wsgiPullAdvogato()
app = wsgiMemcached.wsgiMemcached(app)

and you'll have a Web app that pulls down posts from Advogato via XML-RPC and caches the results via memcached.

The advogato pull is simple and error-prone, but functions; the memcache middleware works fine, but I'm unhappy with its implementation as middleware. As is, it requires that downstream apps be able to report if a given URL is stale; otherwise the caching system doesn't know when to recache an entry. While I don't think there's any way around changing the app to support caching, right now the system breaks if the downstream app doesn't have this function. After sleeping on it, I think I'll try implementing it as a service passed in via the environ dictionary. Maybe that will be cleaner...

Hmm, then I can write another piece of middleware that looks at the If-Modified-Since headers, groks the cache, and decides whether or not to send. Hmm. (Here is a simple article on HTTP Conditional Get.)

This pipelining ability of WSGI is really nice; I'm constantly thinking of new uses for it. The ability to deploy a Web app in multiple environments by changing only a few lines of code is also very nice -- I did some benchmarking last night, and I switched between CGI and SCGI serving in about 30 seconds.

--titus