Older blog entries for apenwarr (starting at number 26)

This weekend, me and a few friends went to see a short film. It was a pretty good film, but that's not the point of this story. After the (free) film screening, there was free wine and sushi for all comers, because it turned out that this was the first ever showing of the first ever film by this particular director, who was also the main actress in the film. Neat.

I stood around for a while among the wine-drinking sushi-eating people, which included my friends, and eavesdropped on some conversations. The director, meanwhile, was drifting between groups, asking people what they thought about the film. I noticed that she replied to almost every compliment by complaining about the contrast - it wasn't how it looked in the pre-screening; she couldn't decide if it was her own fault or if it was something fixable on the projector. I, of course, hadn't noticed anything about the contrast, which seemed fine to me, insofar as whites were mostly distinguishable from blacks.

Me and my friends also discussed some other things we noticed in the film, such as the colour, the use of different imagery, and so on. The film meant different things to different people, which is normal in an artsy film like this. I memorized some of those things.

Eventually, I gathered up the courage to greet her myself. I started: "Hi, nice to meet you. I thought your film was really good - but I found the contrast particularly interesting. Did you do it that way on purpose?" Well, that certainly got her attention. I then tried out some of the post-analysis I had gathered from my friends, and I could tell she was pretty impressed. I knew this because she asked, "Are you studying film?" "No, I'm a computer engineer." "Wow, well, you're certainly observant. Most of the people here just give me empty compliments and tell me how nice it was, but you really have an eye for detail."

I'm not, of course, particularly observant, and I still don't know anything about contrast. The point of this story is that I don't have to. I do know about other things, and I learn more every day - and against my will, the sum of this knowledge gives me the ability to lead, mislead, encourage, discourage, all by putting the right words and actions in the right places. I used to avoid people and play with computers because they were so predictable, while people were incomprehensible. I'm always disillusioned to find out that people can be understood and controlled just as easily. And it's much easier than programming, actually. Perhaps that's the saddest thing of all.

Prototype-based object-orientation, component interfaces, first-class regexes and iterators, anonymous functions, a built-in HTML parser and streaming multi-connection, asynchronous HTTP retriever, an event-based programming model, and a built-in standards-compliant UI library (ie. the web browser). Not to mention a worldwide repository of open-source sample programs.

If you're like me, everything you think you know about Javascript is wrong. Last week I spent a few days actually reading through the latest relevant standards (HTML4, CSS2, DOM2, Javascript 1.5) and writing some test programs ("html pages") to try them out. I discovered two amazing things:

- IE 5.5, Mozilla 1.5, and Konqueror 3.1 all mostly comply with all three standards. There is a common useful subset of those standards supported just fine, without random variations, by all three browsers. (Give up on Netscape 4 support. It's stupid. Nobody uses it anymore. If they do, just send the losers home.) By the way, Javascript is implemented just fine in all browsers. All real-life problems are mistakes in implementing CSS2 or DOM2, not Javascript itself.

- You can write useful programs using Javascript and DOM2, and those programs aren't even ugly anymore. Really! I mean, you can say things like "document.forms.myform.mybutton.background = 'blue'", and you can pass buttons, forms, tables, rows, columns, or anchors into functions or objects that do things to them. How much more sensible can you be?

In the process of my testing, I wrote a function that easily sorts any HTML table by any column you click on, another function that automatically applies alternating CSS styles to alternating rows in all tables in your document, and another one that retrieves an HTML file, parses all URLs from its content, and dumps those into a listbox. It took a bit of work to "discover" the subset of DOM2/CSS2 that is supported by all browsers, but in the end I didn't need *any* weird browser-specific conditionals to make it work. You hear me? None!

Kudos to Microsoft

Congratulations to Microsoft, by the way, for getting more than 80% of the people in the world to use IE 5.5 and higher, and thereby nearly eliminating the need to support any non-DOM2/CSS2 browser. Microsoft may or may not have gotten their browser share by being evil - but they got people to *upgrade* their browsers by being amazingly good. IE 6.0 isn't noticeably slower than IE 4, it runs on Win98 and later (and I think IE 5.5 works in Win95, even), and upgrades are free. Those things put together mean that we can have what Mozilla promised but never delivered - a world in which everyone of interest is running a standards-compliant web browser.

CSS is designed by idiots

While Javascript and DOM2 are works of genius, I have nothing at all good to say about CSS. Without going into too much detail, I'll say that the following is an extension, added in CSS3 (yes, version 3.0 of CSS, which nobody supports yet) to allow odd-numbered rows to have a different style than even-numbered rows:

tr:nth-child(odd) { /* stuff */ }
tr:nth-child(even) { /* other stuff */ }

That's right. CSS is so bad at representing what it wants to represent that it needs a *special extension* just to be able to identify even/odd rows. Javascript is simple, easy, scriptable, and because of the lousy tag-matching model, more browser CPU time is spend rendering CSS2 than running Javascript that does the same thing. Somebody deserves to be shot.

As I learn again every day, the art of management is not really about telling people what to do. It's about making people think the way you want, so that they automatically do the right thing without even consulting you about it. A little nudge here, a little nudge there. Politics, I guess, is even more subtle.

And then one day, you notice that big parts of the world are doing exactly what you want without you nudging them first, and it starts to seem a little suspicious. Who, exactly, is the one being managed?

...and don't even ask me about the Chinese accordion guy.

Last week it was Iraqi Hip-Hop Bubble Tea, and now this. My life is definitely too weird.

In my continuing attempts to make software testing at NITI sound as cool as it is, I've revised the job descriptions again. Perhaps you should apply as an Evil Death Ray, an Evil Minion, or an Evil Visionary. Jobs available in our Toronto and/or Montreal offices.

Oh, yeah, and we're still looking for coders too.

After reading a few semi-depressing books and articles about the way standard-economics-as-taught-in-school doesn't actually work (because almost all markets are inherently imperfect), and mostly agreeing with them, I realized that right now in Montreal I'm surrounded by examples of perfect free-market capitalism: restaurants.

Some background: some major reasons capitalist markets fail are incomparable goods and high cost of entry. For example, I can buy cotton from just about anybody with cotton, but CorelDraw just isn't the same as Adobe Illustrator. They're similar, but I can't only compare them on price; I have to decide which one I want more. Furthermore, some guy off the street can't just walk in, make a better graphics package, and sell it to you: he has to spend a lot of money (the "cost of entry") first. One big reason Microsoft can maintain their monopoly is that the cost of just getting started competing with them can (and does, regularly) put just about anyone out of business.

So basically, the software market is depressing, because I only took Economics 101 and it can't be explained by that. In my continuing attempts to simplify my model of the universe, however, I moved to Montreal, which brings me to my point: restaurants. Useless trivia: Montreal is second only to New York City in North America for "number of restaurants per capita", ie. the number of steps you have to walk down the street before stumbling into some food.

There are all imaginable prices of restaurant (and at varying quality, of course), and lots of each type, so they have to compete on price. You can buy as large or small a quantity of food as you want. You can start a cheap restaurant and quickly move up in the world, so the cost of entry is relatively low. And the market, in action, is amazing: just watch the consumers get sucked dry! No matter how much money you have, you can spend every last penny, no more, no less, on food, and still feel good about yourself, as this example shows. If you normally buy groceries, then eating out is a treat. If you normally eat at Burger King, then eating out anywhere else is a treat. If you normally eat at cheap diners, then eating at a nicer cafe/bistro is a treat. If you normally eat at nice bistros, then eating at a fancy restaurant is a treat. And if you always eat at fancy restaurants, well, Canada has a lovely progressive income tax system.

Yay, after all these years, my popularity-contest has made it to slashdot. Of course, most people seem to have assumed the code is totally dumb; they missed the fact that it only votes for packages that you actually *use*, not just the ones you install, so it's not inherently wrong. There are also lots of silly comments about how you shouldn't choose your packages based on popularity... well, maybe that's not how you should decide what to install (and maybe it is), but it makes perfect sense for the most popular packages to be on the first (and maybe only) disc!

I have to say, the people who took over popcon from me (since I'm lazy) do seem to have put a fair amount of good work into it. At least, they have cute graphs of the results.

Update 2004/03/29: Italicized "inherently." It just needed it.

I'm in Germany right now at Chemnitzer Linux-tag. Chemnitz is a town, and Linux-tag means "Linux day," which is slightly wrong because it's actually two days. The going theory for the misname is that the correct, pluralized name would be "Linux-tage", but we stupid anglophones would probably mispronounce "tage" much worse than we mispronounce "tag", so they leave it the way it is. Plus it might have been only one day, once upon a time.

Speaking of stupid anglophones, there are only two of us here (me and dcoombs, who is doing a talk on WvStreams), in a conference that some estimate has about 1800 German-speaking attendants. This is strange. In fact, I was getting really worried at first about the fact that we spent a lot of money to fly us here and neither of us understands, well, any of the talks. However, pphaneuf was totally right: the point of a technical conference isn't the presentations at all. It's the conversations you have with people outside of the presentations. And I've learned a lot by talking to people:

IPv6, Itanium, Dylan, Subversion, Non-sucky X Compression!

It turns out that IPv6 sucks much less than I thought, and Intel's Itanium sucks much more. Both of these revelations came to me from Matthias of symlink.ch, who showed me seamless, painless, easy IPv6-over-IPv4 and told me about his actual experiences programming efficient Itanium assembly language. So in both cases, he ought to know.

I also had an interesting discussion about the Dylan programming language. Tim Pritlove, a big proponent of the language, told me why it was theoretically so great, and I bounced him some tough theoretical criticisms (thanks, slajoie and pphaneuf, for providing me with most of these) that he mostly answered satisfactorily. Of course, nothing proves a language like actual real-world use, and, well... it doesn't have any. Still, very educational. I might try it out and see if the only reason it's not popular is that people like me don't try it out.

Other bits of wisdom: Subversion seems really good, except your whole repository is in one big, probably corrupted bdb database. Nomachine has now GPLed the important parts of their very excellent (I tried it months ago) NX X/VNC/RDP/etc protocol compressor. If you've tried other X protocol compressors before (eg. the worse-than-useless LBX), forget them; this is the real deal. And now you can get away without paying for it!

But most importantly of all:

German Bathroom Construction

German bathrooms are totally awesome. And I mean that not in the, "Yo, d00d, that awesome stuff totally r0x!" kind of way, but rather that when you experience one of these bathrooms, you are actually in awe. It's the kind of awe that makes you think, "You know, if me and this bathroom got in a fight, the bathroom would definitely win, big time."

In Canada, bathroom stalls simply don't lock properly. Don't get me wrong - they all have locks - but the locks, when they aren't actually broken, almost never line up with their sockets. This can't be just shoddy construction - it's shoddy construction combined with some kind of fundamental misdesign.

All German bathrooms, on the other hand (and let's be frank: I've tried a lot of them now, so I oughta know) are built like tanks. Not coincidentally, the Germans also invented tanks. The doors are super-solid, always line up perfectly, and have double locks. "No, I don't think that's locked enough, let's turn it one more time so it's extra locked!" And you're just not going to get out until you unlock - twice. I know, because I'm locked in my hotel bathroom as I write this.

In fact, all German construction seems to be this solid. You get the distinct feeling that anything built by a German is simply not ever going to fall down. And then you realize that we - yes, we, the people with the vastly inferior bathrooms - were actually at war with these people, twice, and somehow they lost. It's hard to believe.

So, rather than doing real work, I was talking to pphaneuf at work about RSS and NNTP the other day after reading Simon Law's comments. pphaneuf convinced me that Simon was essentially correct; RSS is nothing new, and most RSS "aggregators" are pretty sucky, and usenet newsreader technology is mature, established, and generally pretty good, so why not just make an RSS-to-news gateway and read your newsfeeds that way? Great!

So I did. Well, that was easy. Add noffle, 52 lines of perl (yes, I did do serious XML "parsing" in there using some really dumb regexes - what's it to you?), 15 lines of shell, and a few automated wgets, and there we have it - instant RSS-to-news gateway. (Anyone wanting my crappy script is welcome to it - just email me.)

The next step is to go track down one of those mature, excellent, efficient, user-friendly newsreaders and enter RSS heaven. Right? Right? But wait! I forgot! Newsreaders suck too! In the last hour or so, I've tried knode, gnus, mozilla-mailnews, pan, slrn, a horrible mutt-nntp hack, Outlook Express, and manually telnetting to the NNRP port. I have to say, the telnet is looking pretty good right now.

The all-time winner for worst newsreader UI ever is of course gnus, but mozilla-mailnews comes a close second. At least the others make some attempt at making sense; mozilla just kept on trying to connect to a host named 'news', which I didn't configure anywhere. slrn actually worked like I wanted, except it couldn't display html (which all blogs are written in, of course). Outlook Express, while annoying, was the sanest one of all, and it even has a cool "offline" mode that I don't need. Could someone just go and clone Outlook Express News for Unix, please? Hint: when I go to news://hostname, why not pop up a list of that server's newsgroups, and stop asking stupid questions about my "identity"?

Anyway, I think this finally explains to me the massive growth of so-called "RSS aggregator" apps compared to newsreaders lately. It's very simple: all newsreaders, like all RSS aggregators, are horrendous, and therefore people keep rewriting them. Maybe someday they'll get it right.

For my next adventure, perhaps I'll try an RSS-to-IMAP gateway. At least I know some reasonable mail clients exist.

More Curses

I realized that my ESC-is-a-key-and-a-sequence problem can be solved simply by making my own ESC key send something other than ASCII 27; after all, if I'm doing the "liberal" thing with input, I can have both ASCII 27 and my new ESC be treated as KEY_ESC. The 27 has the stupid delay, and my new key won't. Essentially no apps other than curses ones use the ESC key as input anyway, so this wouldn't be so bad.

As for people with their DEL key mapped to 127, well, they can just suffer with having their DEL key act like backspace. They probably won't notice anyway.

Okay, so I finally did it. As part of a project I was working on at work for the last few days, I decided to ignore curses entirely (for various reasons, most of them bad; leave me alone), and in the process, the library I wrote solved two problems:

- regardless of your TERM setting, it displays correctly (and I mean perfectly, modulo the lack of colour in win9x telnet) in the Linux console, xterm, rxvt, putty, minicom, Win2k telnet.exe, and - yes, really! - in Win9x telnet.exe.

- regardless of your TERM setting, in all of the above programs, my HOME, END, PGUP, PGDN, and INSERT keys work as they should (except for programs which happy refuse to send those codes at all - notably the Win2k/Win9x telnet programs).

How did I do it? I did what curses and ncurses never did. I followed the first and most important law of successful communication, attributed to Jon Postel: Be liberal in what you accept, and conservative in what you send.

My program sends only the most basic VT100 codes: gotoxy, change colours, change-to-wacko-line-drawing-font. We could be fancier, but then my output wouldn't work everywhere. Be conservative!

On the other hand, it accepts *any* of several possible codes for HOME, END, etc. Every stupid bloody terminal does it differently, and I don't care; I'll take them all. Nobody who says ESC[7~ doesn't mean HOME, even if not everybody who means HOME says ESC[7~. Be liberal!

Of course, I don't support non-basically-vt100-compatible terminals. Now first of all, I don't care, because (hello, join the 1980's!) there aren't any. Secondly, nothing stops curses from doing my basic-vt100 thing by default, and different things if you *do* set your TERM specifically. You're the weirdo, you go suffer. Unfortunately, curses stupidly tries to do something optimal by default. Well, this is to cut down on wasted output, you'll say. Remember 2400 baud users, you'll say. I want my email reader app to be legible in this crappy terminal emulator without spending three hours trying to guess which of the 5 million 'xterm-*' terminfo settings is the right one! ARGH!, I'll say. This isn't so hard. Be conservative by default, and if I'm a weirdo with a 2400 baud modem, I can set TERM to something more efficient. Easy. The "output conservativeness" problem is only a fault of the people who write terminfo databases, so technically we won't blame curses for that.

Unfortunately, for input, curses made a fatal mistake: the terminfo format *itself* has a one-to-one mapping between escape sequences and input codes. There cannot be more than one HOME. That means, basically, there is no way to "be liberal in what you accept". This is a fundamental design flaw in the terminfo file format, and AFAIK you can't fix it in a backwards-compatible way. But you can still fix ncurses. I'd be more than happy if someone would just finally do so.

There are two flaws, however, that I haven't solved: the ridiculous "ESC is a key and also an automatic sequence", bug, that means pressing ESC to cancel a dialog is essentially never going to work right. And there's the ridiculous "nobody knows what code backspace is" bug, that originally (ie. in a VT10x/VT220) was never a problem, but eventually someone (I think it was the X Consortium) mangled completely by sending ASCII 127 for the keypad DEL key. There's no saving people with that keyboard mapping (backspace->8, DEL->127), and unfortunately there are a lot of those people. But I can save everyone else, because CTRL-H is backspace (shut up, emacs users), 127 is backspace, and several things like ESC[3~ are DEL.

There. I'm glad I got that off my chest. (I think I followed pphaneuf's rules for flaming because I went and implemented something better before I flamed the crappy library we poor losers have been suffering with for decades.)