Recent blog entries for laburu

Advocating Free Network Computing

Recently, I nearly allowed myself a frenzied rush to the keyboard because I thought I had found a good way to spend the $5M grant announced by the Knight Foundation for an open-source technology-based project. The pitch was so simple in its conception and so compelling in its humanity that one could almost hear Sally Struthers delivering it. Unfortunately, Sally wasn't available for a YouTube cameo, and Stephen Fry was busy celebrating GNU's birthday, so my own rendition will have to do.

Here, in Switzerland, there are people who still live way up in the mountains, in remote locations that are very hard to reach and not at all profitable for utilities to serve beyond the level required by law. For them, fast, affordable network computing is a pipedream — or would be if they knew what benefits it brings. If this sort of scenario doesn't justify investing in the development of a "One Laptop per Person" (OLPP) device with a free software-defined radio and custom ASIC, what does?

Fortunately, I snapped out of my self-satisfied trance. I realized that, aside from looking ridiculous in a blonde wig, I was already "late" hours before the deadline, and that a project like this, when not properly executed, might have turned into a very public failure for skeptics to gloat over. And I'm telling you all this because what I figured out may well apply to you and your region, too.[0]

To start with, there is the issue of my own authority — or, rather, the lack thereof. Knight is almost certainly looking for an outfit with a track record of working successfully with local authorities to deploy this sort of solution. I contribute my time, money, and effort to various public interest causes, but I am not employed in any capacity by any of the organizations that would see the grant, and neither they nor Knight have reason to take my word that a project of this magnitude could be completed on this budget. I just don't have that kind of cred.

Also, I wondered what level of cooperation one could expect for this type of project from local institutions in a country where the government has a history of engaging in corporate welfare schemes ranging from ludicrous concessions on basic infrastructure to ill-conceived bailouts à fonds perdu. One can hardly expect these remote communities to take the leap of faith that an OLPP project would require without first having seen "free" work. Why should they damage their relationship with powerful utilities, on whose beneficence they depend, over a project that could fail?

So, instead of writing a brilliant proposal, I am doing what I should have done years ago: looking for opportunities to work with and for regional institutions to help bring affordable network computing and computer literacy to people I can assist materially, advocating whatever free solutions are available and suitable for this purpose. In other words, I am going to "sell" Luke Leighton's fully-free OLPP device locally, on a very small scale, as an abstract goal rather than as a concrete product. Because a product can be made to fail, but the dream of a free world has proved notoriously hard to defeat. And people inspired by a dream make change happen.

[0] If it doesn't, did you apply? If not, why not?

Enough Hot Air

Once upon a time in the not-too-distant past, a hacker I know blogged about using object-oriented C to implement a lightweight imitation of some of C++'s features for his latest project; almost immediately, somebody saw fit to reward this charming piece of acceptably self-congratulatory writing with a stern and quite public deconstruction. Does this scene seem familiar? Why does this keep happening? And what, if anything, can we do about it? We can hardly hope to appease all of hackerdom's malcontent — but we can at least try to avoid stepping on each other's toes. Accordingly, this article will waste no time on a platitudinous condemnation of the surf-by put-down; rather, I wish to take a moment to reflect on what the hacker did to earn it, and to consider what he might have done (or, rather, not done) instead.

Let me start by stating for the record that I think the hacker's solution was meritorious as such; that is, it solved the problem he formulated, so I approve of it on the grounds of its workingness. The techniques employed in this approach are simple and effective, and were considered best-practice C in the eighties. Some people found this style long-winded, though, and facilities were soon devised to address the awkward verbosity of the incipient object-oriented C idioms. The pre-processors that implemented these and other improvements to C went on to become distinct languages (namely, C++ and Objective C) and communities formed around them.

Or perhaps it was the other way around, says the sociologist looking over my shoulder — but that is neither here nor there for our purposes. The point I want to make is this: one can reasonably expect the people who embraced these solutions to wince when someone promotes the use of the very idioms their community has so laboriously abstracted away. Did the hacker really not anticipate this response? If he did, why did he preface the presentation of his solution with an explicit rejection of C++?

Whatever his motivation was, it's fair to say he paid for it. Indeed, justifying one's work by scorning a mature system on which one is not an expert works against the legitimate goal of being lauded for one's skills. It puts people off, and will only earn one points with the weenies who happen to agree. Moreover, it is likely to elicit overt retaliation from vigilantes (mostly the weenies who happen to disagree) and a spot on someone's idiot list. Indeed, in the aftermath of such a gaffe, any subsequent plea aimed at demonstrating compunction and competence – however unassailable and incontrovertible – is likely to be shrugged off as so much prejudiced ignorance and answered with a curt dismissal.

So here's my twopence-worth of unsolicited advice for programmers who desire the recognition of their peers: share your wins (and your FAIL) and toot your own horn if you like, but try not to steal the wind from another's sails when you do. Because there's more than enough hot air to go around already — certainly enough for every foghorn blowing at C.

I wish to acknowledge the contributions of Nathan Myers, Luke Kenneth Casson Leighton, and R. Steven Rainwater, whose suggestions and encouragement helped make this a better article. Any remaining flaws are my sole responsibility.

This essay started life as a diary entry, but was rewritten and submitted as an article in response to Luke's lamentation. The canonical version of this document may be more up-to-date than the version here.

Why Hackers Write Free Software

Free software has long been celebrated by some as

a highly effective vehicle for the transfer of wealth from the industrialized world to developing countries
— How the Tech-Poor Can Still Be Software-Rich

and denounced by others as

a transfer of wealth from the programmers who created it to the corporations who use it without payment
— Support for open source software is based on several misconceptions

but all of this high-stakes talk can give people the wrong idea.

I would prefer that the general public not think of people who choose to write free software as visionary philanthropists or – worse still – the unwitting victims of corporate exploitation. In fact, I suspect a lot of free software is produced for the noblest of reasons: it pleases the programmers who write it.

In support of my claim, I hereby present the following list of CPAN modules:

Lingua::Tolkien::BURL

an Orkish scripting language

Acme::MetaSyntactic::lotr

The Lord of the Rings theme

Date::Tolkien::Shire

implementation of the calendar used by the Hobbits (via Stefan Nesbitt)

DateTime::Fiction::JRRTolkien::Shire

(re-)implementation of the calendar used by the Hobbits

What, I wonder, might the capital value of this software be to someone in a developing country or to a corporation looking for code it can use without payment?

…

When I think of free software programmers, I think of the so-called birds of paradise — each indulging in its uncommon, seemingly gratuitous[1] display of beauty and grace[2]

simply because it is possible.

The comparison to these animals, several species of which are on the brink of extinction, is not capricious: ultimately, programmers write free software not only because they like it, but also because – despite the relentless lobbying and legal maneuvering of some corporations, and thanks to the tireless vigilance of some organizations[3] – they still can. Let's not take that for granted.

[1] I know, I know: Eric Raymond claims that our exotic plumage and behavior (like the birds') are actually the product of natural selection and, not coincidentally, improve our chances of mating well, often, or at all. OMG! Does that mean I can hack my way to evolutionary fitness? Sweet! *hack* *hack* *hack* *hack*. Uh… no.

[2] I am talking about the "beauty and grace" to be found in program source code, of course; the free software programmers themselves can be as hard to find in broad daylight as the aforementioned Paradisaeidae! [See the foregoing endnote for a possible explanation and an egregious example of FAIL.]

[3] Here are, in no particular order, some of the organizations that make it possible for hackers to continue writing free software:

• Electronic Frontier Foundation
• Foundation for a Free Information Infrastructure
• League for Programming Freedom
• Free Software Foundation
• Software Freedom Law Center
• GrokLaw

Now would be a fine time to show your support.

[canonical version]

An Uncontroversial Reason for Sharing Source Code

In Software Tools, Kernighan and Plauger said that

[g]ood Programming is not learned from generalities, but by seeing how significant programs can be made clean, easy to read, easy to maintain and modify, human-engineered, efficient, and reliable, by the application of common sense and good programming practices. Careful study and imitation of good programs leads to better writing.

This, I think, is a good argument in favor of sharing source code. One cannot easily study and imitate another's program without access to its source code, so we share our code with the reasonable expectation that others will correspond — whereupon, our altruistic behavior would yield a benefit.

A friend of mine (who wishes to remain anonymous on the web) pointed me to Leslie Lamport's unpublished paper on Buridan's principle and expressed her disagreement over the following statement:

[I]f the ball is positioned randomly, random vibrations are as likely to keep it from falling off as to cause it to fall.

I would love to know what “randomly” means here, since we're evidently constrained to placing the ball on the edge of the knife. Does it mean “not symmetrically about it”? In any case, I agree that random vibrations will not keep the ball on the knife's edge: it seems highly unlikely that the effect of a first de-stabilizing impulse on the ball will be subsequently countervailed by a sequence of corrective impulses that keeps the ball on the knife's edge, and even more unlikely that this continues indefinitely.

So, while I grant that it's possible for the ball to remain on the knife's edge in the presence of random vibrations, I also think it's unlikely that it will. Unless we've misunderstood what Leslie is saying here, it seems plausible that this claim about the "ball on a knife's edge" is partly responsible for the paper's rejection, as it is bound to trip most physicists' wires.

Leslie Lamport on Thinking for Programmers

On March 14, 2006, Leslie Lamport gave a wonderful guest lecture to undergraduate students at the University of Lugano's Faculty of Informatics. This article contains what little I managed to commit to paper then, as well as my impressions and my totally subjective recollection of events.

Content

Abstract

Leslie Lamport's Thinking for Programmers was a talk that advocated an approach to software development, based on mathematics, that results in solutions that are succinct, intelligible, and correct.

Complexity

Hardware doubles in complexity every two years; software, on the other hand, doubles in complexity roughly every three years. One way to avert this fate is to rein the sprawl in at the very earliest stage by thinking hard about what we really want and writing it down before we start coding.

Writing

Writing is nature's way of letting you know how sloppy your thinking is; writing down what we want our software to do helps us refine our understanding of it.

The Trouble with Writing

Unfortunately, an informal specification of what a piece of software should do is often not enough to dispel all ambiguities; that requires math. Consider, for example, the description of delete() that one finds in the Java documentation: what should the function do when the file does not exist? Avoiding this kind of trouble requires the conscientious application of mathematical rigor to the specification stage of software development.

Math

Math is nature's way of letting you know how sloppy your writing is; mathematical rigor has proved a solid foundation for other engineering disciplines — why not ours? This notion was first advanced by Robert Floyd, whose 1967 book Assigning Meanings to Programs advocated a new way of looking at computer programs: Floyd proposed that programs be construed and expressed as mathematical relationships between an initial state and a final state. Dr. Lamport illustrated this point by describing how he designed and implemented a prettyprinter for TLA+ using mathematical rules, rather than procedural code.

The Trouble with Math

Though that particular exercise was a noteworthy success, math can give one a false sense of comfort, he said: a mathematical specification written by someone whose math skills are unequal to the task can lead to confusion. Consider, for example, the complications that arise from misapplying UML in reasoning about a system's design. This is where formal math comes to the rescue.

Formal Math

Formal math is nature's way of letting you know how sloppy your math is; inventing new kinds of math that purportedly supercede static, old math is an exercise in vanity and ignorance. Indeed, existing math is up to the challenge; we just need to admit to ourselves that we, software engineers, simply don't know enough math yet. Dr. Lamport then went through the iterative refinement of a mathematical solution of a graph problem, showing us how, for many problems, software bugs are just obvious math trivia. In closing, Dr. Lamport insisted that we must embrace mathematics, and recommended that we read pages 1 through 84 and chapter 14 of the TLA+ book for ideas on just how.

The Trouble with Formal Math

The trouble with rigorous math is that it's hard work; that, however, turned out to be a topic for another lecture.

Form

The Good

Dr. Lamport spoke audibly and intelligibly. The pace was neither too fast nor too slow, and there were plenty of well-placed micro-pauses for reflection. The base knowledge assumed of students (basic discrete math) was reasonable. Entertaining anecdotes motivated and illustrated points well. He spoke his mind freely concerning various alleged best-practices and earned the approbation of the progressive faction of the undergraduates in so doing; his earnestness and his confidence in the value of his approach certainly impressed me. After going through the slides, he answered our questions patiently and generously.

The Bad

I loved every bit of it, but a discussion of the event in the first-year lab revealed that several slides were too deep for some of the students.

The Ugly

It seems Dr. Lamport raised some hackles with his candid appraisal of some current best-practices — though this probably says more about the offended listeners than about the lecturer. Also, he claimed that many in the current generation of software engineering professionals and educators are ill-prepared to teach mathematics at the level required to help the next generation advance the state of the art in our profession — and that may have bruised some egos, too. So, long-story-short: if you want to shake things up a bit, you should call on Dr. Lamport! :-)

Summary

Lamport impressed me as the rare kind of theoretician who, not content with advancing the state of the art, reaches out to practicing engineers in the hope of raising the standard of so-called best-practices. He seemed genuinely concerned about the challenges faced by software engineering professionals, and demonstrated a patient understanding of the many reasons why the pace of progress is slow. Moreover, he seems to be as fond of young people as he is of mathematics, and casts a resolute vote of confidence in them.

[canonical version]

The LSD-Meth User's Guide

I wrote this article as a sort of LSD-Meth Explained to accompany The Lone Software Developer's Methodology. The target audience includes the fun-loving, FPS-playing first-year students at the University of Lugano's Faculty of Informatics. Suggestions on how to improve this half-serious, half-humorous rant would be most welcome.

What LSD-Meth Is

Most people claim that there is a method to their madness; I, on the other hand, prefer madness to method. Accordingly, The Lone Software Developer's Methodology (LSD-Meth) is not really a methodology, but rather – at least when you read between the lines – a rant about embracing madness. No, really — madness is the modus operandum in software development, so you'd better love it or leave it.

When to Drop LSD-Meth

This doesn't mean that you yourself have to be mad to try LSD-Meth, though; it's probably enough that the following short list describe your predicament:

• you can choose what feature to work on next and how to write it,
• you aren't sure what the guts of the finished program will look like, and
• you don't have to waste your time arguing about your decisions.

In other words, if you aren't working for or by yourself, you should drop LSD-Meth like a hot potato. Not that I've ever seen a hot potato dropping LSD-Meth. I just think that, if you have no say-so regarding your code, you're in a bad spot and probably feel like you need to drop something. Your job, for example. But I digress.

Profile of an LSD-Meth User

LSD-Meth is a good choice for people who don't mind

• panicking all the time,
• procrastinating all the time,
• throwing everything together all the time,
• screwing everything up all the time,
• rewriting everything all the time, and
• managing to do just one thing at a time.

If this sounds like you, LSD-Meth is just what you need. Read on to drink my Kool-Aid.

Panicking All the Time

Some time ago, I started thinking about how to avoid bungling yet another hack. The first thing I chose to take into account was The Sacrosanct Deadline — the date on which a final product must be delivered. Now, deadlines are a funny thing because, for all intents and purposes, any deadline is reducible to next week, tomorrow, or even now. (I have found an excellent proof for this theorem that this web page's margin is too small to contain.) Therefore, being the sort to panic when facing deadlines, I figured I might as well get in that frame of mind ASAP and learn to live with it.

Procrastinating All the Time

A corollary of the Truth of the Sacrosanct Deadline is that one shouldn't plan too far ahead. This was a welcome realization because I think planning sucks; a lot of the time, it only leads to disappointment and wasted effort. Seriously, given that you have no time and you're already screwed, why bother? Anyway, the only plan I was interested in was the plan to plan tomorrow. Or the day after. Maybe. So, in this frame of mind, I decided that, instead of planning really well so as to make the best use of my time before the deadline, I would instead avoid as much of the planning as possible and set my aims on the earliest attainable, improved, working system and then do it again… and again… and again… Each time, I thought, I would plan just a little (no more than was absolutely necessary) and get that little bit done, telling myself how I could add the other stuff later if I wanted to or had to; that way, when the real deadline came up, I wouldn't have to lament how I wasn't able to get this or that wonderful idea implemented, and (for once) I could feel good about having put things off.

Throwing Everything Together All the Time

So, having accepted immediate deadlines as a fundamental reality, and needing to harness my panic without a thought-through plan, it seemed consequent to pretend that any given coding session was going to be the last. This meant that every little bit of code would have to be integrated immediately after being written. After all, if I was going to have to submit a working product any moment now, I wanted my product to incorporate as much of the work I had actually done as possible because I don't want to look lazier than I am.

Screwing Everything Up All the Time

Now, such an approach precludes things like multiple release candidates, which are embarrassing, and final testing, which I didn't want to do. Actually, that's a lie: when each build might well be the last, each build is a release candidate, and all testing is final testing. But how might one achieve a final-tested release candidate for each build? With no show-stopping bugs, please? Answer: by writing unit tests that try hard to trip up any and all new code when it is written and running these tests automatically for every build cycle. I don't mind telling you that writing thorough tests all the time seemed like a lot of up-front toil, but I managed to convince myself that it was the sort of up-front work that, in the long run, would save me time: after all, in the context of the project, all new code would be just a patch against an existing code base, so… as I would already have automated unit tests for most of it, I would only ever have to worry about (and write tests for) the very latest code. [Yes, I am a great rationalizer.]

Rewriting Everything All the Time

Actually, automated unit tests did more than take the worry out of adding code to the project: they also took the worry out of changing existing code. That's good because, whenever I write any code – whether it be for a new feature or an old bug – ugliness happens, mistakes happen, and madness ensues. Fortunately, though, not all code is born equal; that is, there is no silver bullet, but there is a silver spoon: once you embrace comprehensive unit testing, any code you rewrite is born to a well-designed interface with associated tests. This was great news: I mean, here was a way of going about business that leveraged my weakness for tackling the easy things first to improve the quality of my code!

One Thing at a Time, All the Time

So, now I had an unexpected problem: how to make adding new functionality more attractive. For me, this meant that changes to the code base would have to look a lot like finishing touches – anything else would seem like too much work – while accomplishing more than just a face-lift. As it happened, the agile development crowd had already found the answer for me: during each release cycle, I would add that single feature which, amongst the immediately attainable, added the most value to the program. Thus, the most important things would be sure to get coded, and each release cycle would be as quick and painless as possible.

A Simple Question

And, so, I came at last to the issue of how to go about implementing a feature. I wondered how one might choose an implementation strategy for a feature; specifically, I asked myself whether, for a given interface specification, any two fully-compliant implementations would contribute the same added value to the system. If the answer were yes, then the cheapest implementation would be the best. But I had trouble believing that.

Software Engineering Methodology as Speculation

I reasoned that, if I only cared about increases in the net worth of the production system achieved during the current and immediate release cycles, the answer could in principle be yes, but that if the choice of an implementation strategy for the current cycle were to bear on the cost of future development, the answer would be no; therefore, I concluded, if one wanted to achieve a high net worth for the finished product, one would have to take into account the cost of all of the release cycles and choose the implementation strategy accordingly. But how could I reconcile that with iterative project scheduling, continuous production, and aggressive scope management?

A Better Question

At this point, I admitted to myself that I could not know enough about the future of the project to choose between two implementations on that basis, and I did not want to engage in sheer speculation; all I had to go on was the code itself. So I asked myself another question: given two pieces of code, each implementing the same interface specification, could I say that one was somehow intrinsically better? Because, if I could say something like this code is, in and of itself, more valuable to me than that code, then all I'd have to do is figure out what makes a piece of code better and just do that all the time.

Software Engineering Practice as Housekeeping

Mind you, I am not talking about the kind of better that means runs faster or consumes fewer resources or anticipates such and such a case or conforms to such and such standard or indeed most aspects of quality discussed in software engineering texts. I mean the kind of better that implies more valuable in the context of my approach. So what would make code more valuable to a fellow in my predicament? The answer is simple: that which makes code the most maintainable. I figured that, when a development methodology includes continuous refactoring, continuous testing, and continuous production as mandatory techniques, it is fair to say that the program is under a sort of continuous maintenance. That's why the patron saints of software engineering are Tim Allen and Lucille Ball.

=:-.

OK, that was pretty lame — and a clear sign that we have reached the end of this retrospective… at least for now.

[canonical version]

Recently, I looked at The Lone Software Developer's Methodology again and (after much soul searching) decided to own up to it. What do you think?