Older blog entries for apenwarr (starting at number 157)

26 Sep 2006 (updated 26 Sep 2006 at 00:02 UTC) »
Delirium 2: Imbalance

Imagine you're in a speedboat testing an experimental new engine with 5x the propulsive power of any engine that came before. Moreover, the lead engineer for the new engine, who is also an expert technician, is there in the boat right next to you, excited, tuning the engine as it runs. "Egads!" he exclaims. "I just found a way to double the power one more time! This thing is really going to go now!"

Impressed, and listening to the scream of his supercharged nuclear turbine as it spins at full speed, you yell out the only obvious question: "So why aren't we moving?"

The engineer, an expert at engines but not much at boating, looks around in dismay. "I guess... it's still not good enough." He turns back to his engine with a determined look on his face. "We just need more power!"

Hours pass. The engine roars louder and louder. The churn beind your still-stationary boat is getting out of control. The engineer, apparently rather motivated, has doubled, tripled, quintupled the power. By now a crowd has gathered around the dock to watch. They all agree that this engine is the most powerful one they've ever seen at this size and weight - by a long shot. Your friend the engineer has discovered at least two new laws of thermodynamics right there on the spot, and people are sneaking up behind him with honourary PhD's. But still - the boat's not moving.

Suddenly a thought occurs to you. "Hey, are we still tied to the dock?" you yell. It's hard to hear you over the wailing engine, but he stops, looks at you, and then turns in the direction you're looking, to see the lines still holding the boat in place. Then he slowly sits back, stunned and silent. He throttles back the engine, and the world gets a bit quieter. You look at your audience, still there watching from the dock. "Why didn't someone say something?" you ask.

"We thought you knew," someone shrugs, not meeting your eyes.


By now the sun is already setting in the evening sky, but it's not too late. You unlash the boat from the dock. "Let's do it," you say quietly, and take the steering wheel. The two of you begin to edge away from land at last.

But before the engine his even 10% throttle, it's already threatening to flip the boat. "It's too fast!" you exclaim. "I can't steer! Slow down!"

Your friend looks back at you, then looks away and begins to throttle back to a normal speed. "What's the point then?" he asks.

"I think we'll need a better boat, and we'll have to find a better driver. An engine this good must be useful... this boat just can't handle it. But the engine is way too good to waste!"

"You don't understand," he replies. "This engine was tuned for this boat. The concepts can all be generalized, but it's going to be a huge amount of work... I guess this particular engine is never going anywhere."

The two of you make your way back to shore. The engineer, looking exhausted and frustrated, gets out and starts to walk away. "That's it. I'm too tired. I'm done here."

"That's it?" you ask. "What about the engine?"

"You can keep it," he says. "I left my notes. If you can make it run half as well as it did today, it'll probably make you rich."

Just don't tie it down.


It's such an odd problem - if all you have is a hammer, everything looks like a nail. If all you have is duct tape, everything looks like a duct. If all you have is an engineering team, every problem looks like an engineering problem. After all, it... has to be. How can we deal with it otherwise?

You know, if we had kept that engineer working long and hard enough, eventually he might have fixed his problem using pure power. Maybe he would have snapped the lines, or torn the dock right off the shore. But the results would still have been out of control, out of proportion, and even more disastrous and disappointing in the end.

Even though this is a sad story though, the point I'm trying to bring up is actually a happy one.

A great "engine" got built, based on new "technology" and "laws of thermodynamics" that we discovered out of necessity. The bad news is that the engine was unnecessary - a merely passable one would have worked just as well, if what you want is a really fast boat. But the innovations themselves were still useful and can be reapplied elsewhere, on a new boat equipped to withstand that kind of propulsion. Someone just has to build the boat.

Those innovations would never have been made at all if it weren't for the original grotesque error in judgement. The engineer will keep those innovations - along with a determination never to make that same mistake - with him, ready to be applied when needed, for the rest of his life.

And if you were in the boat along with him, or even just in the crowd watching, maybe you learned something too.

Delirium 1: The Genius of Obsession

My friends dmg and apm recently lent me a book called The Soul of a New Machine, by Tracy Kidder. It's a work of long form, non-fictional journalism that describes the process of designing a new minicomputer at Data General around 1980. It's much more about the people and their social issues than about the computer itself.

What an interesting book to recommend to me right now. I certainly recognize the story: team of 30 or so crazy people in larger company isolate themselves, gel as a team, build impossible product on crazy timeline, live through pressure, team reorgs, smart and stupid managers, and have way too little real life experience. Product gets completed (a bit late, of course), team leader moves on, and team essentially disintegrates and carries its experience elsewhere, person by person, while parent company lives on, continuing to sell the resulting product and ones based on it. I guess we can be proud - we held together the culture longer than they did.

But the things that really struck me about the story weren't so much the similarities as the differences. They did a few things "right" in that story - for some definition of "right" - that I never quite managed. Their team lead, Tom West, managed to bring out of everyone on his team something that I've only ever managed to bring out in myself personally - total obsession with the end goal.

And obsession, as I learned for sure in the last few months of not being obsessed, is where it's at. Obsession is the only way to really get into The Zone. It gives you energy you didn't know you had. It turns you from a normal mortal into something more. I've done it for myself a few times. I used to wonder if you could even do it for other people, because I never had much luck. But after reading this book, I have a few clues. Not surprisingly, they echo the environment I create naturally for myself - but I have to change my style if I'm going to create the same conditions for others.

Mutual Motivation. You can't obsess alone for long. The key thing isn't so much having help as being needed. You have to not let people down. That fear is what makes stopping harder than going on.

"Signing up." It has to be voluntary. You have to know what you're getting into on day 1, and decide it's worth it. You have to decide it out loud, and promise it to others. You have to be visibly, publicly in the game, so any failure will be an obvious one.

No way out. There can be no escape clause. If you fail, we're all dead. Nobody will save you. If you're late, we're all late with you. There's no superhero who will do it better and bail you out. You're going to have to grow up and be the superhero. If you really have to do it, maybe then you really can do it.

Too stupid to know it's impossible. It has to be harder than you thought it would be, not easier, so the pressure builds instead of dissipates near the end, when it's easiest to lose interest. And so experience kills - you'll never be able to obsess the same way twice. That's why younger people, especially students, are better at it than oldsters. (As someone on the verge of being an oldster myself - in the book, 30 years is the magic age - I already have to find crazier and crazier projects each time, always outside my existing experience, or I simply can't obsess enough.)

Too blind to see it doesn't really matter. In real life, almost nothing you do makes a serious, long-term difference. Realizing this will kill your fun. You need blinders - and if you're lucky, a really good reality distortion field. Again, older people have trouble here, because once you've been disillusioned by dedicating a year to an ultimately pointless project, it's hard to fall for that again. Except that falling for it is fun.

Too restricted to take the easy way out. Smart, sensible ways of solving problems will knock you out of The Zone. Sensible schedules, flexible and forgiving bosses, 8-hour work days, good nutrition, the newest equipment, a social life (outside work), even using low-cost direct mail services instead of composing your own poetry and stuffing your own envelopes - all these things make it too easy. That means less pressure. Less pressure means less obsession. There's a good reason why entrepreneurial companies are responsible for most of the wildest innovation in the world: because they're so helpless, disorganized, and resource-constrained that nothing you do there isn't a challenge. And that makes your brain work harder, nonstop, without any chance to cool down and drop back from hyperspace into normal space.

When I've done the best work in my life, those are the rules I followed. I turned my load factor down to levels I knew perfectly well were too low to be safe. I set things up so I had no escape and nobody could possibly bail me out. I bet my reputation that I would be done on time. And because it's my special skill, I always, always, failed to realize how impossible and unimportant the job was - at least until afterwards.

And that, let me tell you right now, is the magic combination that makes life worth living. From now on, I'm going to be a very different kind of manager. You should be glad you don't work for me anymore!

Delirium 0 (second try): Nerve Center

When you play a computer strategy simulation, there might be hundreds of agents doing your bidding - workers, warriors, transports, cities, convoys, and maybe even angels and demons. They march to your orders, they do more work than you could ever do for yourself, faster than you can see, because the computer handles the details - you just handle the strategy.

But the real world isn't quite like that. In the real world, real minions cost real money, and have real feelings, and want real self-actualization and self-expression. Minions are real people, just like strategists, and minions want to grow up to become strategists like you. But managing the feelings of your minions takes work - it takes away from your game - it weakens you compared to the other players who "feel" their minions better than you do.

What if your real-life minions weren't people, but computers? What if whole aspects of your business didn't need human minions anymore - just automatons with some strategic direction? What if every human was a strategist, and every meeting was a strategy meeting, and the details managed themselves?

Well, then your business would have more power than any other business. You could do more with fewer, smarter humans, all of whom would actually be worth hanging out with after work. One change to your management logic would retrain all your hordes instantly. This sort of power is dangerous if you don't know how to use it - but soon, it'll be the difference between winning and losing the game. And the one who rules the game is the most powerful strategist, the one who defines the game itself. The one who programs the minions.

The winning strategist is the one who can hold the whole game in his mind at once, and let the details work themselves out exactly how he would want them to work out.

So what if I told you I've found a real-life game that we can really play this way? I guess then we'd just need the right programmers to control it.

21 Sep 2006 (updated 21 Sep 2006 at 01:09 UTC) »
A Quick Note on Economics

If I was centrally planning an economy, it simply wouldn't have occurred to me to figure out how to make a perfectly good microwave oven for $59.95 CAD. And that's why capitalism isn't so bad.

A Quick Note on Socialism

kjrose linked to an article about "soft paternalism", which is basically a way of subtly adjusting "default behaviours" and stating messages in just the right way so that people tend to do sensible things instead of wrong things. It's very neat, very successful, and a bit scary. And I know I've done it too. Why force people to do things when you can help them do things? Hmm.

Moral quandary: well, the cat's out of the bag. If you don't do it, someone else will.

A Quick Note on Work

Not only do we have rental plants here, but it seems we have rental paintings too. So when you get tired of them, you just send them back and someone replaces them for no extra charge! Oh, modern world, where have you been all my life?

A Quick Note on Syndication

Someone pointed out to me that there is a Livejournal syndication of my diary that people can post comments to. There are two problems with this: first, I don't see them, because I don't use Livejournal. And second, even if I saw them, I can't reply to them, because anonymous comments are disabled on that syndication and I don't have a Livejournal account. So you'll have to email me if you want me to see your thoughts. Sorry.

Welcome Back to Ontario

On the radio today: "Guys! Want to impress that special woman in your life? Make sure to practice good hygiene! We'll have some great hygiene tips after these messages." And they weren't kidding, I swear.

Crackberry Indeed

They kind of insisted on getting me one of those things for my new job. It's even more evil than I ever imagined. But it does Google Maps (and that saved me today already), so it does get high craziness points.

I guess sometimes it can be fun being a total corporate sellout. That said, this is really not your usual bank, in that everybody I've met there so far actually makes sense. And they process expense reports in 24 hours.

The End Result

Today (Sept 15) I was reminded several times that simple engineering is not exactly the meaning of life. First, an extremely opinionated salesperson at a clothing store offered me some excellent fashion advice (which I've already taken, and it helps!). Secondly, I went and saw a random concert by NEeMA and Paula Toledo, both of whom are (for now) relatively unknown Canadian artists.

There's a difference between art and business, and you can think of it like this: business has a customer. Without a customer, you can't usefully do business. But art is different. It really exists as an end in itself. Of course, it's not that great an end if nobody is there to appreciate it, but maybe the artist him/herself is the only person to appreciate it. If so, that's enough.

I'm not much of an artist, and I'm okay with that. I still think back to Deniz's comments about me in my peer evaluation. I exist primarily to step in, help other people do whatever they're doing a bit more efficiently, and step out again. A person who does that can't really exist in a vacuum; the work I do is pointless by itself.

For me, what I need to do is find someone with some good goals, and help them achieve those goals. I think many people, especially engineers, fall into that category, if you really think about it. Of course there's nothing wrong with that, but beware of trying to accomplish things without understanding why you need to. Maybe, after all, there's no point.

Kidney Vacation

Those who know me know that I don't really like relying on others for help. I don't mind people being helpful, of course, but relying on people has always been uncomfortable for me.

Still, it's really nice to know that there are so many people I can rely on when I need to. Thanks to everyone (really, everyone!) for all your support last week. I'm feeling much better now and there's no longer anything to worry about.

Just one lousy hard boiled egg??

Being away from my computer while I was in the hospital, I had a lot of time to think and no place to publish it right away. So I wrote a few entries into my paper notebook, which I'll now proceed to publish here over the next little while.

They were pumping me full of some pretty cool drugs and filtering my bloodstream and not feeding me very much, so the content is perhaps a bit different than usual. I'll try not to filter it too much, and I'll call it my "delirium series." So without further ado...

Delirium 0: The Mind in the Machine

... Ooh. Uh, that one doesn't even make sense. Maybe I'll save it for later :)

Tune in later for the next exciting episode!


With rumours that advogato.org will be read-only soon, readers of my lovely drivel may be interested to know that I'm planning to migrate my site to alumnit.ca/~apenwarr instead. Future postings will appear there (and if I can arrange it and advogato sticks around, here too, I guess).

Curses Now Sucks Less

I wrote a long time ago about the lack of liberal input handling in curses. Well, I added it. It was even easier than I expected.

There was one part that was worse than I expected: not every program uses the input-parsing library from curses, so not all of them will benefit from this change. Anything that parses its own input (which isn't really useful nowadays, as far as I can see, although admittedly non-ncurses versions of curses do suck more) will have to be patched to handle the additional keybindings. The good news is that the patches will all be about as short as this one.

Digression on Really Bad Brain Scheduling Algorithms

real	916d6h56m
user	6h21m
sys	32m
16 Aug 2006 (updated 11 Oct 2006 at 04:24 UTC) »
The Unmaker

In his Alvin Maker series, Orson Scott Card creates a character named Alvin. Alvin is a "maker": essentially, someone with a super power that lets him create and change things with his mind. As a classical protagonist, Alvin naturally only uses his powers for good, and because Card keeps his symbolism very nearly under control, it is only slightly too obvious that this is meant to symbolize the ability of everyone to create and improve things in nature. Only more so for Alvin, of course, because after all, it's a super power.

Every superhero needs a super villain, and Alvin's nemesis is the "unmaker," a mysterious force that seems intent on undoing humanity's creations and most especially, because super villains do tend to be a bit obsessive about heros, Alvin himself.

The other day, someone said something that worried me. They said, "Nothing you can do will last forever." Now, while I'm sure this is objectively true, it does seem a bit defeatist to me. Why shouldn't you try to make things last at least as long as you can?

Well, why don't they last in the first place? In the physical world, the answer is fairly clear, and we engineers call it "normal wear and tear." Dust, sunlight, rain, wind, army ants, the annoying kid next door, and other forces conspire to undo everything that you build, and all you can do is build things that last longer than others.

But the physical world is not particularly of interest to me. These are modern times, and I'm a programmer, and I deal with abstractions. And abstractions like software can't rot, can they?

Apparently they can, but for an interesting reason: abstractions can't change, but their relevance to the universe can. Mathematics is "pure" because once you've proven something is true, then it will be true forever... in the appropriate context. When you try to apply a concept, your context is the real, physical world. And the real world keeps on changing.

So to a large extent, "the unmaker" of your abstractions is just other people making things. And the way to make your abstractions last longer is to make ones that fit better, not worse, with other people's future creations. In other words, you need a really, really good design.

But I'm pretty sure that nowadays, if you have that good design, you really can make something that will last forever.

148 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!