Older blog entries for apenwarr (starting at number 156)

Delirium 1: The Genius of Obsession

My friends dmg and apm recently lent me a book called The Soul of a New Machine, by Tracy Kidder. It's a work of long form, non-fictional journalism that describes the process of designing a new minicomputer at Data General around 1980. It's much more about the people and their social issues than about the computer itself.

What an interesting book to recommend to me right now. I certainly recognize the story: team of 30 or so crazy people in larger company isolate themselves, gel as a team, build impossible product on crazy timeline, live through pressure, team reorgs, smart and stupid managers, and have way too little real life experience. Product gets completed (a bit late, of course), team leader moves on, and team essentially disintegrates and carries its experience elsewhere, person by person, while parent company lives on, continuing to sell the resulting product and ones based on it. I guess we can be proud - we held together the culture longer than they did.

But the things that really struck me about the story weren't so much the similarities as the differences. They did a few things "right" in that story - for some definition of "right" - that I never quite managed. Their team lead, Tom West, managed to bring out of everyone on his team something that I've only ever managed to bring out in myself personally - total obsession with the end goal.

And obsession, as I learned for sure in the last few months of not being obsessed, is where it's at. Obsession is the only way to really get into The Zone. It gives you energy you didn't know you had. It turns you from a normal mortal into something more. I've done it for myself a few times. I used to wonder if you could even do it for other people, because I never had much luck. But after reading this book, I have a few clues. Not surprisingly, they echo the environment I create naturally for myself - but I have to change my style if I'm going to create the same conditions for others.

Mutual Motivation. You can't obsess alone for long. The key thing isn't so much having help as being needed. You have to not let people down. That fear is what makes stopping harder than going on.

"Signing up." It has to be voluntary. You have to know what you're getting into on day 1, and decide it's worth it. You have to decide it out loud, and promise it to others. You have to be visibly, publicly in the game, so any failure will be an obvious one.

No way out. There can be no escape clause. If you fail, we're all dead. Nobody will save you. If you're late, we're all late with you. There's no superhero who will do it better and bail you out. You're going to have to grow up and be the superhero. If you really have to do it, maybe then you really can do it.

Too stupid to know it's impossible. It has to be harder than you thought it would be, not easier, so the pressure builds instead of dissipates near the end, when it's easiest to lose interest. And so experience kills - you'll never be able to obsess the same way twice. That's why younger people, especially students, are better at it than oldsters. (As someone on the verge of being an oldster myself - in the book, 30 years is the magic age - I already have to find crazier and crazier projects each time, always outside my existing experience, or I simply can't obsess enough.)

Too blind to see it doesn't really matter. In real life, almost nothing you do makes a serious, long-term difference. Realizing this will kill your fun. You need blinders - and if you're lucky, a really good reality distortion field. Again, older people have trouble here, because once you've been disillusioned by dedicating a year to an ultimately pointless project, it's hard to fall for that again. Except that falling for it is fun.

Too restricted to take the easy way out. Smart, sensible ways of solving problems will knock you out of The Zone. Sensible schedules, flexible and forgiving bosses, 8-hour work days, good nutrition, the newest equipment, a social life (outside work), even using low-cost direct mail services instead of composing your own poetry and stuffing your own envelopes - all these things make it too easy. That means less pressure. Less pressure means less obsession. There's a good reason why entrepreneurial companies are responsible for most of the wildest innovation in the world: because they're so helpless, disorganized, and resource-constrained that nothing you do there isn't a challenge. And that makes your brain work harder, nonstop, without any chance to cool down and drop back from hyperspace into normal space.

When I've done the best work in my life, those are the rules I followed. I turned my load factor down to levels I knew perfectly well were too low to be safe. I set things up so I had no escape and nobody could possibly bail me out. I bet my reputation that I would be done on time. And because it's my special skill, I always, always, failed to realize how impossible and unimportant the job was - at least until afterwards.

And that, let me tell you right now, is the magic combination that makes life worth living. From now on, I'm going to be a very different kind of manager. You should be glad you don't work for me anymore!

Delirium 0 (second try): Nerve Center

When you play a computer strategy simulation, there might be hundreds of agents doing your bidding - workers, warriors, transports, cities, convoys, and maybe even angels and demons. They march to your orders, they do more work than you could ever do for yourself, faster than you can see, because the computer handles the details - you just handle the strategy.

But the real world isn't quite like that. In the real world, real minions cost real money, and have real feelings, and want real self-actualization and self-expression. Minions are real people, just like strategists, and minions want to grow up to become strategists like you. But managing the feelings of your minions takes work - it takes away from your game - it weakens you compared to the other players who "feel" their minions better than you do.

What if your real-life minions weren't people, but computers? What if whole aspects of your business didn't need human minions anymore - just automatons with some strategic direction? What if every human was a strategist, and every meeting was a strategy meeting, and the details managed themselves?

Well, then your business would have more power than any other business. You could do more with fewer, smarter humans, all of whom would actually be worth hanging out with after work. One change to your management logic would retrain all your hordes instantly. This sort of power is dangerous if you don't know how to use it - but soon, it'll be the difference between winning and losing the game. And the one who rules the game is the most powerful strategist, the one who defines the game itself. The one who programs the minions.

The winning strategist is the one who can hold the whole game in his mind at once, and let the details work themselves out exactly how he would want them to work out.

So what if I told you I've found a real-life game that we can really play this way? I guess then we'd just need the right programmers to control it.

21 Sep 2006 (updated 21 Sep 2006 at 01:09 UTC) »
A Quick Note on Economics

If I was centrally planning an economy, it simply wouldn't have occurred to me to figure out how to make a perfectly good microwave oven for $59.95 CAD. And that's why capitalism isn't so bad.

A Quick Note on Socialism

kjrose linked to an article about "soft paternalism", which is basically a way of subtly adjusting "default behaviours" and stating messages in just the right way so that people tend to do sensible things instead of wrong things. It's very neat, very successful, and a bit scary. And I know I've done it too. Why force people to do things when you can help them do things? Hmm.

Moral quandary: well, the cat's out of the bag. If you don't do it, someone else will.

A Quick Note on Work

Not only do we have rental plants here, but it seems we have rental paintings too. So when you get tired of them, you just send them back and someone replaces them for no extra charge! Oh, modern world, where have you been all my life?

A Quick Note on Syndication

Someone pointed out to me that there is a Livejournal syndication of my diary that people can post comments to. There are two problems with this: first, I don't see them, because I don't use Livejournal. And second, even if I saw them, I can't reply to them, because anonymous comments are disabled on that syndication and I don't have a Livejournal account. So you'll have to email me if you want me to see your thoughts. Sorry.

Welcome Back to Ontario

On the radio today: "Guys! Want to impress that special woman in your life? Make sure to practice good hygiene! We'll have some great hygiene tips after these messages." And they weren't kidding, I swear.

Crackberry Indeed

They kind of insisted on getting me one of those things for my new job. It's even more evil than I ever imagined. But it does Google Maps (and that saved me today already), so it does get high craziness points.

I guess sometimes it can be fun being a total corporate sellout. That said, this is really not your usual bank, in that everybody I've met there so far actually makes sense. And they process expense reports in 24 hours.

The End Result

Today (Sept 15) I was reminded several times that simple engineering is not exactly the meaning of life. First, an extremely opinionated salesperson at a clothing store offered me some excellent fashion advice (which I've already taken, and it helps!). Secondly, I went and saw a random concert by NEeMA and Paula Toledo, both of whom are (for now) relatively unknown Canadian artists.

There's a difference between art and business, and you can think of it like this: business has a customer. Without a customer, you can't usefully do business. But art is different. It really exists as an end in itself. Of course, it's not that great an end if nobody is there to appreciate it, but maybe the artist him/herself is the only person to appreciate it. If so, that's enough.

I'm not much of an artist, and I'm okay with that. I still think back to Deniz's comments about me in my peer evaluation. I exist primarily to step in, help other people do whatever they're doing a bit more efficiently, and step out again. A person who does that can't really exist in a vacuum; the work I do is pointless by itself.

For me, what I need to do is find someone with some good goals, and help them achieve those goals. I think many people, especially engineers, fall into that category, if you really think about it. Of course there's nothing wrong with that, but beware of trying to accomplish things without understanding why you need to. Maybe, after all, there's no point.

Kidney Vacation

Those who know me know that I don't really like relying on others for help. I don't mind people being helpful, of course, but relying on people has always been uncomfortable for me.

Still, it's really nice to know that there are so many people I can rely on when I need to. Thanks to everyone (really, everyone!) for all your support last week. I'm feeling much better now and there's no longer anything to worry about.

Just one lousy hard boiled egg??

Being away from my computer while I was in the hospital, I had a lot of time to think and no place to publish it right away. So I wrote a few entries into my paper notebook, which I'll now proceed to publish here over the next little while.

They were pumping me full of some pretty cool drugs and filtering my bloodstream and not feeding me very much, so the content is perhaps a bit different than usual. I'll try not to filter it too much, and I'll call it my "delirium series." So without further ado...

Delirium 0: The Mind in the Machine

... Ooh. Uh, that one doesn't even make sense. Maybe I'll save it for later :)

Tune in later for the next exciting episode!


With rumours that advogato.org will be read-only soon, readers of my lovely drivel may be interested to know that I'm planning to migrate my site to alumnit.ca/~apenwarr instead. Future postings will appear there (and if I can arrange it and advogato sticks around, here too, I guess).

Curses Now Sucks Less

I wrote a long time ago about the lack of liberal input handling in curses. Well, I added it. It was even easier than I expected.

There was one part that was worse than I expected: not every program uses the input-parsing library from curses, so not all of them will benefit from this change. Anything that parses its own input (which isn't really useful nowadays, as far as I can see, although admittedly non-ncurses versions of curses do suck more) will have to be patched to handle the additional keybindings. The good news is that the patches will all be about as short as this one.

Digression on Really Bad Brain Scheduling Algorithms

real	916d6h56m
user	6h21m
sys	32m
16 Aug 2006 (updated 11 Oct 2006 at 04:24 UTC) »
The Unmaker

In his Alvin Maker series, Orson Scott Card creates a character named Alvin. Alvin is a "maker": essentially, someone with a super power that lets him create and change things with his mind. As a classical protagonist, Alvin naturally only uses his powers for good, and because Card keeps his symbolism very nearly under control, it is only slightly too obvious that this is meant to symbolize the ability of everyone to create and improve things in nature. Only more so for Alvin, of course, because after all, it's a super power.

Every superhero needs a super villain, and Alvin's nemesis is the "unmaker," a mysterious force that seems intent on undoing humanity's creations and most especially, because super villains do tend to be a bit obsessive about heros, Alvin himself.

The other day, someone said something that worried me. They said, "Nothing you can do will last forever." Now, while I'm sure this is objectively true, it does seem a bit defeatist to me. Why shouldn't you try to make things last at least as long as you can?

Well, why don't they last in the first place? In the physical world, the answer is fairly clear, and we engineers call it "normal wear and tear." Dust, sunlight, rain, wind, army ants, the annoying kid next door, and other forces conspire to undo everything that you build, and all you can do is build things that last longer than others.

But the physical world is not particularly of interest to me. These are modern times, and I'm a programmer, and I deal with abstractions. And abstractions like software can't rot, can they?

Apparently they can, but for an interesting reason: abstractions can't change, but their relevance to the universe can. Mathematics is "pure" because once you've proven something is true, then it will be true forever... in the appropriate context. When you try to apply a concept, your context is the real, physical world. And the real world keeps on changing.

So to a large extent, "the unmaker" of your abstractions is just other people making things. And the way to make your abstractions last longer is to make ones that fit better, not worse, with other people's future creations. In other words, you need a really, really good design.

But I'm pretty sure that nowadays, if you have that good design, you really can make something that will last forever.

Taking The Hard Way Out

One thing that has bothered me for a long time was a comment in a lecture by famed Waterloo economics professor Larry Smith. To paraphrase, he said there are a lot of big problem areas in business that need solving that are just ignored. Meanwhile, everyone in the computer industry seems to have a fatal attraction a particular other set of problems, which they solve over and over again.

The example he gave at the time was Windows vs. Linux: who would want to start a company to compete with Microsoft in this day and age? Even if you did manage to win (or even to not die), it would just be so much work! And no, the irony was not lost on me at that point in my entrepreneurial career.

My answer to his question at the time? Well, some projects are just more interesting than others, and I want my career to be interesting. It was abundantly clear to me even then that going into, say, banking software was the way to make loads of money, because the customers are rich and (it was a y2k madness at the time) boy, do they ever have lots of easy problems to solve. But banking software was by no means interesting. So I elected to take the path more travelled, as it were, and go for technical challenge instead of easy megaprofits. In other words, a compromise. I even formed a little continuum in my mind of different kinds of programming projects, in which the most technically challenging ones (hardest I could think of: video games) are the least profitable for the developer. And vice versa: banking software has got to be some of the least technically difficult software in the universe, and attracts some of the very worst programmers, who then get paid a lot to work reasonable hours with very low stress.

But the fact is that I'd be more than pleased to increase my stress levels a bit in exchange for an interesting project. That much isn't a compromise; in fact, too little stress is downright boring. That's why the best developers gravitate toward harder projects instead of easier ones. Video game developers know perfectly well what they're getting into before they start. They're buying into a whole lifestyle.

Well, good for them. I wasn't quite willing to go as far as the insane working conditions in the gaming industry, but I still wanted challenge, and operating systems seemed fun. There was stress, but that's not a compromise, because I wanted stress.

The compromise was the profits. Say what you like, and I do dearly love the free software community, but making massive quantities of money is a great way to make an already-fun company even more fun.

So what's the non-compromise solution in this case? Well, the form of the solution is obvious enough: it must be highly profitable but also challenging and difficult so that it can be fun.

It's pretty hard to take something that's challenging and difficult and not very profitable, and then make it highly profitable. Profits don't just magically appear just because you did something that was difficult. If it was the wrong difficult project, you're still out of luck.

But what about taking something profitable and making it difficult? Why, you can take just about anything and make it difficult if you try hard enough!

Of course, it would be best if the difficulty were actually warranted in some way, for example, to make the product even more profitable, or to also make the world a better place, or whatever. But that's actually only a secondary requirement. Look how much fun we had revamping our knowledgebase, for example.

So what if I could find a useful way to make, say, banking software challenging and fun?

147 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!