Older blog entries for apenwarr (starting at number 90)

Yesterday one person accused me of being an idealist, and someone else overheard that and commented, "Someone says Avery's an idealist? I've got to hear this!"

I've spent a lot of time not being an idealist, because idealism, almost by definition, requires rejecting reality. I've done my best to be a good realist.

But I've really always wanted to be an idealist. It's just that if you understand realism, you can't just leave it out of your idealised world view; that wouldn't be very ideal at all. Most idealists are lucky, because they can't see the contradictions in their ideal world view. Since they can't, they can be successful idealists, although it won't do them all that much good in getting people to believe them. If you understand reality, it seems that accepting idealism is a compromise.

But what if you could come up with a system that was ideal (ie. solves its problem space without making compromises outside its problem space), but also implementable? I guess you might call that pragmatic idealism.

One good indicator that you finally understand something is that your invented name for it agrees with other people. For example, various modern philosophers discuss pragmatic idealism.

Obligatory Link to Open Source

Also, a google search for the term led me to Richard Stallman's Pragmatic Idealism essay, which I suppose takes us full circle. (rms's idealism is not really the same as mine. My idealism is based fundamentally on people being in agreement - since disagreement is a compromise - while his just politely excludes such people. But a situation that excludes such people is compatible with a system where such people don't need to exist.)

I live at the top of a hill on a busy street. In the summer, this can get loud. But in a snowstorm it can get ridiculous.

people helping people (and thus making me feel slightly guilty for just laughing at them)

people helping people until they stop at a stoplight (urban planning at its finest)

public transit triumphing until it stops at a stoplight

Who's happier here?

Very rarely, but sometimes, I'm just happy to have a digital camera.

Moral of the Story

There are several morals to this story. First, don't live on a busy street unless you own earplugs.

Second, when stuck in the snow, don't spin your tires!! As soon as your tires start spinning, you have only dynamic friction, not static friction, which means you have much less chance of succeeding. Plus, if the occasional loud crashing noise is any indication, when you do eventually find some traction, you take off rather quickly by surprise.

Third, it's December. Seriously, stop using your summer tires.

Fourth, SUVs actually do accomplish something. Minivans do not. I'm sad not to have a video of the ambulance I saw pushing a minivan up the hill just so they could get through.

Fifth, nobody except me ever learns these lessons, because everyone down there thinks they're somehow unique and will somehow get out of it just by spinning their tires. But two hours later, I can, with great confidence, say that it's a losing strategy.

Special Thanks

As always, special thanks to NITI for having their bandwidth sucked by these videos. But I may have to pull them soon in the name of, you know, not wasting money.

Tonight someone read me a story about resistance to change. Just sitting in on a story reading was kind of strange by itself, but that's not the point.

What struck me is that I totally understand the point of view of the people in the story - their fear of things changing out from underneath them. But I'm exactly the opposite. I have a fear of stability. If it's too safe and predictable, I quickly start to feel trapped. At the same time, I don't think I'd be terribly freaked out if I lost my job tomorrow and had to move to Mexico doing a job I've never done before. As long as there was something I could do about it, it would be okay.

I've heard people tend to grow out of that eventually. What a depressing thought.

I don't remember who, but a few people have told me in the past that salary is not a motivator - but at worst, it can be a demotivator. I don't know if this is true for everyone, but I think it's very true for certain kinds of people, particularly software developers. If they're getting paid enough that they're satisfied with it, paying them more won't make them work any harder.

I think there are basically three states of motivation:

- demotivated (negative)

- motivated (neutral)

- hypermotivated (positive)

I like to believe that, by default, most people are in the "motivated" state. That is, by default, most people want to do useful stuff rather than sit and not do useful stuff. Maybe I'm wrong, but in any case, I like to at least think that we don't hire people who are unmotivated by default, so it doesn't really matter if literally everybody is the way I think they are.

From this neutral state, you can go in two directions. The things that take you in one direction versus the other are actually unrelated to each other. More of thing A might demotivate you; less of thing A might demotivate you less. But no amount of it, or a lack of it, will ever hypermotivate you.

For example, you will never achieve hypermotivation because we pay you more, or give you clear (instead of unclear) specifications, or don't chain you to your desk and whip you, or don't make you play kindergarten-style HR role playing games.

It's unlikely that you'll become actively demotivated because we don't have a foosball table, or because we don't have an inspirational leader, or because you don't believe the work you're doing can really change the world.

The fact is that merely motivated people can do really good work. Some companies do just fine running entirely on merely motivated people. It's just that hypermotivated people can do even more... lots more.

I've written a lot lately about the two sides of various issues, and the fact that not compromising, but instead maximizing both constraints, is the only way to really succeed. For example, you research, write, and comply with excellent functional specifications in order to make people not hate your product; you do excellent design in order to make them love your product.

Companies and motivation work the same way. Carefully following some boring, well known HR guidelines to avoid the common demotivators can make people not hate working at your company. (Avoid the stupid parts of the well known HR guidelines, though, because some of them are demotivators themselves.) But hypermotivation isn't just about removing demotivators; you have to also have the right hypermotivators in the right places. For every person, those hypermotivators are different.

Not many people are hypermotivated; of the people that are, I'm not sure they could honestly all tell you exactly why that is. I think I'm in that category. But not knowing those things gets in the way of sharing those things with others.

I have nothing to really add on the topic, but I think mathrick's brief post on user interface design is worth a link.

Motivation, Part 1

It's so easy to drain the life out of something, but so hard to put it back in. Doesn't hurt to try though. Mmm, squishy.

In the last week, I spent a total of about 7 hours in trains travelling up the coast from Portland, through Seattle, to Vancouver. Unlike VIA, Amtrak doesn't have (albeit barely functional) onboard Internet, but they do have power plugs, and I was a bit bored, so I left my laptop on and scanning for access points the whole way. As it turns out, there are a lot of different access points along that route, and only about half of them are encrypted.

Sadly, trains move so fast that you can only get about 30 seconds of connectivity - at best - before you move on. Some clever scripting got me an IP address and a PPTP link before time ran out, but sadly I was stumped due to my lack of a good mobile IP setup and good offline mail reading support. There are also a few annoyingly long periods with no connectivity at all. Still, chances are good that, as Linksys-style wireless routers get even cheaper, someone will eventually find a trick to this sort of thing.

When I got back, I disabled WEP on my own router at home, because... why not? There's nothing more annoying than having your train stop for 10 minutes on a side track when the only access points in range are annoyingly encrypted.

Pantsless in Waterloo

My parents used to remind me of the important difference between "good attention" and "bad attention." Others say there's no such thing as bad publicity. Now, imagine that half the article about Waterloo being "best overall" university in Canada is about the pantsless movement, and that movement is being spread mainly by students who did work terms at NITI. But NITI was not mentioned in the article, only Waterloo. So is that good attention, bad attention, publicity, or nothing important? You decide.

Disclaimer: With the possible exception of a casual and brief association with the work of certain members, I am not now, and never have been, a member of the pantsless party, er, movement.

The other day, talking to some people from IBM in Raleigh, I learned a bit about businessspeak. You know, the strange language involving "paradigm shifts" and "issues" and "core competencies" instead of the normal things that normal people talk about. I finally figured out what businessspeak is good for. Yes, I realize that I could be kicked off Advogato for saying that, but I'm going to tell you anyway.

As you might imagine, people from IBM know an awful lot about businessspeak. IBM is also massively rich, big, and powerful, so they're doing something that works, whatever it is. Here are some specific businessspeak lessons they taught me:

1. (You've probably heard of this one.) There is no such thing as a "problem." Problems are bad. Some people would say there are "issues," but that's still a bit negative sounding. The proper term is "challenges." Everyone has challenges. And challenges are, of course, good. What would your job be like without challenges?

2. (An impressive new discovery.) No product can be described as a "something something killer." This is only true after the something something has already been killed, at which time there's no real need to describe it that way. Instead, it can be a "something something fighter."

Why are these two examples interesting? Because I finally found the common theme: "non-arguability." I just made up that word, which in retrospect has about the same meaning as "non-controversial," but I didn't really understand what non-controversial really was until I thought of it this way. It's not that you don't argue with it because everyone agrees; it's that you can't argue with it because the person making the statement wins by default.

Try these:

1. "You know, marketing this software to schoolchildren in Africa, who have neither computers nor electricity, is going to be a real challenge." Even if you think it's possible through some incredible new space-age technique, nobody's ever going to deny that it's a challenge. Conversely, "You're going to have problems selling this software to schoolchildren in Africa" is inviting someone to explain how no, really, it's possible, and it might be hard but it won't be a problem.

2. "This thingy is going to be a real something something fighter!" The person saying that is only saying that your thingy is going to go up against the something something, which is always true in some respect; you're both in the same marketplace, and someone might buy one, the other, both, or neither. If they buy just one or the other, then I suppose one or the other won. Meanwhile, if you say, "This thingy is going to be a real something something killer!" you invite argument. Perhaps the something something will kill the thingy. Wouldn't you look silly then!

Okay, so why would you want to be non-controversial? Well, as an engineer, you wouldn't. That's why engineers who insist on calling their bugs "issues" drive me crazy. I'm sorry, but engineers don't gloss over problems: they state them as clearly as possible, and then they either solve the problem or agree that it's not worth solving.

But sales - and by extension, people hacking - is different. In that case, you're messing with someone's emotions with the goal of getting them to agree with you and eventually do something for you (eg. buy your stuff). And the biggest barrier to sales is (ironically?) defensiveness: the feeling that someone is trying to sell you something. Being non-controversial helps avoid making people defensive.

Here's where I'll add my own bit of spin. The most basic form of non-controversiality is the above: making all your statements more bland in order to keep the recipient from becoming defensive. The good news for salespeople is you actually can do this by memorizing a few simple words and techniques. Unfortunately, blandness also prevents your customers from becoming interested. In fact, controversy results in a lot of emotion, and emotion keeps people interested. Boring, neutral TV news shows don't get any viewers; controversial news shows on either end of the spectrum get lots of viewers.

So here's what you do. If you're really good at this, you can say only controversial things that the person listening will agree with instantly. You have to be really good to pull it off, because you need to really understand your listener. If you misread them and say the wrong thing, they get all defensive, and you're worse off than if they were only bored.

And then, if you're really really sneaky (or perhaps just dumb), you can do what I realized I've done for a long time: you can say mostly non-controversial things, then every now and then throw in a fake controversy. This is either something you can teach them about and they'll like in the end, or something that you can just turn out to be joking about. This is the "just seeing if you were listening!" technique, and used correctly, it can really make a difference in the interest level of your presentation, without causing defensiveness. NITIites who have seen my presentations will probably be able to remember some cases of this from me if they think back.

Now, if you agree with the things I've just said, see if you can find any instances of those techniques in the article I just wrote.

Travel Notes

After visiting Raleigh, NC, which smells nice, I now find myself in Portland, Oregon, home of the Transformers. Take that, mich!

I have been pondering the Great Dichotomy between the two offices of NITI a lot lately. We have one office in Toronto (sales, marketing, support, manufacturing, etc), and another in Montreal (R&D), and their cultures are very different. Neither is exactly wrong, but people from one culture have a lot of trouble understanding the other.

I think I now at least partly understand the difference. And, surely to the joy of the more technical readers here, I can explain it in terms of task scheduling algorithms.

In Montreal we use a system called Schedulator that I originally "designed" (it has since been rewritten). It basically takes Joel Spolsky's Painless Software Schedules essay and automates it.

Schedulator, and Joel's original essay, is about schedule prediction. It helps you predict when your project is going to reach various milestones. Used under the right conditions, it can work very well.

Now what happens if you get some surprise bugs in an earlier release? Joel didn't say anything about that. Well, what happens is your schedule will slip, because high-priority tasks insert themselves in front of everything else. Schedulator manages this automatically, and each day you can look at a pretty graph of your project schedule and watch the end date slip further into the future.

Inside R&D, there are other mini-deadlines as well. We have this concept of a Zero Bug Bounce (yes, stolen directly from Microsoft), where the idea is that all tasks are either complete, found too recently to be reasonably fixed, or shovable into a future release. And it's someone's job to predict when the next bounce will be, then make sure you get done on time.

Hey, what's this "make sure" business? Schedulator updates its predictions in real time, right? So the bounce date might slip, but only for a good reason, right? So there's nothing we can do, right?

Sort of. The problem is, if the last release-critical bug just doesn't seem very important, something more important will always come up, the new thing will jump to the front of the schedule, and the bounce will never get done. Thus, we have to introduce a form of deadline-based prioritization as we get very close to a bounce date. In the deadline-based system, you convert your prediction into a promise: that is, Schedulator tells you, as of right now, when you think you can get it done. You add a bit of time, just in case. And then you promise that you will absolutely have it done by that time, no matter what.

The same overall method applies for software final release dates (sooner or later, you just stop fixing bugs - even important-seeming ones - in the old version so you can finally just finish the new one). This method feels wrong; we're raising the priority of obviously lower-priority tasks above obviously important ones. Developers want things to be simple; they like prediction, but they don't like keeping promises that force them to break their priority scheme.

Here's my big insight. Marketing people are exactly the opposite. They don't care about predictions at all. They only care about promises. You can predict bug fix times, bounce dates, or anything else, but in the end, they want you to commit to a release date, and they want to do their work, confident that you'll be done by the date you promised. Exactly what that date is isn't very important; that you guarantee it is what's critical, because otherwise they can't properly do their thing.

And Marketing-type people carry their concern for promises above prediction way too far. Joel's article, above, talks about how Microsoft Project is useful, but not for writing software. I now understand why: it's because the type of work is very different. Some tasks a salesperson might have to do - like arranging a meeting - can take two weeks, but only, say, 5% of their effort during that time. Software developers work mostly linearly (one 100% task at a time), while salespeople do a lot of tasks in parallel. So it's easy for a salesperson to promise, "I'll have this meeting set up within about two weeks, no problem." Even if something new and "more important" comes up, they don't delay the previous task; they just do one more thing in the same amount of time. But after a person gets sufficiently heavily loaded with 5% tasks, each new task does delay the work, the same way as it would for a developer. That's the exception, however, not the rule, because simply not overloading your salespeople can dodge the problem.

A couple of weeks ago, the CEO told me, "I think we need a Schedulator for the rest of the company." This was true, in a way: we have people in sales and marketing who are having trouble making and keeping their due date promises. Schedulator, in contrast, was created to help developers, who were having trouble making predictions. It doesn't help them keep promises (although better predictions are one necessary part of saner promises); in fact, unless you're careful, it gives them an excuse for breaking promises when the predictions change. The design we came up with for a "marketing schedulator" is actually totally different from the original Schedulator. It involves a central, top-down plan from Microsoft Project, and only three bits of information fed back from individuals: notes about each task, % complete, and a checkbox: "in danger of missing the due date." Despite your best intentions, you might still have to sometimes break promises, but when this is (albeit rarely) the case, now you have to check a checkbox and have the CEO get angry at you.

I suppose the ultimate Schedulator of the future could combine the two concepts. It would predict dates, then help you promise them by auto-increasing the priority and moving tasks around so that future predictions always leave your existing promises intact. Then everyone on both sides of the fence could use it. Meanwhile, I think we need two systems, which will be strangely perpendicular to each other.

Our company has been working lately at partnering with several different other "ISV" companies who want to use our product as a platform for their product. After being basically screwed around by one of the larger of those ISVs in the last few days (nothing really serious; just super annoying), I came to this conclusion:

Trust people in your company, but don't expect to trust people in other companies.

And therein lies a very interesting lesson, which I will attempt to relate back to software via a mild digression in the opposite direction.

One of my pet theories of capitalism (or at least, the non-insane variant practiced in Canada) is that, unlike idealistic theories like communism or libertarianism, capitalism tries to combine the best of both worlds:

- It is impossible to centrally control an immensely complex system, like an entire country's economy, so we don't try. We implement a complex, mostly self-organizing system instead, by carefully controlling the rules of the game.

- Simpler systems, like small groups of people, are much more efficient when organized as cooperative, not competitive, groups with a centrally organized set of goals. That's why employees of any particular successful company aren't generally set in cutthroat competition with one another.

Why does it work? Because up to a certain size, a single mind can hold and optimize the entire structure. I don't mean just the person at the top of the pyramid; I mean that, in a proper organization, anyone can see and understand the structure they're working in. That means they can understand other people's goals and how those goals fit in with the big picture. When you can do that, you can resolve your differences of opinion by finding the one "right" non-compromise answer. In other words, you can work efficiently.

In groups that are too large, this is impossible. You really can't understand why the person you disagree with is doing what he's doing; or worse, perhaps he's doing that because his goals are actually different from yours and the system is too complicated to find a non-compromise solution. That's why we have multiple companies competing with each other, and it works better than if they just tried to all get along.

But what exactly is too big?

This is the fun part. The last few decades have seen a huge increase in the number of smaller companies, while simultaneously bigger companies have merged and gotten even bigger (and fewer). This is because of two completely separate effects.

First, big companies get bigger because of technology: with better communication and management technologies, it's possible to centrally organize larger and larger groups. What used to be impossible with pencil-and-abacus accounting systems is now possible with computers. This trend should continue, and big companies will be able to get bigger.

The second effect is weirder. It's also because of technology, but causes exactly the opposite effect. With technology, smaller groups can do more and more complex things. Complex things are difficult to manage centrally.

So which companies get bigger? The ones doing simple, parallelizable things. Kraft mass produces food products in giant vats. Various companies massively drill for oil. GM mass-produces cars. Sony mass-produces electronics.

And IBM and Sun mass-produce and recombine Java objects that go on to mass-produce billable consulting hours. Meanwhile tiny little companies start up, produce and recombine open source tools, and (sometimes profitably) produce complex, not always open source, products in small quantities, and almost never in Java. Aha. And here we are back in the world of software.

Linus's rule for functions: The maximum length of a function is inversely proportional to the complexity and indentation level of that function. Take that to the world of OO, and the maximum size of a class is inversely proportional to its complexity. My claim is that it's also proportional to Bigness of Thought (BoT), that is, the amount of complexity of this type that can be held in a human brain - the particular brains working on your project - at once.

Java encourages teeny tiny objects that do only one tiny thing, hopefully well. That's because many Java programmers are people like financial analysts, who never really wanted to be programmers, and so their BoT for programming concepts is teeny tiny. These people can program in Java. You might need a horde of them to get anything done, but they can get it done. Pretty neat, really.

Some people have a very high programming BoT. I think I fall into that category. The danger for those people is that they can write amazing programs nobody else can understand, which in the end, doesn't do anyone any good, because it's impossible for someone to maintain it after they move on. Worse, even people with very high programming BoT can't understand the programs, because everyone with high BoT has a different mental structure. I'm great at remembering concepts, but can't remember my postal code; other people can remember dozens of interest rates and formulas with no problem, but they need to draw a picture to see any difficult concept. Both groups have a high BoT, but for totally different things. Both groups can even make very good programmers, but for totally different programs.

What's the point of all this? Well, I could write for hours about the results, but my main point is: the BoT of your group defines the maximum implementation complexity of your objects. When the overall project exceeds your group's BoT (which is virtually always), you need to subdivide your objects into sub-objects with understandable interfaces that reduce the BoT needed to build the object that combines them. And, as technology (eg. programming languages and libraries) improves, the amount of organized complexity you can squeeze into the same BoT increases.

It's just like capitalism. If you can't centrally manage it all, you split it into two companies to make it achievable. If you have two companies but you could centrally manage it, you merge the two companies to make it more efficient.

And if your partner company ("library programmer") has some idiots ("bugs") you're going to have to ask them nicely to get your problems solved, because your poor brain doesn't have the capacity to hold all the details of the whole picture all at once.

Interesting Side Notes

Compromise (solving each person's problems poorly, instead of solving them all perfectly at once) should be necessary only when the problem exceeds the BoT of all affected parties.

Concepts near the limit of your BoT are very difficult for you to explain to anyone else unless they have an exceedingly high BoT. People with a low BoT, but who manage to understand a particular concept anyway, will be good at explaining it.

It is possible to reexplain difficult solutions from one BoT domain such that they make sense to people in another BoT domain. For example, some concepts can be explained using diagrams, even if the person who solved the problem didn't need a diagram to do so. This is sort of like translating from French to English; something is bound to get lost, but it's better than nothing.

By extension, UML is about as useful as, say, Babelfish.