Older blog entries for apenwarr (starting at number 76)

4 Nov 2005 (updated 4 Nov 2005 at 04:32 UTC) »

It often seems like a good idea to throw out all your code and start again; especially when you just took over someone else's code and it's now your job to maintain it. The reason it seems like such a good idea is that the problem space sounds so simple... but the code looks so complicated. So you throw it away, and write it from scratch. That's when you realize why the old code was so complicated: there were lots of special cases for all the weird bits that you didn't realize were part of the problem space in the first place. So your rewrite gets big and complex too. If you're very smart, at least it eventually gets to be better than the original; but even then, it takes a long time to do. And if you're unlucky, your rewrite is merely bigger and slower, not better.

Companies are the same. Company processes are designed and encoded (eg. through forms or software systems) over a long period of time, and they cover lots of weird special cases. When you look at the problem space, it sounds so simple, and you wonder why the solution has to be so complicated. So you throw it away and start from scratch, redesigning a bunch of things to better align with your way of seeing the world. But it means you've lost the benefit of all the tweaking you've been through over the last few years; perhaps your overall model is better, but the surface details are lumpy, inefficient, and downright wrong at first. If you're very smart, hopefully it eventually gets to be better than the original; but even then, it takes a long time. And if you're unlucky, your redesigned company is merely bigger and slower, not better.

With any great redesign, you have to be constantly on the lookout for mistakes. No new design or implementation is perfect right from the start. The biggest danger is to assume that it is, and find out only very late that you're wrong. (That way is called the "waterfall method," and it's very inefficient.)

Looking for a job in Toronto?

Speaking of restructuring...

NITI has a job posting for a QA Manager in Markham, Ontario. If the corporate version of the job description looks uninspiring, see instead my version: Evil Death Ray. Actually, Evil Death Ray is a "QA Person", while the new job is "QA Manager." But the idea is the same, only more so.

You'd be working on testing our award-winning Nitix distribution of Linux, the only one that has ever been successful in servers for small business. Yes, we are open source friendly. And you'd get to play with our super fun, room-filling Project Death Ray test cluster. As well as annoying me, one of the shoddy developers whose code you'd be breaking.

Roadblock Analysis and the 80/20 Rule

I've written down this theory a few times in a few different places, but I still don't think I've explained it clearly. Here's another try.

For years people have been talking about the magical 80/20 rule of business: that 80% of your revenue comes from 20% of your customers, so you should find out who that 20% is and focus your attention on them. When you do, you magically make more money.

Nobody, of course, has ever offered me any evidence of this; only, "Wherever you look, it always turns out to be true. It did for us." This is suspicious, because it implies a selection bias, in which people have a natural tendency to only look for evidence that supports the specific thing they're trying to prove. For example, in the 80/20 rule, the 80 and the 20 measure different things; they don't have to add up to 100%. The 80/30 rule or the 90/40 rule are just as plausible as the 80/20 rule, lacking any additional evidence.

But I'm willing to accept a weaker formulation: the majority/minority rule. The majority of your revenue comes from a minority of customers. There are all sorts of reasons this might be true, most obviously the fact that most customers are small and therefore a few larger customers add up to more money than a few smaller customers.

Roadblock Analysis

I recently learned a rule, obvious in retrospect, that is critical to understanding business. Let's define a "roadblock" as a convincing reason not to buy. In that case the following must be true: no customer will buy your product until you eliminate all of his roadblocks. How do I know? If you invert the statement, it's obvious: if there remains a convincing reason not to buy, the customer will be convinced not to buy, by definition, and so will not buy your product.

This is important: if you solve 90% of the roadblocks for 100% of people, you don't have 90% of people buying your product; you have 0% buying your product. "Roadblock analysis" is my name for a process that I certainly didn't invent: the process of identifying a group of people (a "market segment") and the complete list of their roadblocks.

Now, in reality, people's needs are distributed randomly, and your features are distributed randomly, so some people will have their roadblocks solved just by random luck. But not most of them.

Note that roadblock analysis, unlike the 80/20 rule, is not magic: it's just a simple, logical statement. If you do eliminate all the reasons not to buy your product (remember, many of these reasons are non-technical, such as "I've never heard of your product"), then they will buy your product.

Why 80/20 Works

Once you understand roadblock analysis, you can understand why following the 80/20 rule (whether it's precisely 80% and 20% or not) actually helps. It's like this: the current customers who actually make you the most money are the ones who currently have zero roadblocks for many of their situations. The others are the ones who currently have more than zero roadblocks, at least for most of what they do.

People known to have more than zero roadblocks might in fact have lots of roadblocks; maybe hundreds of them. Who knows? But people who already have zero roadblocks in many cases probably have near-zero roadblocks for a bunch of other related things. It just makes sense; they probably do a lot of similar things, so if there are many situations where your product fits, and some situations where it doesn't, you can probably improve just a few things and solve those problems too. Not so with the other 80% of customers; for those, by default, you should assume you're nowhere close.

80/20 is a Random Process with Convergence

Repeatedly following the 80/20 rule causes you to converge on the closest market segment to the randomly-selected customers you originally chose.

That is, you start off by spamming the market with a technology-driven product that does something cool; you find out who buys it; you optimize it for those people; you find out which of those people buy it; you optimize it for those people; and so on. This is a feedback control system which will eventually converge on the local maximum market segment. Notice how the 80/20 rule, by focussing on a smaller and smaller subset of customers each time through the loop, decreases the "hop size" each time. This is a well-known technique for guaranteeing convergence. As we know from calculus, this kind of method works pretty well: but the local maximum is often very different from the absolute maximum.

Characteristics of 80/20 Solutions

The 80/20 rule is a major management fad at the moment, presumably because it works much better than completely random guessing about which customers are important, which is what most companies would resort to otherwise. After all, a simple mathematical method that gives a very high probability of making an existing product even more profitable is nothing to sneeze at.

But 80/20 solutions will show some very specific tendencies, which you can see all around you by looking at your favourite companies.

- The tendency to annoy about 80% of customers by ignoring or mistreating them. (In this case, the 80% is real, because companies define their strategy by literally choosing the 20% of customers they will care about.)

- The lack of new customers. By focussing on very specific existing customers and never implementing features someone else might want, you limit your ability to attract new ones and slowly get further and further away from other market segments.

- The irresistable tendency to move "upmarket." The one thing this algorithm guarantees is that when you have one big company and one small company as a customer, the big company will always win. There's no way any one small customer can land in the top 20% of your revenues. So you get more successful only as you serve fewer and fewer bigger and bigger customers.

This leaves a badly underserviced 80% vacuum at the low end, which in the software industry is basically the "small business" market.

The Missing Markets

Roadblock analysis is a more general method than 80/20 for finding and servicing a market, based on one important insight: there might be a huge market that you completely cannot serve right now because you left all of the customers in that market with a few, actually rather simple, roadblocks. These customers aren't in your top 20%, because all of them have problems.

The roadblock analysis method is much more risky than 80/20, in fact, because it's hard to know the list of all roadblocks you have to solve. It might look like there are only a couple of them, but after solving those, you might discover a dozen more. 80/20 gives you a virtually guaranteed path to expansion, albeit at an unknown pace; roadblock analysis guarantees nothing, but offers higher potential gain.

The Best of Both Worlds

Finally, the good news: if you explicitly choose a market segment using roadblock analysis, you can then use a variant of 80/20 to improve your performance inside that market segment.

In math, this is like choosing a better initial value for your convergence algorithm; if you give your algorithm a clue where to start from, it's more likely to converge on the "right" local maximum.

So there you go - no more magic.

Veiled Historical Reference

There's a lot of crazy, crazy people in this world. Trust me. I know.

20 Oct 2005 (updated 30 Oct 2005 at 19:31 UTC) »
Disturbing News

I got back to Montreal tonight and flipped on the TV. The local CTV news was just starting. One of their only two headline stories:

"Parents are demanding that the city place a crossing guard outside an elementary school... (dramatic pause) and it seems the government is listening."

Yikes! We need crossing guards now? Outside of our elementary schools? And the government is listening? What is this country coming to?

English Grammar is Almost As Dense as Perl

There, no kitchen is an island.

There is no kitchen island.

There is Kitchen! No! Island!

There is no island, Kitchen.

There is Kitchen Island, no?

No! There is Island Kitchen!

Kitchen is there. Island, no.

Is there Island Kitchen? No? Is Island Kitchen -- no, there! Island Kitchen is there, no?


So please don't ask me why compilers never auto-correct your punctuation.

6 Oct 2005 (updated 6 Oct 2005 at 01:33 UTC) »
Super Powers

adewhurst, I think you'd find that clairvoyance would get you a lot further than telekinesis, if you used it the right way. You don't really need both.

Cryptic Insight


27 Sep 2005 (updated 5 Oct 2005 at 22:12 UTC) »
No wonder things didn't make any sense!

UI design isn't specification, it's design. You'd think I would have guessed from the name.

That makes a big difference in how you should approach it.

Backdate (2005/10/05): Not really an update, but it appears I was thinking of this same stuff about a year ago, without coming to an answer. Spooky.

Membership Control and Groupthink

sfllaw expands nicely on the topics I brought up in my previous entries, but agrees with me a bit more than I'd like, so I'll have to resort to arguing with myself. That, incidentally, is exactly the crux of what I'm about to explain.

In one class in first year Engineering, Professor Aplevich mentioned in passing an effect called Groupthink. It's really important, and it stuck with me.

Imagine you take a group of four people, and you give them the basic outline of a project. Isolate the group from co-workers, other groups, and especially customers. It doesn't matter much whether the members initially agree or not, as long as their personalities don't totally clash. Make them work together on their project - four people on a single project can be a very tight team - for a few weeks or months. Now, let them out of their cells, er, cubicles and see what they've accomplished. Two things are almost universally likely:

- Despite any number of initially differing opinions, the group is almost always in final agreement about the way they executed their project.

- The mutually agreed-upon project is usually complete garbage because the group has lost all perspective of reality. Trying to convince the team members of this, however, can be extremely difficult.

It all makes sense, of course: over time, all the objections that four people can think of will be addressed, and they'll eventually come up with a mutually satisfactory "point solution" - one that deals with all the problems any of them can think of. But there are lots of solutions that fit that small number of points. Unfortunately, reality contains many more points, and the solution that a small, isolated group will come up with is almost always incompatible with reality. In fact, a solution that all four members wouldn't agree to, but which agreed more closely with reality, would be a much better solution.

Now think about the mechanics of Membership Control: how can it not lead to Groupthink? Strictly controlled group membership is effectively self-imposed thought isolation; members of the group aren't exposed to reality, because they deliberately block out the aspects of reality they don't like. We're free software developers; we don't talk to those Proprietary Weenies. We're techies; we don't talk to those Business Weenies. And so on.

The result is exactly the result you get from Groupthink: a bunch of people who can work together towards a common goal and produce a self-consistent solution that fails completely in the real world, because the real world is larger than the group's sphere of understanding. Free software misses problems that are obvious to people making proprietary software, and vice versa. Brilliantly cool stuff designed by isolated techies can't sell, and ideas proposed by marketing are unimplementable.

You can surround yourself with people who agree with you - not just because they're Yes Men, but because they really really believe in the same things you really really believe in. In fact, the Internet guarantees that, no matter who you are, you can find such people. But it doesn't mean you're right.

Membership Control always leads to Groupthink. If you are right, this is amazingly efficient. But chances are you're not. If you're not, who will tell you so?

sfllaw seems to have gotten interested in my last entry, and I'm trying to procrastinate right now, so I guess I'll fill in my next segment a bit sooner than I planned. Read Simon's analysis here.

(Quick response: Simon says many correct things, but I think he suffers from a bias towards ignoring the Exclusivity rule. Most Open Source-type people do, and the failings of Open Source compared to commercial software - and yes, there are some - can largely be explained by this.)

The Meaning of Power

The trick here is not to disprove one of the four rules, which should be difficult; each rule is pretty self-contained and almost "inherently obvious." The trick is to find a way to obey all four rules at the same time. To do that, you have to realize that one or more hidden assumptions - what you used to make decisions for action based on the rules - was wrong.

I'm sure there are lots of "premises" - fundamental assumptions - you can reject in my earlier analysis. That seems to be the way life works. Here's the one that fascinates me the most: the meaning of power.

Power over people comes in many forms. The simplest form is physical coercion; if I push you over a cliff, you're just plain going to fall off the cliff, and all the Quantization rules in the world aren't going to save you. However, this is virtually worthless for getting intellectual work done. You can force someone not to think, but you can't force them to think.

Another kind of power is a bit closer, and many organizations (especially military, and other highly hierarchical organizations) use this as their defining structure: the threat of coercion. There are consequences if you disobey - sometimes even death. So you obey. By doing this, you're "taking responsibility" for the consequences of your actions, ie. Not Getting Punished, so you're in compliance with the four rules. But now you're getting into my earlier comments about dueling stupidities in Company Policy.

There are other kinds of power as well, which are much more subtle but which I think can work much better. Here is one, which Simon also alluded to in his response earlier.

The Power to Control Membership

Also known as "hiring and firing." Here's how it works: if someone agrees with what you're doing, you hire them. If they disagree, you fire them. Repeat this process enough times (people's opinions change over time, of course, and so do yours) and you eventually converge on a group whose interests are aligned. There's no longer any reason why the person with primary responsibility will have to worry about disobedience from the others; the others have a natural tendency to want to do exactly those things that he wants them to do.

Note that this method is different from the threat of firing: that's just the threat of coercion again. There is no fear involved (strictly speaking) in the Membership Control method.

I think people underestimate how much Membership Control is at the root of the problems/successes with most organizational behaviour, because it sounds like such a blunt tool. Firing people just because they don't think the right way? Ouch!

But you do it anyway. When you voluntarily leave a group, you do it to yourself. How about these examples:

  • People who use HTML-only mail get ignored on technical mailing lists.
  • The Apache Core team is selected based on how well the candidates' interests align with the interests of the existing core team.
  • Debian won't let new members in unless they agree to the DFSG (and nowadays, possibly other rules). Conversely: there are no Debian developers that are violently opposed to Free Software, even though such people (often very smart ones) exist.
  • The most successful companies attract the best job candidates.
  • People who really love video games and are willing to work long hours (everyone knows game developers work long hours!) apply to video game companies.
  • "Techies" don't really like "suits", and vice versa. Every company has its place along that continuum, and you won't work there if you can't handle the particular environment.

You can talk about charismatic leaders, information sharing, and natural consensus, and there's plenty of truth to it, of course. But in real life, organizations often stick together simply because the members self-select themselves for compatibility with the goals of the group. Once you have that, your charismatic leader has a much easier job.

67 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!