Older blog entries for apenwarr (starting at number 138)

14 Jul 2006 (updated 15 Jul 2006 at 02:48 UTC) »
Optimizing for Productivity

I was reading in the aforementioned Good to Great about different "great" companies and their way of reducing their vision to a single "hedgehog concept," which has various useful boring attributes that I won't go into right now, as well as one more that I will: an optimization goal, which they call the "key denominator." Basically, to be maximally financially successful, a company needs to optimize its operations for the best ratio of profit to something.

Some companies use things like "most profit per store" or "most profit per product line." They talked about a bank in the U.S., which optimized its operations for "most profit per employee" when banks became deregulated. The most obvious way to do this is to reduce the number of employees and/or otherwise chop your HR costs, but in a world where banks are suddenly not protected monopolies anymore, that might be the only thing you can do.

That particular optimization goal got me thinking. I don't really like big companies very much. Once they get too big, they get annoying because it's hard to keep track of everyone and there are always some stupid people. So optimizing a new company in terms of profit per employee - which isn't nearly so painful in a company with no initial employees as it is when you'd have to lay off a bunch of people - might be quite entertaining.

And then I thought, wait, that's not quite it. You can increase profit per employee by simply making your employees work longer hours and paying them less, which isn't really what I want. Okay, so I work really long hours, but that's just me, and it's still sort of cheating; it makes me look more productive but I'm only producing more, not producing more per hour. If I was really smart, I would be able to produce more per hour than the next guy. If I also work more hours, well, great, but that's just a bonus.

And so that's it. We want the smartest, most productive people around, and the people we get, we want to make even smarter and more productive. So why not maximize profit per man-hour? Why not do it all the way across the company, and why not help your customers do it too?

In other words, laziness, impatience, and hubris. It's not just for programmers anymore!

A few loosely correlated comments this time.

Stubbornness is a Virtue - Sometimes

I've been thinking a lot lately about what makes someone a good high-level leader. There are lots of things to consider, of course, but here's one of the things that has struck me most in the last little while: the best leaders are stubborn and also good listeners.

What, it's possible to be both?

Yes. People who are stubborn persist in a particular way of thinking despite encountering huge resistance. If you're trying to convince someone about a revolutionary new idea that will solve their problems, stubbornness is critical. Meanwhile, people who are good listeners adapt their thinking when someone convinces them that they've made a mistake or were missing some information. If you're wrong - which is frequently, if you have a lot of revolutionary ideas - you have to be a good listener in order to correct your mistakes fast.

The proper combination, though, is important. The best leader will persist in thinking something until someone properly convinces them otherwise. Once they've been convinced, then they change their mind and they're stubborn in the new way. That's not flip flopping; that's just a smart way to learn from your mistakes.

If you're missing one of those two traits, you'll fail in one of two very common ways: you'll persist in believing something stupid even it's been made totally clear and obvious to everyone else that it's wrong. Or else you'll never settle on a single direction, constantly changing your mind whenever anyone raises an objection.

Some people take it a step further and switch back and forth between being too stubborn and being not stubborn enough. One example I've seen a few times is a stubborn belief that "the investors are always right." So whatever it is that the investors want, you'll give them... even if they change their mind day after day. So you're left spinning in circles because of your persistent belief in someone else's idea. Other popular variations of this are "the customer is always right" and, most annoyingly for me at the moment, "the CEO is always right."

So my suggestion: believe in yourself. And then don't be afraid to let other people try to change your mind.

Trust and Respect

On another note, I was thinking about what would be the ideal company culture. This goes back to my earlier notes on membership control for working in a group.

I think a culture based on two primary concepts would be nice: trust and respect. It works like this:

First, you need to trust that people will be honest and upstanding (if you're paranoid, you can check their backgrounds before you admit them to the group). Since you trust them, you don't need annoying controls to make sure they're not trying to screw you, because you're not worried about that. If anyone ever gives evidence that they can't be trusted, remove them immediately; there's nothing more deadly to a social environment than a person who can't be trusted.

Second, you only admit people to the group whom you respect. That is, if you don't think they can do a good job or think clearly or impress customers or whatever, then you don't want them in your group. (In his book Blink, Malcolm Gladwell discusses a researcher who has discovered the #1 indicator of marriage breakups: a spouse who obviously looks down on the other.)

These two factors work together to produce interesting results. For example, if you respect someone, you will automatically become more trustworthy; would you screw over someone that you respect? Would you slack off at work, if it would let them down or hurt the project they're working on? Probably not. And that means they can trust you even more.

Meanwhile, because you respect them and so they trust you, you're actually working harder and producing more. That makes it easier for other people to respect you. And since they do, they won't want to screw you over, so they'll be more trustworthy and work harder. And so on.

This system covers most of the other factors in building a group as special cases. For example, if you're not a good listener, you obviously aren't giving the people the respect they deserve when they talk to you. And if you're not stubborn - if you just keep changing direction randomly - then eventually people will realize that you can't be trusted, because your opinion is based on whichever way the wind is blowing at the moment.

And about that Smart and Gets Things Done rule? Well, if you're not smart, that's going to show up pretty fast, nobody will respect you, and you're out. If you don't get things done, then you can't be trusted in an emergency, and you're out.

This system even deals with interpersonal compatibility in an interesting way: different people respect other people for different reasons. For example, a group of musicians might highly respect someone with a lot of musical talent, regardless of their table manners; a group of aristocrats probably wouldn't. (They can buy musicians for a dime a dozen, but it takes money to become an aristocrat!) This is interesting because the groups are probably fundamentally incompatible. If each group selects its members based on the trust/respect rule, things should rapidly work themselves out into two separate groups.

And so on. I like this rule because it makes the "membership control" system easy. I wouldn't feel the least bit guilty about kicking someone out of my company if I don't respect their abilities or if they can't be trusted. And yet those two things cover all the other important things.

Good to Great

I just finished reading the book Good to Great. It's interesting and makes many believable points, but it does scare me a bit with its selection bias. They tried really hard to avoid sampling errors, but didn't quite make it: the problem is that highly risky behaviours are correlated with both huge success and huge failure, while highly unrisky behaviours are correlated with mediocrity. So when you compare a set of "great" companies with a set of "merely good" companies, you will definitely find some statistically significant differences. Those differences might very well be absolutely critical to becoming great. However, they might also greatly increase your chances of killing yourself.

To name just a couple of examples, the "Level 5 leadership" concept makes perfect sense in terms of growing a company for the long term. But if you're "almost" a level 5 leader, then maybe you're just a quiet person with a vision and nobody really listens to you and so your company goes in random directions. Or the "hedgehog concept" - do one thing, and do it well. This is great, as long as it's the right thing. The book even gives some good advice on how to choose the right thing. But what if it's not quite the right thing? Then you're screwed. Massive diversification may prevent greatness, but at least you can hedge your bets.

Beware of advice that seems too good to be true.

Dynamic Equilibrium

I wrote before about precious instability, the idea that I design my life in an "unstable" fashion, then carefully control it, in order to make it more manoeuverable. For people who know me, some recent events might have put that into a bit more context.

But one thing I realized since then is that this concept is much more widely applicable than just me and my idiosyncratic ways. Modern high-tech business requires this kind of instability, but only when it's in the right places.

Put simply, there are two fundamental ways to manage any project: creatively (ie. a prototype) or using a well-defined process. Well-defined processes are efficient and scalable; creative solutions are resilient and flexible. (I wrote about efficiency vs. resilience before too.) An obvious way to balance the two is what I've been trying to do lately: do it (whatever "it" is, even if it's a business activity) the first time as a creative prototype, then try to generalize what you did into a repeatable process. As with software, making a mostly-right generalized design is usually possible, for a good designer, after you've done it about twice creatively.

But once your process is well-defined, then what? I've experimented with the "hand it off entirely to process-driven people" technique, but that often doesn't work very well, because if your process isn't perfect at the start, process-driven people aren't very good at fixing it. Meanwhile, trying to find some middle ground ("we'll follow the process, but bend the rules whenever necessary") is a disaster, because it's not a combined solution, it's a compromise, and it ends up being neither very efficient nor very creative. Gross.

I don't exactly have a tried-and-true solution to this, but here's what I'm thinking. Joel Spolsky has written about The Development Abstraction Layer, which is a partly-right variant of what I'm going to suggest. What he wants is a programmer's ideal world: the programmer just builds what he wants, which is assumed to be the right thing, and the rest of the company is an "abstraction layer" that gets all the annoying repetitive cruft out of the way. In terms of creativity vs. processes, that means the "developers" are creative, and the rest of the company delivers things based on defined processes (or at least inter-group communication follows defined processes).

The part where I disagree with Joel is the part where I've already specifically screwed up so I figure I can serve as a warning to everyone: developers shouldn't be the ones running the show. Not quite. Smart people who get stuff done (another Joelism) need to be running the show, but it's not about programming and designs and version control and operating systems and XML. It's about business and customer relations and good communication and... being smart, and getting stuff done. It just turns out that Joel's kind of programmer is exactly the right personality type... if they're deliberately caring about the right things, which is more than just the software.

So how about this. Classify your people into two categories: the creative solutions people and the process-driven people. Don't separate them physically or logistically; they have to work together. The creative solutions people design and update business processes. The process-driven people execute those processes - maybe only after the first prototype - and give rapid feedback on results. Don't worry, Joel, both groups should be smart and get stuff done. But they're two different kinds of people. Some very smart people really have a great time fixing bugs and optimizing failure rates and avoiding creativity at all costs. (Seriously, there are lots of people like this!) These aren't my favourite people to work with, but they're extremely useful because they like doing exactly the kind of stuff that I don't.

At the same time, process-driven people risk falling into dangerously stable ruts and getting blindsided when changes become necessary, because they enjoy their stability and will try to retain it at all costs.

So as a business designer, you need to set up the incentives of the creative and process people correctly. My suggestion? Don't let the process people define their own processes. Make them responsible for executing defined processes efficiently and make creative people responsible for changing the processes to make them efficient.

I call this "dynamic equilibrium," after the term from high school science class. It looks pretty stable, because the majority of people are sitting around executing repeatable processes. It's just that it's not really stable, because exceptions will come up, and when they do, weird stuff happens automatically to re-stabilize the system.

Disclaimer: I haven't tried out this advice yet, but it sounds like fun.


Crap, I really wanted to work in my example about airlines losing my baggage, but it didn't fit... so here it is anyway.

Airlines process absurd amounts of baggage every day very efficiently, and lose a vanishingly small percentage of it. That's an efficient business process.

But every now and then they lose a bag, and they don't know what to do, because if they knew what to do, it wouldn't be lost, sort of by definition.

You need business process people around to deal with the majority of cases, where the bags aren't lost. And you need creative people around to handle the exceptions, learn from them, and fix the processes so even fewer bags get lost.

28 Jun 2006 (updated 28 Jun 2006 at 03:24 UTC) »
Practical Extraction

Well, that could have been worse. Presumably.

Pathologically Eclectic

Since people keep asking, yes, I am currently interested in projects related to distributed storage, network path optimization, unified configuration (UniConf, GConf3, KDE4 KConfig), and several other loosely related things. Like unified multi-contributor documentation and decent Unix printing and NaNoWriMo and Business Design (hah! See! I didn't make it up!) and and and... so yeah, I'll just pick one of those.

Also, because I'm obviously insane, I have subconsciously arranged to be in Toronto for the entire Toronto Fringe Festival. Ha ha. Very funny, brain. This is not good at all.

Unraveling at the Fringe

Several people have posted reviews of some Fringe Festival shows they've seen, along with their ratings. For me, though, the purpose of the fringe festival isn't so much to see or measure quality entertainment. After all, as I've mentioned, as far as art is concerned, I have no idea what I'm doing and thus it's easier for me to appreciate.

No, the purpose is to gather another year's worth of random input. So without further ado, I bring you: 40 tidbits of randomness, in alphabetical order by show name. (Yes, I have a leaning toward puns. So shoot me.)

Needlephobic spasms. No, no, his house - the Hur house; it's enough to turn gay men straight.

Wah - I love pizza - but I Kant!

People never to break up with; people never to buy dinosaur souvenirs from.

Airport jokes and flamenco made interesting; human relationships the MIT way.

Eat here - get gas for free! Dancing in the rain and the worst audition ever.

Brain sucking ingenious crazy dance! Anorexia personified, and at least one continuously improvised dance scene!

Dinosaurs aren't souvenirs, they're EOLed products. White clothes make great projection screens! Breakdance fusion!

Queuing was stolen from the British; also, There was a young man from Peru, whose limerick stopped at line two. Non-lost innocence and a squirt of toilet humour. Non-toilet-related private moments made public and an encounter with Sweden's orb of literacy.

Speaking in tongues, the musical. Verbal smilies!

The missing minute from my life.

You can't escape a guilty conscience... and eventually, your creations will take on a life of their own (ergh, tell me about it).

Nunavut -- twice. Dames in disdress, vs. a very different form of fake but extremely convincing shyness.

Always have a lawyer look over your contracts, particularly if you make them with the devil. Speaking of which: Stephen Harper and David Suzuki, together at last, along with Terry Fox's right sock.

Octopus dance (I wish I'd had a fork). Vegetable dance (enough to make you dizzy).

Learning English from Entertainment Tonight and a wishing well talks back to cheapskates. Criminals are people too.

Honourable mention, even though it was free and thus doesn't count: 11-second dance parties, 16-second slowdance parties (thanks, Cheer Politeness), 22-second dance parties, and so on. To heck with the popular kids! To heck with respectable theatre!

Peer pressure and mutual motivation

And now for the obligatory Advogato tie-in.

Last year at the Fringe Festival, I saw 26 shows in 10 days, and it was exhausting. When I set out this year, I decided: no sense overdoing it. I'm not going to try to beat last year's insane record. I'll just see as many shows as I want to see, then I'll stop.

So then I saw 40 shows. Oops. What happened?

What happened was peer pressure. None of us really wanted to see that many shows. But when you have a few people working together and encouraging each other, the positive feedback loop can be irresistable. "Come on, you can't be tired yet! I want to see one more!" "Come on, there's a show on - let's leave work early!" "Come on, you already bought the multi-ticket pass, so you might as well use it up!"

I learned a couple of years ago now that you should never let a person work on a project at work alone; it's hard for someone to get motivated that way. This is the idea behind pair programming, but you don't have to go that far; two people can work on parts of the same project, as long as their work is slightly mutually dependent - say, at least one reason to rely on the other person per day. Then if one person falls behind, they're disappointing the other person, and vice versa. This mutual motivation technique can be extremely effective, and it vastly outweighs the cost of having a second person working on the project, because it'll probably get done more than twice as fast.

So now, thanks to all that motivation, I feel like I've accomplished something. It turned out to be something ill-conceived and stupid; who the heck needs to see 40 shows in 10 days?? But I guess that's my point. Motivational techniques work, even if you're trying to get people to do ill-conceived, stupid things.

12 Jun 2006 (updated 12 Jun 2006 at 20:34 UTC) »
Independent thought

    The purpose of a designer is to make happy coincidences happen more frequently.
    -- me

Someone else's independent thought

    So, what benefit am I taking from all the freedoms I have because I am not a duck? ... That's the sort of question you should expect poets to be asking on your behalf.
    -- Jem Rolls
Trying and Being

    Do or do not. There is no "try."
    -- Yoda

The thing I find most interesting thing about yoga classes is the difference between how it feels when you're clueless (like me, usually) and when you know what you're doing (which I do in a few rare cases). When you're clueless, every position is a struggle, and it's really hard to maintain it for more than a few seconds. But when you actually figure out how to do one of the stretches properly, it becomes easy all of a sudden, and it doesn't feel like a stretch at all.

Sadly, none of the instructors have actually brought this up in class. You seem to be left to figure it out by yourself. But once you realize it's true, you stop trying to stretch yourself into a particular position, and you spend more time trying to figure out the trick of getting into that position properly. It's much more productive.

The rest of the world works like that too. And there are various philosophies that suggest that if you're truly enlightened, you can find the right place to stand and the world will revolve around you, working out just like you want.

But there's a catch. As much as it would be nice to be enlightened and just let things be the way you want, if you're not quite so enlightened, all that will accomplish is randomness or stagnation (depending where you land). In yoga class, you have to attain a certain degree of flexibility before you can even really start - and that takes solid effort. Once you get that far, you have to figure out how to do the different positions properly - and that takes effort too, albeit of a different kind. And the further you progress, the more convoluted it becomes.

And yes, I can tie this back to programming. I'm glad you asked.

I did a presentation at work a couple of years ago called "Coding Faster." In it I described the well-known concept of being in "The Zone" and also the opposite, which I called "Ooze." Most programmers aim to spend as much time as possible in the zone, but that's not quite right; the zone is being. You know what you want, and you know how to do it, and really: you get into the zone and it just happens by itself. (If you're a programmer and that's not how it feels for you, then you're doing it wrong. Trust me on this.)

The part people miss, though, is the ooze time leading up to the zone. It takes a lot of hard work to understand the universe well enough to be able to be properly productive in the zone. Ooze time, although it's slow and feels unproductive, is critical to overall productivity. Ooze time is trying.

So I'm afraid I'm going to have to disagree with Yoda on this one. There is definitely a "try." Anyway, he was probably just telling Luke that to keep things simple for the early lessons. But I agree that "try" is really not the same thing as "do," which I think was his real point.

The Curse is Broken

109 days. So there.

I had to change a lot of things about myself and the company in order to break the curse, but the core is intact and finally it's self-consistent.

Albeit in a different way than I was expecting.

Traits and Roles

The distinction between "traits" and "roles" seems to be a popular topic in object-oriented design lately. Once you think about it, the distinction is obvious, but when you mix up the two, your programs are crap.

Being the Perl god is a role. Being a stubborn cuss is a trait. :-)
-- Larry Wall

The same distinction applies to designing companies as well. People are defined by their traits; what they do are their roles. I've been terrified in the last few days by the confusion between these two concepts: people who actually believe that your role defines who you are.

"You're in marketing now, why should I listen to you about how to design this?" WHAT?? After four years of working with you as an architect, it turns out that my architecture skills depend on my job description?

A sledgehammer is big and powerful. Someone with excellent sledgehammer skills can use that sledgehammer to great effect. But if the role is swatting flies, the sledgehammer is just not going to work. Meanwhile, even an idiot can use a flyswatter to swat flies.

Traits aren't the same as roles. Your traits very much determine your effectiveness at a role. But yeesh, changing your role doesn't change your traits! How could it?

Architecture is Strategy

Someday I hope to be immortalized for my meaningless wise-seeming quotes, in the same way as "The network is the computer" or "The medium is the message" or "The chicken is the egg."

And so I bring you: the architecture is the strategy.

At the risk of explaining myself and therefore seeming less wise and inscrutable, what I mean is: the strategy of a company is its long-term goals. With every little short-term decision, you need to be thinking of how it will move you one step toward your long-term goal. Most short-term decisions that seem beneficial won't get you where you want to go; they'll just keep you alive. The magic of a good strategy is that it takes not much more work in the short term, but you actually get somewhere in the long term.

It's the same with software architecture. Sometimes you make a decision to implement something one way instead of another because you know that you're going to be better off in the long term. "Sure, I could just use global variables for everything and would save me some time today. But none of my functions would be reusable later." But what if you have no long-term plan? If you're a consultant, for example, developing precisely whatever someone pays you develop on each contract, there's no pattern to the work, and your short term gain is your only gain. This can be profitable, but you'll never get anywhere but where you already are.

Some people refuse to think beyond the scope of their current project; some people, even beyond the scope of the current phase of their current project. Those people don't understand strategy and don't want to; they'll work for someone else's strategy, if you pay them, because they don't particularly care where they end up in the long term. People like that make good workers, but terrible architects. Because if you don't understand the strategy, you don't understand how the work you do today is torpedoing the work you'll need to do tomorrow.

This also explains why business non-strategists don't see the value of software architects. It doesn't fit into their world view. If we can do our work today, and we don't know what's coming tomorrow, what good could a different architecture do?

21 May 2006 (updated 21 May 2006 at 18:28 UTC) »
Zen, Betty Crocker, and End-to-End Testing

I don't know if this story is true or not, but I heard it somewhere and it makes a point, so here we go.

A maker of cake mixes was investigating two cake mix designs. One, their traditional recipe, included powdered egg in the mix, and a new one required the end users to add their own eggs. Chemically, the result was about the same, but a competitor's "add your own eggs" mix was starting to gain market share over their existing all-in-one product. They decided to do some market research to find out why.

They conducted a taste test in a statistically valid fashion, with double blinds, etc. Presented with two cakes side by side, tasters on average had no particular preference for one mix or the other; they tasted the same.

This was very nice, but didn't explain the problem: why would the new market entrant be getting so popular, then? They had less advertising, no name brand, and you had to add your own eggs!

They had an idea and decided to run another test: this time, the tasters had to also bake the cake themselves, and they obviously knew which cake was made from which recipe (but were hidden from the brand name, packaging, etc). In this test, the tasters clearly preferred the cake where they had to add their own eggs.


They had more of an emotional investment in the egg-added cake; it contained "fresh ingredients" that they had to buy and add themselves. It took a tiny bit more work. They didn't feel like they were cheating as much when they made a mix that didn't have all the ingredients built in.

Ironically, we tend to believe that people buy cake mixes to save time and effort (ie. they're lazy); this makes the "add eggs" mix seem inferior. But what the researchers discovered was that people buy cake mixes not just to save time, but because they don't feel competent to add all the ingredients themselves (ie. they're scared). Making the end user do slightly more work makes them feel more productive and gives them more emotional involvement in the end product, without scaring them away with complexity. That's the first lesson.

The second lesson is about unit testing; the cake mix engineers were looking in the product specifications for why it wasn't selling, and constructed a perfect unit test to identify and help correct their assumed problem (if the eggs-included recipe had measured worse, they could have improved the recipe). But the test, while scientifically and statistically valid, included some hidden assumptions that also needed to be tested: that taste affects the desirability of the product; that less end-user effort is an improvement. The proper test to find the problem was closer to an "end-to-end" test: it included much more of the "cake mix consumption" process in the test, not just the part where you eat the cake.

And the third lesson is about the Zen of cake mixes and how capitalism has helped us get closer to it. Years ago, under adult supervision, I remember using cake mixes that didn't require you to add your own eggs; they were just fine. But how many mixes can you find nowadays that don't require eggs?

129 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!