Older blog entries for apenwarr (starting at number 131)

109 days. So there.

I had to change a lot of things about myself and the company in order to break the curse, but the core is intact and finally it's self-consistent.

Albeit in a different way than I was expecting.

The distinction between "traits" and "roles" seems to be a popular topic in object-oriented design lately. Once you think about it, the distinction is obvious, but when you mix up the two, your programs are crap.

Being the Perl god is a role. Being a stubborn cuss is a trait. :-)
-- Larry Wall

The same distinction applies to designing companies as well. People are defined by their traits; what they do are their roles. I've been terrified in the last few days by the confusion between these two concepts: people who actually believe that your role defines who you are.

"You're in marketing now, why should I listen to you about how to design this?" WHAT?? After four years of working with you as an architect, it turns out that my architecture skills depend on my job description?

A sledgehammer is big and powerful. Someone with excellent sledgehammer skills can use that sledgehammer to great effect. But if the role is swatting flies, the sledgehammer is just not going to work. Meanwhile, even an idiot can use a flyswatter to swat flies.

Traits aren't the same as roles. Your traits very much determine your effectiveness at a role. But yeesh, changing your role doesn't change your traits! How could it?

Architecture is Strategy

Someday I hope to be immortalized for my meaningless wise-seeming quotes, in the same way as "The network is the computer" or "The medium is the message" or "The chicken is the egg."

And so I bring you: the architecture is the strategy.

At the risk of explaining myself and therefore seeming less wise and inscrutable, what I mean is: the strategy of a company is its long-term goals. With every little short-term decision, you need to be thinking of how it will move you one step toward your long-term goal. Most short-term decisions that seem beneficial won't get you where you want to go; they'll just keep you alive. The magic of a good strategy is that it takes not much more work in the short term, but you actually get somewhere in the long term.

It's the same with software architecture. Sometimes you make a decision to implement something one way instead of another because you know that you're going to be better off in the long term. "Sure, I could just use global variables for everything and would save me some time today. But none of my functions would be reusable later." But what if you have no long-term plan? If you're a consultant, for example, developing precisely whatever someone pays you develop on each contract, there's no pattern to the work, and your short term gain is your only gain. This can be profitable, but you'll never get anywhere but where you already are.

Some people refuse to think beyond the scope of their current project; some people, even beyond the scope of the current phase of their current project. Those people don't understand strategy and don't want to; they'll work for someone else's strategy, if you pay them, because they don't particularly care where they end up in the long term. People like that make good workers, but terrible architects. Because if you don't understand the strategy, you don't understand how the work you do today is torpedoing the work you'll need to do tomorrow.

This also explains why business non-strategists don't see the value of software architects. It doesn't fit into their world view. If we can do our work today, and we don't know what's coming tomorrow, what good could a different architecture do?

I don't know if this story is true or not, but I heard it somewhere and it makes a point, so here we go.

A maker of cake mixes was investigating two cake mix designs. One, their traditional recipe, included powdered egg in the mix, and a new one required the end users to add their own eggs. Chemically, the result was about the same, but a competitor's "add your own eggs" mix was starting to gain market share over their existing all-in-one product. They decided to do some market research to find out why.

They conducted a taste test in a statistically valid fashion, with double blinds, etc. Presented with two cakes side by side, tasters on average had no particular preference for one mix or the other; they tasted the same.

This was very nice, but didn't explain the problem: why would the new market entrant be getting so popular, then? They had less advertising, no name brand, and you had to add your own eggs!

They had an idea and decided to run another test: this time, the tasters had to also bake the cake themselves, and they obviously knew which cake was made from which recipe (but were hidden from the brand name, packaging, etc). In this test, the tasters clearly preferred the cake where they had to add their own eggs.

Why?

They had more of an emotional investment in the egg-added cake; it contained "fresh ingredients" that they had to buy and add themselves. It took a tiny bit more work. They didn't feel like they were cheating as much when they made a mix that didn't have all the ingredients built in.

Ironically, we tend to believe that people buy cake mixes to save time and effort (ie. they're lazy); this makes the "add eggs" mix seem inferior. But what the researchers discovered was that people buy cake mixes not just to save time, but because they don't feel competent to add all the ingredients themselves (ie. they're scared). Making the end user do slightly more work makes them feel more productive and gives them more emotional involvement in the end product, without scaring them away with complexity. That's the first lesson.

The second lesson is about unit testing; the cake mix engineers were looking in the product specifications for why it wasn't selling, and constructed a perfect unit test to identify and help correct their assumed problem (if the eggs-included recipe had measured worse, they could have improved the recipe). But the test, while scientifically and statistically valid, included some hidden assumptions that also needed to be tested: that taste affects the desirability of the product; that less end-user effort is an improvement. The proper test to find the problem was closer to an "end-to-end" test: it included much more of the "cake mix consumption" process in the test, not just the part where you eat the cake.

And the third lesson is about the Zen of cake mixes and how capitalism has helped us get closer to it. Years ago, under adult supervision, I remember using cake mixes that didn't require you to add your own eggs; they were just fine. But how many mixes can you find nowadays that don't require eggs?

People have told me that the only way to win at negotiating is to "be willing to walk away."

So what happens if you've aligned everything perfectly to make your point logically unassailable, and you are willing to walk away? Perhaps you then realize: if you were really willing to walk away, then you didn't care about the outcome anyway.

If that's what it takes to win, then you have a pretty funny definition of winning.

That kind of negotiation is a "win-lose" negotiation: if you're trying to negotiate in a non-compromise way, then there is no "loser" from an exchange. There is only the "right" answer that makes both parties better off.

I think the best negotiations are the ones where everyone cares too much too walk away. That way everyone wins.

But if you're going to believe that, then beware the sort of person who is willing to walk away. They exist, and they'll eat you for lunch. And they won't even care.

Did I mention that we had 30 of them and they're 2 feet tall and have Nitix T-shirts? That's 60 linear feet of bunnies!

As we distributed the bunnies, it was very interesting watching us cross The Tipping Point as awareness of us (well, at least our tradeshow booth, and maybe our product) began to rise exponentially.

Also, all sexism aside, the bunny promotion would have been quite ineffective if there weren't a much higher percentage of women in the accounting software business than in normal IT. It was very obvious which gender was and was not bunny-focused. Not that this should be very surprising, I suppose, but it's interesting to note since women are of course badly under-targeted in technology marketing. That's probably a major reason why the gimmick was successful.

Gratuitous Misuse of Technology

Except in the US (where there is a person named Mr. Meese), Google Trends says mooses are more common than meese. This is of course very disappointing to me. In an incidental but Linux-related note, Finland is by far the most opinionated on the topic, with almost no searches for "meese" compared to "mooses."

I currently live in a miniature domed city with artificially increased oxygen content and humidity, tropical plants, and lots and lots of fountains and waterfalls. I like it.

The "bad" news is it's unlikely I'll actually see the outside of this little ecosphere (ie. the rest of Nashville) before it's time to go home. That means my opinion of Nashville will be 100% pure bias. But it does seem nice.

Marketing

Today, my life is truly complete, for I have witnessed a pile of 30 stuffed Nitix bunnies, all about 10 times cuter (and cuddlier) than I ever imagined. I don't care anymore if this marketing gimmick works. *I* want them *all*. And I don't even really like stuffed animals.

Note to self: bring your camera next time you go to a conference, idiot.

Direction

You know you're doing something right when actual respectable people start wanting to deal with you. Today I helped an obviously really, really good technical trainer tune his 3.5 hour presentation (which people are paying to go see) about our product. And he asked some really good questions.

I remember a few years ago when I first saw the "SalesDance" video of people actually selling all this stuff me and dcoombs had made up, like Tunnel Vision and Double Vision and idb. It weirded me out, thinking I had screwed up, err, improved the lives of so many people. This is the same feeling again, because it's really a new phase: people we're not even paying are spreading this stuff around. And that means they really mean it.

It's an honour, really. It feels weird.

Tonight I found out that it's actually possible to discover, to your dismay, that things are even more like what you thought they were than you thought they were. And so the same solution is, of course, that much more necessary.

Museums

Montreal free museums day is Sunday, May 28. I have marked it in my calendar. Have you?

There are some people who just can't help it - certain things just go better when they're around. I write about design a lot, but you can lack design ability and still make things better. And it's not really about motivation, because it can happen despite your every intention to stay out of the way. It's not even about raw skill or talent - there are lots of people with great talent who go nowhere because they don't use their talent for anything. A combination of design, motivation, and talent can be a powerful one, but that's not it either; that's just a good substitute, in case you don't have it.

Let's call what I'm talking about the "spark." Think of it sort of like a nuclear reactor in a person's core, spewing energy and radiation everywhere. Sure, you can direct the energy and do amazing things; but even if you don't direct it, or you consciously try to contain it, the excess energy has to go somewhere. It leaks out, or it explodes.

I have great respect for motivated, talented people. Plop them into a well-defined engineering or business process, and good, useful things get produced pretty efficiently. That's nice.

People with the spark aren't quite the same. They might also be motivated or talented or both, but they might not. The spark isn't exactly an object of respect - it's more an item of curiosity. People with the spark are interesting. People without it, however nice, or useful, or productive they might be, aren't.

The spark is hard to isolate, because its symptoms are mostly the same as symptoms of other things - motivation or talent, for example. I make a game of trying to see if it's there or not. This turns out to be a useless skill by itself, because a spark, by itself, is uncontrolled energy that will mostly just randomly change things. To actually accomplish a major goal, you need more than just a spark; you need the motivation to follow through, and the talent to be able to fill in all the extra details.

Still, there are reams of books written about how to keep people motivated, and how to train people to improve their skills, and how to identify people's key talents. All those things can be added on later. But I don't know how to generate a spark that's not there.

The most interesting people to me are the ones that have the spark, but are failing to get anything out of it. They have the energy, but they don't know how to control it, and the leaks are just random. With just a few little tweaks, you could get something much bigger, much more useful, and at least a little bit less random.

The most reliable way I've found to identify people with a wasted spark is this: they keep producing small, good things, but can't explain why, and can't seem to sustain something bigger. Some entrepreneurs have the spark, but many fail because they lack dedication or the right skills. Some artists have the spark, but many just have enough skill and dedication to fake it convincingly to the untrained eye.

I'm convinced that all the really interesting things in the world come from one person's spark; the tragedy is how few of those sparks actually go anywhere in the end.

The core of any design, whether it's a painting, a building, a computer program, or a company, is its "Raison d'Etre" - its Reason for Being, or, with conveniently the same initials as the French version, its Reason to Exist. Its RE.

Your skill as a designer depends on your ability to clearly imagine an implementation for your RE. And the degree to which the different implementers of a design understand its RE - or else, the level of control you give you people who don't understand the RE - strongly affects the overall coherency of the result.

Let's say the RE of your software project is the customer requirements or functional spec (they're different things, but since one leads directly to the other and they both come earlier in the pipeline, you could think of either one as the RE of the software). For some software, this isn't the case; some software, like Plan 9 or Scheme, exists for some other kind of RE that allows it to attain beauty while sacrificing usefulness. But in general, software with customer needs at its core is the kind that makes money, and it still has the potential to be beautiful. And as we all repeatedly learn, software which is beautiful can easily still fail to make money. The two attributes are really independent. But the kind of software I really like, the kind that makes money and is beautiful, works because it's consistent with its RE (ie. it's beautiful), and its RE is a well-defined set of customer needs (ie. it's useful).

A company's design works similarly, but there's a problem: as you progress through the technology lifecycle, the way you service the market needs to change, so if your RE was based on a particular way of acting - for example, doing heavy research for the early market - it obsoletes itself. The result is a strangely rotten core around which you've built a highly consistent implementation; you remember the beauty that existed before, but now that the core is different, the beauty is gone, even though the implementation is exactly the same. And trust me, although no person is to blame, the designer finds all this to be rather a PITA. People who enjoyed building something beautiful don't want to continue working on the implementation if they know the result won't be appreciated anymore.

Imagine you're writing a love letter to someone, and it's coming out really well, and suddenly you realize that person actually has really bad breath and glowy red eyes and you don't love her after all so you break up. As soon as this happens, the work you've done so far will go sour. Strangely, people who don't yet understand the newly-discovered flaw can still see the beauty of the work you've done so far, because they imagine a RE that is still there. But you, the artist, will never be able to honestly finish the work. Since beautiful work comes from an honest implementation of the RE, once you know the RE is gone, you will find that everything you do just makes it worse. If you're famous enough, it's better to just leave the unfinished work in a museum so a curator can dust it off occasionally and sell retouched reprints at huge profits. At least then people can continue to admire the beauty that once was. Don't ruin it by trying to keep building a fake implementation around a RE that has been lost.

Meanwhile, just as a work of art is meaningless without a RE, so is an artist, and a good artist won't take long before finding a new RE. If he's lucky, it will be one that's as compatible as possible with the old implementation; even the old implementers.

It's time to fall in love again. And we've got a half-written love letter that we can recycle. I'll show you how.

Normally I'm a programmer with a high output rate, which means that I can turn ideas into code pretty quickly. But what with my self-imposed marketing responsibilities lately, I've had almost no time for coding.

The problem is that I still think about coding. So my designs are ever-improving, while the amount of code I write has been ever-decreasing. A few times lately at work I've gotten caught explaining how I remember certain code working, only to find out that I only imagined it that way in excruciating detail; nobody implemented that feature or even talked to me about it yet. Oops.

But another weird effect from this is that my code design quality is improving. I spend so much time pre-considering things lately that when I finally actually write it, I've got it all figured out. But it results in what I will name the "pphaneuf syndrome": the problem is solved so thoroughly by so little code that nobody can believe your code actually solves the problem... or that you did any work.

Here is my example:

The Gracefultavi "Attach" Plugin

GracefulTavi is the wiki we use for our internal documentation. Over time, it's been used for more and more things, and we're finding the need to write more and more "formal" documentation (eg. specs that need to look nice for presenting to clients, who will sign a contract to pay us hundreds of k$ for implementing them).

So our first problem is simple: the wiki has all our development information, cross-referenced in different ways, because we've used a wiki since day one and we've done all our development documentation (with a few, ironically unremembered, exceptions) in the wiki. But it's not customer-grade documentation; sending customers wiki page dumps is pretty lame.

Secondly, even when no customer contract is involved, writing specs in wiki format is really annoying, because (like most traditional wikis) GracefulTavi doesn't support attachments and screenshots. As with any html, you can link to outside attachments and images, of course, but these are hard to administer properly and definitely break with the "anybody can fix your mistakes" wiki tradition.

Thirdly, we're trying to implement internal processes whereby (for example) even an internal spec can get signed off by several people and that version becomes completely tamper-proof: despite the "anybody can edit" wiki tradition, anybody can edit, but nobody can edit *that version*. And that version might contain images, tables, attachments, and so on. And there will be more versions after that. And you might want to refer to those same versions of those images and attachments from other pages even when new versions appear (for example, the "1.0 test results" want to reference the "1.0 test plan" even after 2.0 is released).

Thanks to mich and kjrose, it's easy to add new macro plugins to GracefulTavi. So I did... in about 1.5 hours. But that very short time was possibly only because I thought about it so much first. (*)

Other wikis, like MediaWiki also support attachments, but in a much more extensive way that took a lot more coding effort. Here's how my solution works. For most purposes (probably not WikiPedia), mine is better. You don't have to believe me, but it's true.

My attachment plugin has only three concepts: upload, delete, and lock. (There's also "undelete" and "view", but those are too obvious.) Upload is simple: a non-uploaded attachment is empty, and has a file submit button. Delete is simple: an uploaded but non-locked attachment has a delete button, which will permanently delete the file and let you upload a new one. And lock is simple: an uploaded but non-locked attachment has a lock button, which will permanently protect the file from deletion. Preventing deletion, in turn, prevents re-uploading.

The filename space is global. That is, across the entire wiki, if you upload a file called "foo.txt" and lock it, that's it; there will never be another foo.txt.

Despite its ridiculous simplicity, this model is not nearly as obvious as it sounds. Permanent deletion - in a wiki? Permanently locked documents - in a wiki? Names that nobody can ever change ever again? An attachment scheme with no revision history or version control - in this day and age?

Yes, yes, yes, and yes. And only those four things together make it work. In other words, breaking any of those "common sense" rules would have resulted in an unusable system; breaking all four of them at once is why it works.

Here's why. First, the wiki already has revision control; adding another versioning layer for attachments would be inelegant. Instead, what you do is encode the version into the filename. Sounds lame to you, Subversion users? Just remember, the kind of people who prefer binary documents to wiki documents are exactly the kind of people who don't like Subversion either. Furthermore, fully revision controlling binary documents isn't useful, because the diff/blame/branch/merge features don't work anyway. And auto-assigned revision numbers are meaningless, while human-assigned ones are memorable. And because you can't re-upload a locked document, nobody can change your document once you're sure you've uploaded it correctly - in other words, we implement the one remaining feature of Subversion that's actually useful for binary files.

So you can easily do 100% protected revision control with my plugin, which doesn't even support revision control. Don't upload "myproject-spec.doc" - upload "myproject-spec-1.0.doc", ie. the 1.0 version of the spec - and lock it. From then on, anybody who wants to talk about the 1.0 spec (on any page in the wiki, past, present, and future) will be confident that they're talking about that exact file. And the MyProjectSpec wiki page can link to that file for now... and in a future version, it can link to "myproject-spec-1.1.doc" instead or in addition. What did the spec look like last September? Go back in wiki history and view the hosting page, and it definitely points at the same attachment it did at the time. No magic auto-assigned rev numbers to remember, no branches, no tags, no syntax, and no uncertainty.

The pent-up demand for this was obviously huge, because within days it was being used all over the place in the wiki, and the usage is ever-increasing. Meanwhile, some people are still ironically claiming that it can't possibly solve the problems it intends to solve.

I say... the results speak for themselves. Which is good, because the above explanation is really long and confusing.

Off-topic Disclaimer

(*) People will say that we could have saved a lot of time arguing about it internally by just doing the 1.5 hours of work in the first place, but that't not how design works; I wouldn't have spent so much time (much more than 1.5 hours) thinking about the problem if there weren't so many arguments about it before. And just leaping into coding with a bad design would have taken more time than thinking/talking about it first.