Following a link, I stopped by the ANU library this afternoon to look at the IEE Proceedings - Software special issue on open source. (It's only available on paper, not on the web... how quaint :-)

The Asklund and Bendix paper I was originally looking for was pretty interesting: they have some insightful things to say about the way configuration management is done in open source, as compared to conventional development. For example, your father's CM textbook probably shows proposed changes going to a Change Control Board, and if they are approved they will be implemented, integrated, and QA'd. The open source way is generally to implement first, and then approve or reject. There are a few interesting observations along these lines, based on interviews with people from Mozilla, KDE, and the Linux kernel. Perhaps nothing earth-shattering, but interesting nontheless.

Some of the other papers were really deeply disappointing though. I'm not talking about incorrect technical details about Linux -- that would be quite forgiveable -- but gaping assumptions that ought to be obvious to anyone with some kind of scientific background. Off the top of my head I could name a handful of counter-theories that would equally well explain some of the results (either pro- or anti-open source.)

I hesitate to go into details because I don't have time to go over all of them carefully, but after all this is just a diary so I'll go ahead: the "Trust and Vulnerability" paper is desparately in need of the thoughtful statistics-based assessment of program reliability that informs, for example Ross Anderson's recent paper. It's completely missing; they analyze a single variable when there are obviously many more and the result is completely unconvincing. The thing that makes the security-vs-obscurity question essentially hard is that you need a complex model of the various communities; they missed the point as far as I can see. The overall result is so poor as to be not even worth criticizing.

I'm don't think I've read that journal before, so I'm not sure how this compares to their usual standard. It does really seem like a shame, because academic SE can be very worthwhile, but at least in this instance it seems disconnected from open source. The first derivative is good: people are seeing open source as being serious, as having something to teach the rest of the world. But it still requires more work on both sides to build a good understanding.