As Dave has mentioned we've gone with O'Reilly for the book and we're going to be doing much of it online where everybody can see what we're up to. It's all very exciting.

Now that we have a gorgeous cover and a press release it all feels very real.

The book's called: Software Craftsmanship: From Apprentice to Journeyman and it's going to be in the same series as books like: Beautiful Code, Prefactoring and Practical Development Environments.

In the past couple of months I've been to 3 events that exemplified their communities. I'll describe them in decreasing order of fun.

BarcampLondon3 took place in the Google London offices. It was dynamic, self-organising and full of strangers. Everywhere I looked people were introducing themselves to each other. The sessions tended to be less than polished. In fact in any given session there would be a person writing their presentation for the next session. Despite, or perhaps because of, this the sessions tended to be filled with information that someone passionately wanted to share.

The people tended to be younger and more diverse (in every dimension) than the Agile crowd. For example Ian Forrester's presence meant that I'm not "the black guy." Practitioners like Matt Biddulph showed up and talked about the technologies being used by their startups. Developers from the BBC demonstrated unreleased systems and everywhere I looked I saw people having conversations about advancing the state of their art. It was invigorating.

XPDay 2007 was full of workshops, consultants and old colleagues. A lot of the sessions seemed to involve a presenter with the germ of an idea who expected the audience to flesh out the idea. This sort of 'stone soup' sessions can be a lot of fun but after a while they become monotonous. You start to long for a presenter with an agenda, an idea to share or even an axe to grind.

David Stoughton's talk was the only one that delivered. David (and his co-presenter whose name I've sadly forgotten) presented difficult ideas that generated debate and principled disagreement amongst the audience. I'm not sure I agreed with all the ideas they raised about brands, projects and value but they challenged my existing beliefs. That in itself is valuable.

The Agile community, to its detriment, has focussed on helping laggards improve rather than advancing the state of its art. This has resulted in boredom for experienced agile practitioners. Apart from networking, meeting potential clients and catching up with old friends in the pub there's precious little to do or learn at these conferences. It's just the same old people teaching the same old lessons.

I think this happened because there was a dearth of people from other communities at the conference. Glazier's Hall was an island populated by consultants and people who work in investment banks. As someone who used to be both I should have fit right in. However I spent large parts of the conference feeling disconnected from everybody else's preoccupation with enterprise IT. It seemed hard for people to adjust to the idea that there's a whole world of people building systems that aren't foisted upon a captive user base.

There was a presentation from a team at Yahoo Europe but I felt that they didn't focus enough on the lessons that other people could take from their experience of adopting Scrum. The key insight I took away from the first day of the conference was from Jeff Patton's talk. He enabled me to see the connection between the roles of the agile Customer/Product Owner and that of a Product Manager by showing that Product Managers are really just 'professional' Customers.

Apart from that the conference only reinforced my sense that Agile seems to be approaching senescence. We've forgotten our origins in lightweight methodologies focussed on finding ways to perform our jobs better. We've turned inwards and are re-iterating the same old ideas when we should be looking to see if other people have harder problems, newer challenges and different lessons to teach us.

I'd like next year's XPDay to reach out to a wider group of people and be more open to new ideas. This isn't just a matter of embracing the technologies that other conferences are using. We have to go further and try risky things like supporting an explicit 'hallway track' by having shorter sessions with staggered start times so that people have more time to mingle between sessions. I'd also like to see a BarCamp style track where random attendees could run sessions about whatever caught their fancy. This would finally give us a conference that could respond to feedback more frequently than once a year. I would also like to see XPDay inviting more 'strangers'. People like Dave Snowden, David Stoughton or even Tim O'Reilly could bring an infusion of ideas from outside the enterprise IT/consulting/academic circles that would really enliven this conference.

A little while later I went to the Royal Institution for a talk on Machine Learning by Professor Chris Bishop of Microsoft Research. In its heyday the RI used to host talks by Faraday and the leading scientists of the 19th century. Unfortunately the present administrators have confused the accidental and essential aspects of the Faraday era. So they've replicated all the features of Faraday's lectures right down to the servants holding the doors open for the speaker, the gong that indicates that the allotted hour of lecture time is finished and of course the "no questions" rule.

Consequently the middle-aged crowd of bourgeoisie spent 60 minutes being taught about Bayes' theorem, where Bishop was brilliant, and being bewildered by demos of fancy image editing features that Microsoft Research are working on. Since we couldn't ask questions we had to sit there in our black-tie outfits and hope that the speaker would pick up on our bafflement in time to tie all the threads of his presentation together. He didn't.

I left the session wondering why anyone would attend this kind of presentation in person if there's no interaction with the presenter. If I'm going to sit numbly while someone else pontificates then I might as well be at home watching TV. Sadly the RI doesn't really seem to solicit feedback so there's not much I can do about it except be disappointed.

I'm glad to see that Raph has finally de-cloaked.

The video from his Tech Talk on Lessons Learned From Advogato is now available. It's a candid look at the issues that affect any community that tries to use trust metrics to facilitate large scale collaboration.

Aggrevator is finally available (in beta). According to my LaptopWiki I started work on this in July 2003 because I wanted a web-application based aggregator to replace Flock. Since then I've played around with implementing the essential idea in technologies ranging from Python to Java. And I finally settled on a rich client application written in Java. It uses SWT (the same library as Eclipse) for the GUI and MySql for the database back-end.

There really ought to be a name for the process of searching for a license that is compatible with the licenses of all the libraries that you're using. I ended up going with GPL mainly because the MySql JDBC drivers use that license and all my other libraries are either LGPL, BSD or Apache.

It's good to see that Advogato is back up. Although I'd stopped checking to see if it was still up and I only found out because Google sent me a web-alert after re-indexing my diary here.

So what have I been up to since the last time? Well I switched employers, met mwh at this year's UK Python conference and I'm doing a lot less travelling.

In relation to things open source: I never did finish that OsCache guide. On the bright side I've made significant progress on Harvester: an RSS aggregator. Of course everybody and their dog has an aggregator nowadays but mine is different because it's based on storing everything (for offline reading, later searching, analysis, etc) in a MySql database and using scoring to order feeds.

Like everybody else I tried using an approach based on Bayes' Theorem but swiftly ran into a problem where to be able to rank all entries by their classification (how interesting are they based on your previously expressed preferences?) you need to classify every entry. What's more every time the user expresses an interest we need to re-classify every entry. Unfortunately I'm testing my aggregator with about 853 feeds containing 55,570 entries for the last 2 months. The need to re-classify because I'm showing the relationship between all these entries rather than a binary spam/ham distinction pretty much rules out anything similar to Classifier4J. Pity really as it's a nice little library.

In other news I've rejoined the church of emacs. Even if it's only for looking at logfiles.