Against the Grain: Getting Projects To Work Together

by Gregor J. Rothfuss and Paul Everitt

Last year we wrote a piece here questioning whether open source projects were motivated to work with each other. This article was in the context of our work with OSCOM and its attempts to bring together open source content management projects and developers.

The article stimulated some interesting questions. Now that we've had some time to take some steps and try some ideas, we thought it appropriate to report back.

In summary, OSCOM interop has caught on more than expected, but the hard work remains ahead. Also, other open source interop efforts such as freedesktop.org and Linux Standards Base have shown momentum despite the odds. We have found that open source projects collaborate much less than the open access to source code would suggest.

Why does open source fall into the Not Invented Here syndrome so often?

As before, the number one barrier to open source interop is cultural. Does open source make closed minds? How do tribes form a nation? Tackling this issue isn't in the domain of software engineering, rather, social engineering.

A Recap of the Previous Discussion

The Interop in the Bazaar article described three disincentives to interop: unfunded scope creep, tribalism as a cultural issue, and the boring challenge of writing standards documents. We then listed interop's beneficiaries and upsides before making some predictions.

Let's review the accuracy of our guesses:

Prediction One: Interop between open source projects is a fool's errand.

100% true, as we're fools, and we're still at it. :^)

Prediction Two: If we stay practical and focus on small steps, we can provide value with lower risk.

Prediction One notwithstanding, we're finding within OSCOM that practical, bite-sized interop is interesting to implementors. Bottom line: If you can whip up a prototype in a day, and brag about it over beers that same evening, then chances for success go up considerably.

Prediction Three: We'll stumble across the Big Idea that is the bait to get the fish (project leaders) on the hook in a big way.

Below we discuss how we used an OSCOM sprint in Zurich to jointly develop an app that gives incentive to implement WebDAV.

Prediction Four: Somebody will get sued over a patent infringement and we'll all move to Norway.

No news yet from Interwoven. :^)

As noted previously, the first step (a will to work together) is the hardest. This is discussed below. Suffice it to say, open source does not always equate to open minds. Thus, we have a real accomplishment here.

OSCOM 2 in Berkeley proved that OSCOM was still interesting. The conference proved smaller than Zurich, but more intense. We found a deeper connection to our users and learned the most relevant directions to pursue.

From the press coverage, we also found that OSCOM is, in many ways, a better story than our individual projects. Journalists are bombarded by story pitches from the teeming masses of CMS vendors and projects. Everything sounds the same. However, OSCOM -- with its pitch of cooperating to better serve customers -- is a fresh story. Fresh means that journalists will write stories, and thus we can attract the desperate sustenance of attention.

For OSCOM 2, we launched a project called SlideML to encode and produce presentations. SlideML was a good, appropriately-sized first step, and we learned from it. Primarily, projects only work when someone cares enough to work on it, even if nobody else does. In this case we mean Roger Fischer from BitFlux. These champions become the glue that promotes and accumulates sporadic efforts from contributors.

OSCOM also tackles marketing issues, and one glaring need was an authoritative, independent grid of CMS projects. This is the CMSML project at OSCOM. It was more ambitious, as it was more about marketing than software development. CMSML has also exceeded expectations, as other similar efforts were interested to collaborate.

CMSML marks an effort where we crossed another cultural barrier, that between Free / Open Source and proprietary software. We're still alive to talk about it. One roadblock that we found was that many projects do not want to be compared head-to-head with their proprietary competitors (and vice versa). Lesson: Don't be afraid of the proprietary world, and seek collaboration for open standards with it wherever possible.

We also hosted a developer "sprint" in Zurich. This was the first attempt at a close-hand development effort between different projects. We were nervous that the sprint would devolve into two days of mail-reading. Instead, it greatly exceeded expectations.

Twingle, an OSCOM project to build a common CMS authoring tool, was a more ambitious attempt to pool energy to meet a common need. Though very young, Twingle sparked interest and was quickly able to spur small steps towards interoperability. Specifically, it helped incent servers to move WebDAV support up on the priority list. Since Twingle uses Mozilla's RDF engine, it brought CMS attention to the Semantic Web. Twingle reinforced the lesson that quick, tangible efforts are ideal for bringing unity.

Culture Wars

As mentioned in the lead-in, the number one challenge open source interop faces is cultural biases within communities. Why work with the other guy? Tribal forces are particularly strong in open source.

Tackling this first requires an understanding of the open source motivation. Open source projects do not have much besides their social fabric. All "payment" is in the form of reputation within that small subculture. Thus the subculture is vigorously defended against intruders, and cross-culture efforts like OSCOM need to build their own culture and reputation systems to compete.

The memetic explanation for these effects is that autoimmune responses kick in to defend the culture of a project against threats from the outside.

Stated differently, open source interop needs to seed the interop effort with the same upside as traditional open source projects. Without a strong community that rewards interop, project collaborations are doomed to fail.

Lessons Learned

• Developers care more than expected about working together.
• Working together bears fruit when it is perceived as fun, such as a sprint.
• We all have a lot to learn from each other.
• We're too busy to launch ambitious standards agendas, and that's not our strength anyway. Focus on practical standards that overlap our common agenda.
• The shared agenda is good for customers, in a way that proprietary vendors can't match without facing a shareholder lawsuit. (For related material, see Permanence, Replacability, and Inventor's Disease.)
• The real opportunity with OSCOM is that these are the people actually making the systems. If something is agreed to , and they have the time, then it's done. And when it's done, it becomes part of several large communities, reducing the time needed to hit the tipping point.
• Journalists have told us that the common story (OSCOM interop) will sell better than the individual stories (Apache Lenya, Midgard, Zope, etc.). It's a crowded market and we need to attract attention. In this case, "attention together" is better than "ignored alone".

OSCOM is a place where CMS implementors hang out. Sure, we have an agenda: make the mainstream CMS market aware of the open source advantage, foster inter-project interop, share the cost of common needs, etc.

But the first and hardest task is in the bag. We have created a community. We do things together, we know each other, we look forward to seeing each other. Inside OSCOM, we really set aside our tribes and we rarely jockey for position by promoting our narrow interest. "Me" yields to "we".

What's Next

OSCOM 3 is focused on "Semantic Web meets CMS". This brings OSCOM into an important, albeit contentious arena. Also, we're changing the conference format to be more about participating than listening.

Twingle, if it reaches its goal of improving the user experience, can be a reward for work spent on interoperability.

Conclusion

The first article posed the question, "Do open source developers want interop?" It remains an open question. Still, OSCOM is making a bet that the answer, when the right circumstances are created, can be yes.

This second article shows OSCOM at its first steps in the journey. We've created a small community and culture of CMS implementors. We all know each other now, we enjoy each other's company, and we expend real effort towards mutual success.

With this base, we are tackling practical interop between open source CMS projects. We are also tackling common marketing initiatives.

However, the question remains. To leave a mark and make a real difference to projects and customers, we must deliver results to the question, "Do open source developers care about working together?"