11 Jul 2010 mindcrime   » (Master)

Welcome to ScrewPile

(edit: everything below is true, with the addendum that "ScrewPile" has in turn been renamed to "Fogcutter". The reason is that "ScrewPile" is just a bad name, that looks and sounds ugly unless you're one of the 15 people in the world who knows what a "screwpile" lighthouse is.)


Welcome! Some of you may have known this project under its old name OpenQabal. We've recently decided to rename the project amidst some other dramatic changes, and this new blog is now the home for news and updates about the project.

If you made it here by accident, or don't know exactly what ScrewPile is, let us explain:

What is it?



ScrewPile is an umbrella project, sponsored by Fogbeam Labs, which is building a suite of powerful, open-source tools for knowledge management and collaboration. ScrewPile was formerly known as OpenQabal but has been renamed, as part of an effort to develop a consistent naming pattern for related projects.

Why "ScrewPile?"



A Screwpile is a particular type of lighthouse. Fogbeam'ers like lighthouses, so we're using lighthouse related terms for all of our open source activities.

Tell Me More



The 65,000 foot overview description would go something like this:


"ScrewPile is an open-source suite of APIs, components and applications - driven by the principles of federation, composition, open protocols, and open standards - for building and enabling intelligent enterprise applications for collaboration, social-networking, knowledge management and discovery, organizational learning, Information Retrieval and decision support."


ScrewPile proper, then, is an "umbrella project" or over-arching structure for sub-projects that handle different parts of this vision. Think of how Glassfish has become an "umbrella" project for a series of related projects: OpenMQ, OpenESB, SailFin, Portal Server, etc.

With ScrewPile as the overall encompassing framework, sub-projects will deal with provide various APIs and/or applications / subsystems that are part of this overall "intelligent application" vision.

Subprojects will deal with things like: an API / system for managing tags, recommendations engine, mechanism for dealing with voting/ranking things, something for managing a social graph and allowing queries against it, etc.

Of course, as before, in a lot of areas existing open-source code exists to do these things. In that case, we may (within the letter and spirit of the respective licenses) just "borrow" existing code, possibly wrapping or modifying it to fit the model of what we're doing here. Other bits will have to be written entirely from scratch, and that's OK.

Tell Me About These Subprojects



Well, here's what we think we know about them so far:


  • Neddick - An interface that builds on the APIs for tagging, ranking and recommending items, to provide a platform for sharing and discovering useful links, documents, people, etc. What we have right now is pretty simplistic, but there's a LOT of room for growth in this.


  • Quoddy - a sort of "mini Facebook" like social-networking interface. Builds on the APIs for social-graph management, activity-stream, activity profiling, tagging, etc. Provides the front-end for managing connections and for letting users provide information about themselves, their interests, etc. But unlike Facebook, no silly Pirates vs. Ninjas or Farmville stuff.

  • Heceta - This may not actually come into existence as a standalone project; but the general vision is a search engine that leverages all of the various bits of information from Neddick, Quoddy, and "TBD" to provide better / deeper / more insightful search results than you can get from simple document content analysis. Intranet search in the Enterprise usually sucks, largely because page-rank type algorithms don't work well due to the lack of links between documents. But by supplementing the content analysis with scoring based on tags, social graph connections, activity-stream information, etc., it should be possible to do a much better job. This is not, by the way, a totally novel idea. It's something people are referring to as Social Search.

  • GraphEngine - as before, an API for storing and managing the "social graph stuff." Doing this on a large scale is still a problem, and I'm intrigued by the idea of using an incremental evaluation system approach for this, but haven't done much on this yet.

  • ProfileEngine - name says it all, really.

  • RecommenderEngine

  • ActivityEngine

  • TagEngine

  • Etc.



Of course since this is all intended for an enterprise setting, a big focus will be on integrating with other systems (see the point about "open protocols" and "open standards" above).

Previous discussion of Neddick (formerly code-named "Project Shelley") mentions discovering documents and people, despite the screen shots only showing stuff about links so far. That's where integration comes into play... part of the vision is to integrate with, for example, a document management system like Alfresco, a forum system like JForum, a groupware / calendar system, a HRM system like OrangeHRM and/or a social-networking application like Quoddy (formerly code-named "Project Poe").

That, of course, also plays into the vision for Heceta, which is all about searching across all the different domains, and using the knowledge aggregated across all of them, to enhance our search capability.

Get involved by visiting the project page on Google Code.

Syndicated 2010-07-08 03:01:00 from ScrewPile Dev

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!