OpenSocial working (sort of)!
The first primitive bit of OpenSocial support is now wired into Quoddy. Don't believe me? Fine, here's the first screenshot:
[ Home | Articles | Account | People | Projects | FAQ ] OpenSocial working (sort of)!
The first primitive bit of OpenSocial support is now wired into Quoddy. Don't believe me? Fine, here's the first screenshot:
ScrewPile Update 12-22-2010
Since our last update, a lot of work has gone into Quoddy. The latest code includes support for using LDAP for user account information, has support for creating and listing "friends" and "followers" and has some primitive activity stream stuff in place... and to top it all off, work has begun on adding OpenSocial support using Apache Shindig. The OpenSocial stuff isn't finished by any stretch of the imagination, but I hope to be able to deploy and render at least a "hello, world" Gadget sometime tonight / tomorrow morning.
ScrewPile Update 12-01-2010
Not a lot to report since last time... I (Phil) have been in more of a "research" mode than a "coding" mode lately, trying to get more up to speed on some of the theoretical stuff vis-a-vis Information Retrieval, Network Science and Evolutionary Computation. But expect some progress on Neddick - at least - in the near future.
In the mean-time, take a gander at some of the cool stuff that's going on over at the ASF. A lot of new projects are showing up there, that may be useful for the ScrewPile projects. In particular, there's a lot fo Semantic Web related activity going on, as well as some Content Management stuff. Check out Stanbol, Chemistry, Clerezza, and note that Jena is becoming an Apache project.
ScrewPile update for 07-25-2010
So, what's new since last time? Well, not everything one might have hoped for, but a few Neddick bugs have been fixed at least, and progress continues on. We still hope to have Neddick TPR2 out by the end of August, and maybe a solid start on Heceta.
Specifically regarding Neddick, since last time, the bugs that have been resolved are: Bug #12 - Make "tag" box work when pressing ENTER] - FIXEDBug #8 - Get arrows for up/down vote links - FIXEDBug #25 - Create scheduler mechanism for asynchronous updates - FIXEDBug #35 - Need scheduled job to rebuild entrycache on some periodic basis - FIXED
The scheduled task stuff turned out to be pretty simple, thanks for the excellent Quartz Plugin for Grails. Now the EntryCache gets refreshed once a minute. Additionally, even though it wasn't in the bug list as a discrete bug, a problem with recalculating the score of an entry - after an upvote was removed - has been fixed.
As to what's coming next... mainly work on the next Neddick TPR, so see the Roadmap for details. The bugs on the roadmap page are now sorted (more or less) by the order in which we intend to attack them, so it should be pretty easy to tell where things are going.
And that's a wrap, for this week.
What's new with ScrewPile?
Since the last post, all sorts of stuff has gone on! Things are progressing nicely, especially with Neddick. Since last time we've registered each of the ScrewPile subprojects with sited dedicated to F/OSS, such as Ohloh.net, Advogato.org, and Freshmeat.net. Source code has been pushed to GitHub and the Google Code SVN repository. We've gotten Bugzilla and IceScrum installed and configured, and we have a live Neddick demo site up and running. Additionally, documentation has been written on our development process, and some roadmap documentation has been put together. We've also put together a site at Fogbeam.org dedicated to Fogbeam's open source activities.
Not a bad week's work, eh?
So, for your browsing convenience, here are links to all the new hotness:
Neddick demo site
http://www.fogbeam.org/
http://dev.fogbeam.org/
Bugzilla
IceScrum
Development Process Doc
Neddick Roadmap
Neddick on GitHub
Quoddy on GitHub
Heceta on GitHub
Neddick on Freshmeat
Quoddy on Freshmeat
Heceta on Freshmeat
Neddick on Advogato.org
Quoddy on Advogato.org
Heceta on Advogato.org
Neddick on Ohloh.net
Quoddy on Ohloh.net
Heceta on Ohloh.net
I think that's about it. Whew...
Anyway, what's coming next? Well, all of the stuff from the Neddick roadmap, and we also need to get a proper Ant build setup, some tests written, and Hudson installed and configured to do continuous integration and automated testing.
The Neddick screencast should also be forthcoming shortly, which should be fairly enlightening.
And that's about it. Any questions, just leave a comment here, or join the ScrewPile-dev or ScrewPile-discuss Google Group and post there.
Welcome to ScrewPile
(edit: everything below is true, with the addendum that "ScrewPile" has in turn been renamed to "Fogcutter". The reason is that "ScrewPile" is just a bad name, that looks and sounds ugly unless you're one of the 15 people in the world who knows what a "screwpile" lighthouse is.)
Welcome! Some of you may have known this project under its old name OpenQabal. We've recently decided to rename the project amidst some other dramatic changes, and this new blog is now the home for news and updates about the project.
If you made it here by accident, or don't know exactly what ScrewPile is, let us explain:
"ScrewPile is an open-source suite of APIs, components and applications - driven by the principles of federation, composition, open protocols, and open standards - for building and enabling intelligent enterprise applications for collaboration, social-networking, knowledge management and discovery, organizational learning, Information Retrieval and decision support."
OpenQabal renamed, development moved, etc.
(edit: everything below is true, with the addendum that "ScrewPile" has in turn been renamed to "Fogcutter". The reason is that "ScrewPile" is just a bad name, that looks and sounds ugly unless you're one of the 15 people in the world who knows what a "screwpile" lighthouse is.
Syndicated 2010-07-08 02:29:08 (Updated 2010-07-08 02:44:18) from openqabal
Capability Case #2: Technology Radar
Following on from yesterday's post about Project Shelley Capability Case #1: Collaborative Filtering for Information Retrieval, here is the next detailed capability case for Project Shelley Capability Case #2: Technology Radar. Also available in (pdf) and (odt) format.Name:
Technology Radar
Intent:
Identify technological developments - which may present either a threat to the enterprise, or a groundbreaking new opportunity - as early as possible.
Description:
New technologies are being developed at a dizzying pace. Worldwide, private enterprises, academic researchers, and open-source hackers are all constantly pushing the envelope, developing new approaches and tools. Some of these advancements may represent a huge threat to your organization, perhaps by enabling a competitor to cannibalize your existing business model with a much less expensive alternative. Others may represent an opportunity to break new ground with products, product features, or services that can represent sizable new revenue streams. It is advantageous to identify these advances as soon as possible, in order to out maneuver the competition and take maximum advantage of new developments.
As Downes and Mui point out in their book Unleashing the Killer App, this kind of awareness requires a technology radar consisting of a fat pipeline, a sensitive radar screen and sophisticated intelligence.
Solution Story:
At MegaCorp, developers of the market leading Flozzit product, leaders are constantly jousting with rival HyperCorp, each striving to release the most advanced product in order to steal customers from the other. Recently, HyperCorp has released a multiple new versions with features that no one at MegaCorp had considered, or believed possible at the time. After the most recent release, MegaCorp leaders dug in and discovered that HyperCorp had integrated advanced technology developed by researchers at Miskatonic University. “Why,” asked MegaCorp CEO Howard Phillips, “did we not know about this sooner? This is actually a better fit for our product.. if we had done this first, we could have taken a huge chunk of HyperCorp’s market share, instead of letting them jump further in front of us” No one had an answer.
In order to address this lack of awareness of emerging technologies, MegaCorp decide to implement a Technology Radar. An “emerging technologies” channel is created, where every member of the organization can submit links to documents, articles and news-feeds that touch on technologies related to MegaCorp’s industry. Users throughout the organization vote, tag and comment on each submission, allowing the collective intelligence of the organization to filter the less important items, while pushing the key ones to the top. Product Managers and executives begin to make browsing the latest ‘top items’ on the channel a routine habit... and some users configure the system to send them a dynamic alert via instant messaging when an entry reaches a certain score.
A few months later, the Flozzit Product Manager receives such an instant message... the link is to an article published by Arkham University, announcing the development of a new algorithm which solves a problem that MegaCorp engineers have been struggling with. AU has released the source code under a permissive open-source license, and MegaCorp begin integrating the new approach, and also recruit two of the students from AU who worked on the project.
Using the new technology, MegaCorp are able to deliver a new release of Flozzit with several features which they believe that HyperCorp cannot match. Amazingly, the new HyperCorp release comes out six weekslater, and is almost totally equivalent to the new Flozzit.
CEO Phillips talks to his mangers and explains why he’s happy with developments; “We weren’t able to leap-frog them this time... but if we hadn’t rolled that new stuff out when we did, their new release would have been a dagger into our heart. Now we’ve shown them, and the market, that they aren’t always the ones on the forefront of technical advancements. And the two new guys we hired from Arkham are already hard at work on some stuff that’s going to blow everybody away.”
Vintage:
Mature Commercialization
Challeges:
Corporate culture which fosters a “Not Invented Here” syndrome.
Lack of incentives for participation in the system.
Lack of belief in the utility of the system.
Lack of participation in the system by executives and other decision makers.
Forces:
TBD
Business Results:
Better awareness of technological advances which are significant to the organization. Ability to gain early mover advantage over competitors by incorporating advances
sooner.
Lower risk of being one-upped by the competition with a significant technical
advancement.
Capabilities:
Share links to web-sites, documents and other items of interest
Categorize links by topics using channels
Tag links with specific keyword
Rank items by voting them “up” or “down”
Search and filter by topic, keyword, and/or score
Sort view by various statistical measures, such as “all-time score”, “hotness”, and
“controversiality.”
Dynamic alerting, via email, instant messaging, etc., when items reach certain thresholds.
Typical Use Scenarios and Guidance:
A technology radar is established to pull in information from many disparate sources:
RSS feeds, Twitter streams, email lists, and user submitted links to websites, documents and articles. Collaborative filtering through collective intelligence is used to filter the lower value submissions, while ensuring the relevant information gains visibility.
Employees through the organization view the radar, through the “emerging technologies”
channel and take advantage of the information.
In some cases this may represent a “bottom up” scenario, such as an engineer finding an interesting new library which enables a feature the engineer likes... he quickly knocks out a prototype, shows it to senior management, and it is eventually adopted into a product release. In another case, this may be a “top down” scenario, where a senior leader discovers a new technology, and issues a mandate that R&D investigate it’s applicability to their product.
Applicable Technologies:
Fogbeam Labs “Project Shelley”
Other corporate knowledge repositories (blog servers, forums software, document
management systems, HR management systems, etc.)
Existing Data Warehouses / Databases / Knowledgebases
External information sources (web pages, databases, etc.)
Implementation Effort:
TBD
Integration:
Project Shelley can easily integrate any knowledge source which can be accessed via
HTTP and which exists in a format which can be parsed into text tokens for indexing by
Lucene. Where text extraction is not possible, location through metadata is still possible (ex, mp3 audio files, video, etc).
Integration Mechanism:
RSS feeds, HTTP, OpenSearch
Integration Status:
TBD
Syndicated 2010-06-12 17:02:14 (Updated 2010-06-12 17:03:15) from openqabal
So, what's a "Capability Case" and why should I care?
One of the things that we're doing with Project Shelley (and all of the OpenQabal projects, really) is expressing the initial requirements in terms of capability cases. That link explains capability cases in more detail, but the gist of it is this: A capability case is a business problem, linked to a set of technological capabilities, through a scenario. A capability case could be considered somewhat similar to a use case, but capability cases are more specifically about linking the scenario to a business problem and envisioning a solution, expressed as required capabilities. To illustrate the point, and to get the ball rolling with describing Project Shelley in terms of capability cases, here's our first Project Shelley Capability Case. (pdf), (odt).Name:
Collaborative Filtering for Information Retrieval
Intent:
Use voting/ranking by individual users to tap into the “wisdom of crowds” effect to filter / select the most relevant information in a given context.
Description:
Knowledge workers - and especially executives - face nonstop demands on their time, and have to make key decisions in ever decreasing time spans, in order to adapt to the rapidly changing business environment. Balanced against the need for rapid decision making is the need to consider and evaluate as much available information as possible before making a decision. The information needed to make correct decisions often exists, either within your enterprise - often locked away in the collective, accumulated wisdom of every member of the organization - or somewhere outside your enterprise. In either case, it can be nearly impossible to solicit the correct information before risking a strategic mistake. In any medium to large organization, it simply is not possible to review every document and poll every employee, customer, partner and vendor before executing a decision. Even if time were available to do this, a small nugget of essential information could easily be lost in the sea of noise. However, technological tools make it possible to rapidly filter, rank and correlate various sources of knowledge, helping to ensure that what is important makes it to the people who need it, despite it’s origin.
A group of individuals can often be “smarter” than any one member of the group. By aggregating the wisdom of individuals via voting / ranking / correlation using collective intelligence it is possible to tap into the wisdom of crowds effect within your organization. Collective intelligence ensure that relevant information is seen by those who need to see it, even if it “bubbles up” from an otherwise obscure source.
Solution Story:
At MegaCorp, the worldwide leader in enterprise software with their flagship Flozzit product; sales were down and managers were scrambling to increase revenues. A group of managers decided that the solution was to create a new, feature-enhanced Flozzit 3.0, which would add missing features and solve long-standing issues that were resulting in lost sales.
Lacking sufficient resources to add every desirable feature, it was critical that the Flozzit Product Manager identify the features which would most directly impact sales. So the Product Manager began scouring over enhancement requests in the bug database, and scanning old emails from account managers, field reps and engineers. After a few weeks work and several meetings, the PM thought she had a pretty good handle on which features should go into Flozzit 3.0.
Before committing resources to the new roadmap however, she decides to peruse the “Flozzit” channel on the PS portal, and look for items tagged “complaint” or “enhancement.” At the top of the list is a report written by a customer support representative (who the PM had never met, or even heard of; she wasn’t even sure if he was still with the company) titled “Why BigCorp hates Flozzit.” Intrigued, she examines the filtering metadata and sees that nearly every CSR in the company has upvoted the report, as well as one or two of the engineers. She downloads the report and digs in, to find a detailed summary of the top issues that end-users at BigCorp (the largest customer of Flozzit!) had complained about when talking to the CSRs. The language was detailed and some of it was not kind to Flozzit. After reading the report, the PM arranges to meet with the CSR who wrote the report, and identifies 5 top issues which had never been discussed in the many meetings held to identify the new Flozzit 3.0 roadmap. She then calls her top contact at BigCorp to discuss the issues and the first thing he says about issue #1 is “Yes, our users have been very concerned about that. We noticed that HyperCorp is releasing that feature in their 4.0 product and might consider switching if MegaCorp doesn’t answer soon.”
Armed with this new information, the PM polls a sample of other Flozzit customers about the 5 issues identified and find that 3 of them are so important that they must go into Flozzit 3.0.
Six months later Flozzit 3.0 ships with the 3 new features and a slew of bug fixes. BigCorp immediately commits to an upgrade, and are so happy with the new version that they purchase another 50 licenses a few months later.
Vintage:
Mature Commercialization
Challenges:
Corporate culture which stifles dissent.
Lack of incentives for participation.
Lack of belief in the utility of the system.
Unsupported document formats, databases with proprietary formats which are difficult to
integrate.
Forces:
TBD
Business Results:
Better identification of actionable news and information which might otherwise remain
lost in the sea of information inside the enterprise; leading to better decision making at
both the strategic and tactical levels.
Capabilities:
Share links to web-sites, documents and other items of interest
Categorize links by topics using channels
Tag links with specific keyword
Rank items by voting them “up” or “down”
Search and filter by topic, keyword, and/or score
Sort view by various statistical measures, such as “all-time score”, “hotness”, and
“controversiality.”
Typical Use Scenarios and Guidance:
Knowledge workers view channels of topical concern to their jobs, or of general interest, on a regular basis, voting and tagging existing items, commenting on existing items, and submitting new items, creating a view of what’s important - and adding to the corporate memory - using collective intelligence.
Knowledge workers discover relevant information through casual browsing; and through directed searching by tag, channel, submitter, score, or other attribute, when specific topics are under review. By limiting causal browsing to the most highly ranked items, an employee can maintain a “finger on the pulse” of what is considered important at a point in time, wthout reviewing every item. But directed search makes all of the other items accessible when they are relevant to a topical query.
Applicable Technologies:
Fogbeam Labs “Project Shelley”
Other corporate knowledge repositories (blog servers, forums software, document
management systems, HR management systems, etc.)
Existing Data Warehouses / Databases / Knowledgebases
External information sources (web pages, databases, etc.)
Implementation Effort:
TBD
Integration:
Project Shelley can easily integrate any knowledge source which can be accessed via
HTTP and which exists in a format which can be parsed into text tokens for indexing by
Lucene. Where text extraction is not possible, location through metadata is still possible (ex, mp3 audio files, video, etc).
Integration Mechanism:
RSS feeds, HTTP, OpenSearch
Integration Status:
TBD
Syndicated 2010-06-12 02:03:44 (Updated 2010-06-12 03:40:08) from openqabal
OpenQabal, Project Shelley, Project Poe, Project Collins, WTF?
If you were paying attention to the previous post and its predecessor you should kinda have an idea of what Project Shelley is. At least, you may be thinking to yourself, "Ok, it's a Reddit clone, written in Groovy/Grails, and incomplete, what's the big deal?" And that's a perfectly fine attitude to have at the moment. The point of this post is to start clarifying where this is going, how Project Shelley relates to Project Poe (namesake) and Project Collins (namesake) and how it all fits into the OpenQabal vision. Let's start by looking at what OpenQabal is meant to be. In the past, I had described OQ as: ...an open-source social-networking and collaboration platform / suite driven by the principles of federation, composition, and openness; with a special emphasis on enabling "distributed conversations" and the "federated social graph." So, let's start by throwing that paragraph out, while keeping some of the essence of it. This is still - in part - about social-networking and collaboration, but "distributed conversations" (especially at "Internet scope") aren't really part of what I'm interested in tackling right now. Not because it's not interesting, or because it's a solved problem, but because I don't see as much value for it in the setting(s) that I'm focusing on. But, again, if members of the community see a place to do that kind of work within the OQ umbrella, then that would be great by me. Now, the "federated social graph" bit... I wouldn't say that's being thrown out, but rather placed on the "back burner." So, that leaves us with, what, "an open-source social-networking and collaboration platform / suite driven by the principles of federation, composition, and openness?" Well, yes. But to make the scope of intended application(s) clearer, and the way things will decompose, I'd re-word that now as something like: "OpenQabal is an open-source suite of APIs and applications - driven by the principles of federation, composition, open protocols, and open standards - for building and enabling intelligent enterprise applications for collaboration, social-networking, knowledge management and discovery, organizational learning, Information Retrieval and decision support." No, that definition isn't perfect, and it'll morph over time. But I think that gets closer to the heart of things. "Intelligent enterprise applications" is really what interests me right now. And what I mean by that is, using technologies like collaborative filtering, tagging, social-graph mining, explicit semantics, data mining, machine learning, etc. to build (or integrate) enterprise applications in a way that makes them "smarter" or better able to help humans find the information they need, when they need it (even if they don't know they need it yet!) I've said before that I don't really like the label Enterprise 2.0, but for lack of a better term, you could say that that's what this is. Except we should probably call it Enterprise 3.0 just to one-up the competition, eh? With that said, what OpenQabal becomes is sort of an "umbrella" or over-arching structure for sub-projects that handle different parts of this vision. Think of how Glassfish has become an "umbrella" project for a series of related projects: OpenMQ, OpenESB, SailFin, Portal Server, etc. With OQ as the overall structure, sub-projects will deal with provide various APIs and/or applications / subsystems that are part of this overall "intelligent application" vision. A number of the pieces we talked about in the old OpenQabal model will live on now, pretty much as they always would have. I still see the need for an API / system for managing tags, something for doing recommendations, something for managing a social graph and allowing queries against it, etc. Of course, as before, in a lot of areas existing open-source code exists to do these things. In that case, we may (within the letter and spirit of the respective licenses) just "borrow" existing code, possibly wrapping or modifying it to fit the model of what we're doing here. Other bits will have to be written entirely from scratch, and that's OK. So, what about these sub-projects? Well, here's what I think I know about them so far:Syndicated 2010-06-11 01:34:58 (Updated 2010-06-11 02:58:16) from openqabal
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!