Data as a Narrative

I tried surfing to www.scopeware.com to find out what Gelertner was getting at, and I got the above error message.

Does anyone have any other leads?

Just when the NYTimes alert came out, the site was just hanging, sending the ACK, but not responding, hanging forever. Then it was a closed port, then hang, then a closed port, then hanging, and then I did the search to netcraft and, learning what he uses to run his site, understood. He slashdot'ed himself and fried his server from the traffic off a lowly NYTimes tech back-page report.

Either that or it's just bad luck. It's happened to me enough times: You make a big announcement, it finally gets in print, and moments later your ISP calls up to say they are doing "routine maintenance and ..."

It's curious, but not really when it's not opensource/free software, that as you observe, without the proprietary source site online, Google hasn't a clue what this software is about. When will they ever learn?

I was one of the authors for the software of the New Internet Computer web appliance (I'm no longer associated with it). Originally we only supported things you could view in your browser. Later, we added support for things like DAV and USB zip so that folks could move their mp3 and document collections around.

When designing the interface for this, it occurred to us -- as it did garym -- that the desktop metaphor sucks. You want to see 'pictures' and 'music', not a random set of directories. People are very task oriented.

We ran into the same problem everyone else has run into: no one has any idea what would be better. Think about it. The desktop metaphor still applies in 2002 as much as it did in 1970. People organize their stuff in a hierarchical manor. Take a person who never worked at an office, you will still find they categorize. No, they don't put all "Scott" products in one place, but they sure as hell put all toilet paper in one place. It's the way we organize.

The problem is not organization. The problem is that we don't want to access our information in a hierarchical manor. We want something task oriented. Apple's iPhoto is a good example of this. The photos are stored somewhere on disk, but you really don't care where. The iPhoto task lets you do what you want.

So "non-desktop" works for a specific task. You can use iTunes for your music, and there are similar applications for Linux and Windows. The problem is the generic case. The files and folders are a generic metaphor, because while iPhoto knows what a JPEG is, it sure as hell can't figure out a tar archive. There are multiple problems in the generic case. There are three things absolutely required of storage:

• Things are stored (duh). This means they live somewhere.
• Things are findable in a predictable and programmable fashion. Sure, you can happily view your photos in iPhoto without caring where they actually live on disk, but iPhoto has to code to a location. Pre OS X, MacOS used resources to allow wholesale movement of files without breaking this property. Neat, but not essential. What I'm trying to say is that behind any task-oriented (or "narrative") interface there has to be an understood organization outside of that specific narrative.
• People need to be able to view and modify this generic, predictable structure. Back to our example, maybe iPhoto is a better way to access your photos, but a sysadmin still can have a need to access the on-disk photos and move them about if she needs to. You can't manage /usr and /etc and all sorts of random things you need to manage through "narrative" alone. Even though there is a lot that Linux could do to make it better, big systems eventually require management at a level that GUIs don't provide.

The real challenge is coming up with a generic "narrative" that satisfies these requirements while allowing folks to organize things as easily as "folders" allow and access things as easily as "tasks" or "narratives" can. The person who solves this will find a path beaten to her door.

User Interface is hard, let's go shopping.

This is a very complex question that I, too, have been thinking about. This article and the replies contain a very good thought and good reasoning, including the conclusion that this is a very difficult issue.

I personally have thought about going the way of agents to circumvent the problem. It looks like a promising idea. Something like that does exist in (as mentioned) the web and its search engines and also in the digital librarian found in, IIRC, NeXT and perhaps MacOS X.

Another issue that I think this could solve is need for metadata. Some filesystems support that, yes. That's another issue to think about but I still think that it could be kept in a database and you'd ideally never go tamper with the files without using an agent that also manipulates and uses that database. Moreover, different people (users) could have their own databases and metadata that refers to the same files.

As an example of useful metadata, I'd like to mention pictures. You can't rely on the comment field and can't cram data on the filename. A picture may be worth a thousand words and they often contain several concepts. You can't just categorise by one attribute and be happy. Often you want to find files with a certain other quality. If you could go and input some keywords that describe files, which is a fairly trivial idea considering the librarian agent with the database etc, you could for example search for similiar files, duplicates and related files to keep grouped, every photo I have of Joe or every picture I've saved that has a button in it somewhere. Similiarly you could request lyrics to a song that is playing and perhaps photos related to it, dig up something mentioned in the lyrics in a dictionary etc.

Yet another issue would be storage abstraction of a sort. The agent could keep track of your disk or volume space(s) and recommend some data to be, for example, archived on CD because it's rarely accessed. It could manage backups and perhaps versioning. There's so much that could probably be done without even touching the files. There's even read-only media where you can't touch the files!

(Hm.. this reply could almost be an article and I tried to avoid saying anything in length :) Better stop now :)

Some see also's:

• The book Snow Crash by Neal Stephenson
• BeOS
• XFS, file attributes

From the front page:

Due to overwhelming interest in Scopeware Vision[tm], our web site is experiencing significant delays. To help alleviate the problem, we have temporarily replaced our regular site with these text-only pages. We appreciate your enthusiasm and thank you for your patience.

The real challenge is coming up with a generic "narrative" that satisfies these requirements while allowing folks to organize things as easily as "folders" allow and access things as easily as "tasks" or "narratives" can. The person who solves this will find a path beaten to her door.

As long as we think of a directory tree as a two-dimensional branching of lines, we'll be limited to categorical organization structures. Or the other way around; categorical thinking produces very simple two-dimensional trees: folders, files, etcs.

There is another way to organize information, more easily stored by _restructurable meme_ instead of by _hardcoded mime_, say, the way I know my own mind works:

Imagine a computer screen which appears to be a sort of deep bowl, full of three-dimensional objects, rather than a table full of flat objects. If the bowl is deep, maybe even deep enough to appear to be a "tunnel" and if its edges are sticky, we can attach objects into the edges of the bowl, and there they will stay. This way we can organize them in a manner that allows us to pluck three items from three different places on the horizonal axis because they're grouped in a vertical axis.

I mean, we currently have access to either horizontal information OR vertical information. In 3-D, we have access to both at the same time. Yet 3-D can be emulated on a 2-D screen, as any Quake player knows.

The mouse (or a glove?) would have to have an easy way of moving _into_ the screen contents, not just _across_ the way it does now. Probably the final product would look more like staring into the inside of a tree, from trunk upward with a large trunk, branching outward and downward into branches which also branch.

Does this make sense? It's only the nub of an idea. But as narratives go, you can't hardly do better than a tree. A real tree.

On the Palm, the user sees no files (most of the time, some applications show you "files", but they have to go out of their way), they only see applications, and data in them.

You go in your addressbook, and you see all of your contacts. You then go in your calendaring application and set up a meeting with these contacts. One of the things that make this possible is how PalmOS is database-centric, but that could be emulated to some level with a filesystem, although some kind of meta-information is needed. People often go and think about new filesystems (like the one in BeOS) or whatever, but you know, a few dot-files strewn about *could* do the job in many cases.

The point is that when you go in Gimp, you'd see only pictures, sorted in a way that makes sense for pictures, and hopefully other picture-related applications would share the same "sorting". Media players should present the available music and videos in an appropriate way (think a media library, sorted by artist and album, or other views). All of this could still be stored in a plain old filesystem, it's just presentation, like iPhoto and iTunes.

As someone else said, the trick is the generic case.

Replacing 2d tree with a 3d tree gives you one axis, one attribute, more. It requires a big change since it's the fs layout and it doesn't give much.

One thing that gets to me with computing is the failure to consider non-computing solutions. We have tons of data, and we need a way to sort it. Well, let's consider something that has a big dataset that people deal with every day. Libraries. The Library of Congress has somewhere around 20 million titles, as I recall. And people can find stuff in there all the time without much difficulty. Why? Well, there is a two-pronged approach. First, data is organized systematically, broken down by categories. Which is a straight taxonomy. Then, there are 2 interfaces built on top of that organization. One is a card catalog (either the old days with the actual cards, or a computer application) where sufficiently knowledgeable people essentially learn the language of organization, so they can find what they need. The other interface is much more useful, even for knowledgeable folks: the librarians. They are very well-educated and trained individuals that think a lot about ease of access to data. So they can think of things that are related to what you want that you didn't think of yet. And they have the benefits of experience with helping people with similar sorts of things, so they may have even more input for you. So through the librarian "interface" I end up with getting what I didn't even know I needed.

So how do we map this to computing? Either real people to whom you can ask questions, or intelligent agents. Anything else I think will ultimately fail in providing the necessary filter on ever-expanding amounts of data. Direct-access techniques, like going straight to where you know things are stored, is still something that we'll want, but for new areas, we should want someone or something to help us get the information that we didn't even necessarily know we wanted.

I know it's seductive, but the idea that our computers should know what we want better than we do has only ever led to obnoxious designs like the infamous Office paperclip, or the even worse moving menu items introduced in newer versions of Office.

There is a good reason for this: The general problem is "AI-complete". You can't really solve it without making computers intelligent in the same way that humans are.

Rather than trying to make computers second-guess me and figure out what I really want, I'd like to make it easier for me to tell them what I want--because I know. Simplicity in user interfaces should arise from simplicity in the underlying ideas of the code.

I can see utility in a librarian-like app that attempts to find things you didn't know you want, and there exists a good proof of concept. But I wouldn't want my computer to act in that mode all the time.

Gimp has been mentioned, and it's one of my best examples of where the pidgeon holes of data-by-mimetype fails: I have no interest in pictures, I only have an interest in my use of pictures. When I create an image, it is part of something, but my computer says it is only part of the Gimp; anything I want to do where a photo is required forces me to first use Nautilus to find the image, then use Gimp to massage the image, and then, after Gimp has forced me to decide on a hierarchical filename (of which the default is almost never where I want the product to go) I must return to my train of thought to actually use it. I'm forced to think in the filesystem terms. As William Burroughs noted (in that famous Nike commercial) "Technology should serve the body, not enslave the mind" Gimp enslaves my mind: I have no alternative but to think of O/S filesystem organization just to take an image from an email and put a reduced and framed version in my weblog.

What would be better? If the token of the object was passed between applications. For this email-to-weblog example, the attachment from the email is selected "for blogging", probably by the tactile drag-n-drop over into my blog-object, the place where I am writing the blog entry. Under the hood, that creates the temp file passing the ID of it to the blog tool, and in there a right click (like in Nautilus) offers to edit the image -- and this same right click would also apply inside the email application in case I just wanted to correct the image but leave it in my email folder. In both cases, the end-location of the filename might even be different (the blog needs to be eventually moved to the website) but the concept, the use case of the data is the same: If I see an image, I might want to change it, and the facility to change that should not be compartmentalized into vendor-branded boxes, it should be transparent and consistent across all applications.

Thus the agent, seeing that this time I am correcting an image in my email folder or this time I am moving the image to the print queue or this time I am moving it into my weblog, each time the agent decides on whether it is to work on a copy or work on the original and where to store the result.

Drag and drop everywhere would certainly be nice --- it would be great if we could have as much drag&drop-ability as there is across Windows applications. This requires quite a bit of infrastructure, though. Right now you can drag text around and have it mostly work, but it will definitely not work for images. Most applications resort to advertising a filename, not the image data itself, as the drag data.

I intend to write a GNOME Enhancement Proposal with respect to drag and drop; hopefully it will be done this week.

The way to intelligible organization is through a computational modeling of ontology, ie a system of categories, relations, instances. There is a lot of research on this but so far nothing quite usable.

If you could map a working ontology system to UI, you could throw away the desktop interface. Let the computer manage your files.

An unsuccessful example to such an information system is Microsoft's terrible Outlook application. More successful is I think Palm OS. Less successful is "Semantic Web".

However, those tools (PIM, web agents) are too application specific and their language and scope will fail elsewhere.

Solution?

It is AI-complete, unfortunately. :) But we can still do a lot with weak AI.

My guess is that is going to be the killer application of the next decade.

I can't share more, sorry :)

A context object may help in keeping track where the user and its files are. It will probably require a huge support structure to get it going, though.

OLE and Java may be a good starting point, then move to another direction, maybe go to Roster and Translator type of support.

Thing is, a good File System [1, 2, 3] must also be designed just for this requirement. Putting attributes at the file level might help.

Folders/files/tree structure = FS - is a tool

Images in you favorite Image-editing software - is a solution

You could use tool to create (almost)any solution you need.
The idea you are tring to push is - "the only solution is better than a tool" !!!! Nonsence!

Tool have nothing to do with solution!!! It could have sence only to calculate _does this tool allows to create efficient solution.
Does current FS allows you to find all green images(forest/tree/grass) on your coumputer in less then 10 seconds?

If you like MS example - there is "recent documents", "my pictures", "Menu->File->last used files", there is Shortcuts

You like the example of Library - ok. There is card for every book. There is book name and strange code "ABC234" which is needed to find this book. You need not to know what it is! That is FileSystem. But the card is only an interface to it.

OS/2 has a marvelous device called extended-attributes. While NT also has this, OS/2 actually used them. They weren't just part of the file or the filesystem, but of the object that the file may contain part of the representation for.

I would recommend that everyone give BeOS an honest three month trial. When I last used it, I had four disks between 2 and 6 gigs. When I would download large files, I would put them wherever I happened to have room. Then, using BeOS's stored-searches feature, I would leave an icon on my desktop that opened a search for Type == "audio/mpeg" - PRESTO, I would have a window with all my MP3's. I could also keep a folder with queries like Album == "Violator" && Artist == "Depeche Mode"

Images were similar; I can markup with attributes what the images contain, and search for them later. Searches _literally_ took less than a second to complete.

Under BeOS, attributes were also used for marking-up ritch-text files- so my source code can have colors and font specifications where I tell it to... and not with some strange comments, either.

This has been on my mind for a while. Linux _really_ needs attributes and indexing. Attributes don't need filesystem intervention- a system like Gconf might be good enough. But indexing- that's what separates a desktop operating system (BeOS) from, well, everything else.

Indexing could be accomplished with FAM and all-user-mode applications, but unless

(Replying to mrsbrisby)

There is one great program named locate neglected by mostly everyone - Yes, I usually also prefer working with the real, current filesystem than with a snapshot of the directory tree taken last night, but locate is very fast and convenient. Yes, it works only for filenames, but if you are interested, it should not be too hard to make a modified version that would allow you to search, for example, for specific MIME types. Yes, still not as powerful as BeOS's excellent tools, but still very useful...

indexing the name of the file is not terribly useful to me. indexing mime types, while more useful, still brings us into the same world we're already in. having attributes, and the ability to index them is all that will save us.

perhaps i should have said that "Linux _really_ needs attributes and indexing of those attributes"

anyway, indexing mime types is only marginally useful. I give the Album/Artist example as a fantastic reason to need EA. indexing them, once we have them, is actually the easier of the two-parts. we already have the ability to "discover" when a new file is being created, edited, etc, but we don't have anyplace good to store attribute data.

a project like gnome or KDE has to start it- the linux kernel people won't (actually, that's a lie: they did start it... it's just so immature that it's as close to worthless as possible). do what OS/2 does on FAT partitions- create a database and let user-programs maintain it.

peer credentials, and a long-running unprivelged program can be useful enough.... but i'm not _involved_ in any of these desktop projects (i probably shouldn't be either, but that's a different story). i'll be happy to write an interface if I think someone is going to use it- I just don't want to go around all these existing -even "open source" programs and start hacking their code to use my attributes.

you reminded me of something else; OS/2 had the better-symlink; SHADOW's were very much like symlinks, except if the original was removed, the link would disappear as well... this too could be implemented with FAM and the same indexing tool.

I think it's better to let the "user" select the attributes than to have a program figure them out.... actually, I think it's good to have a program to do it, but it's necessary to let the "user" do it. if figuring them out was all that was necessary, _i'd_ write it in a heartbeat... fortunately for my busy day, that's not enough :)

Indexing of random file attributes is a Very Good Thing<tm>, but it comes at a high price--I read Dominic Giampaolo's book on the BeOS file system, and it quickly became apparent that the BeFS paid a heavy performance price for those nifty attributes. There was an option to disable indexing; it doubled performance on a variety of benchmarks.

ReiserFS is ultimately likely to improve on the BeFS, but it would require users to back up and reformat their ext2/3 drives before using any application which depended on attributes. (*sigh* Linux has an installed base. I know it's what I wanted, but it's darn inconvenient.)

IIRC, there's also some grik down in Gnome and/or Nautilus which tries to maintain attributes in an independent database. Gnome developers could try to enhance this database and add queries to their applications, in hope of someday running atop ReiserFS. If that's what users want.

first off; practical file system design is a great book. but the reason indexing slowed down the Be-filesystem is not because indexing has such a high cost. indexing hurt BeFS because it increases the number of writes significantly. many of the steps taken can be reduced easily (something we can go into elsewhere if you like).

I don't think I like the idea of the filesystem handling attributes exclusively. A user-program should be able to handle attributes using on-disk databases in a filesystem neutral way. While EXT3 does support attributes (theoretically), FAT certainly does not, and ISO9660 doesn't either. If you think that's esoteric and/or moot, NFS doesn't support attributes last I checked, and I'm fairly certain AFS doesn't either.

Perhaps the best way to go about it is a set of syscalls that send attribute get/set/etc requests down to the vfs layer, then send a message via klog (or something) to a user-mode program that can handle attributes on foreign filesystems if necessary, and indexing, AS WELL AS SENDING IT to the filesystem itself [if supported].

I'm glad GNOME/NAUTILUS folk are waging this war out... It's an important one to be resolved. I think it's better to let the desktop define the API and the system first. Profiling will teach us exactly what is slow and what is fast, and I think we can work from there...

There is only one thing the computer really has to be dealing with here, and that is to know where stuff normally is.

Say I click on openorifice. Well it knows where I've saved documents recently so its actually got a pretty good idea what directories to poke around in by default when showing my documents. Now if it exported that to nautilus as "My documents" thats useful

If my music player keeps tabs on where I play music from it would easily be able to tell nautilus where most of my music lives.

We don't do enough with the desktop metaphor to judge its sucess anyway

Why can't I

Drag a document to a printer icon Click on the printer to see the queue Use "copy" to make more copies of a printed document Drag a document to a users intray and have it pop up on their desk ? See the users as part of my natural file heirarchy without having to know about /home ? Why isnt my email a folder. Whats this weird "mail application" thing ? Why can't I do ximian vfolder like things on any folder on my desktop ?

Not so easy... There is lot's of small but bad issues

Say I click on openorifice. Well it knows...
But what if you get files on CD or floppy? Or in mail? - DnD? Directories?

Drag a document to a printer icon...
But how aboutimages? How to print is? Need it scale to fill page or print with actual size - so you need some config window.. It is not good.

and have it pop up on their desk...
I have seen a face of sysadm, who discovers, that user just DnD image to new mail and sent it. But it was uncompressed BMP - ~50MB :)

Why isnt my email a folder
M$ Outlook is a collection of folders, but I whould not say it is good.

We don't do enough with the desktop metaphor to judge its sucess anyway
I'll agree with this :)