An Introduction to the Virtual Object System

Posted 6 May 2002 at 03:15 UTC by tetron Share This

The Virtual Object System (VOS) is a hierarchical distributed object system, a desert topping, and a floor wax. It is an infrastructure for object-oriented network communication, and building flexible, distributed object networks. It also tastes good and will keep your floors shiny. In this article I discuss the core library and an example application under development that demonstrates some of the of power of using VOS. (A bit of shameless self-promotion for my free software project.)

VOS is a protocol and architecture design that you can use to build network application components (clients, servers, peer-to-peer nodes) for many purposes. It offers several convenient abstractions for the application programmer to support writing complex distributed applications.

VOS is explicitly object-oriented and supplies the infrastructure for passing messages between objects. Messaging is network-transparent, so the programmer is not concerned with the specifics of message delivery --- he or she simply passes the desired message to the target object. With this it is easy to implement remote procedure calls, where a message replying to a request can be matched up with that request. So far so good, but this describes about a dozen message passing systems already in use. What makes VOS different?

At the risk of stating the terribly obvious, in order to communicate with an object you must first know how to find it. Let me give an analogy with file systems: consider how difficult it would be if you could only refer to files by their inodes. In other words, directories don't exist, rather there is simply a large flat index of numbers, one for each file. To find out what other files exist you have to parse the files you already know about to find the inodes of the new files. Every file stores this information in a different place, of course. This would be a nightmare, wouldn't it? Well, this is essentially the state of the art of current distributed object systems. Typically in such systems objects are assigned unique identifiers by the object request broker (ORB, to use CORBA terminology.) Although an RPC call can return object identifiers allowing it to refer to other objects, every class will have a different way to do this. There is no uniform way to refer to an object by indirecting through another already known object. While it is the case that some systems such as CORBA offer a naming service on the side which does address some of these issues, there are many advantages to be had building such a feature into the core system that a separate naming service cannot provide.

A crucial feature of VOS is providing a standard protocol for expressing object interlinking, to the extent that it is part of the object model for a Virtual Object (hereafter termed "Vobject"). Vobjects consist of the following parts:

  1. The Vobject exists with a unique name as a direct child of a Site. A Site is a Vobject (following all the other aspects of this object model) which acts as a connection point between processes. This is typically accomplished with a network socket.
  2. The Vobject can exchange messages with other vobjects, with sites acting as intermediaries as necessary.
  3. The Vobject has a set of type strings. These may be used to describe what interfaces this vobject supports, how to interpret the meaning of child vobjects, or tag the vobject for other special treatment by an application.
  4. The Vobject has an ordered, associative list of links to other Vobjects. Following normal tree terminology the linked-to vobjects are termed children. Each link is tagged with a contextual name, used by the application to determine how to interpret the child in the context of the parent.
  5. Vobject parent-child links are two-way. A Vobject can be linked to more than once, and the application can get a list of all parents of a particular Vobject.
Nearly all other features of VOS derive from this object model. Although we call VOS a "hierarchical" distributed object system, object interlinking is actually a directed graph. However, it is at its most useful when used to group objects in "contains" or "describes" relationships. We use URLs to refer to vobjects, so the meaning of the following example should be fairly intuitive:
  • vop://interreality.org/world/ball/position
This refers to the vobject storing the position of a ball, which is contained in a virtual world at the site interreality.org. Note that this path is not the only way of referring to this object. Here are a couple more ways:
  • vop://interreality.org/1264095060
  • vop://interreality.org/1474612399/position

The former is the unique name of the vobject on its site. The latter directly addresses the "ball" vobject (instead of indirecting through the world vobject) to the position vobject.

Note that the fact that "vop://interreality.org/1264095060" happens to represents the position of something is entirely contextual, and determined by the parent-child linking. Another vobject could easily link to this with a different name and an entirely different meaning --- not so farfetched as it sounds, because a vobject can take on multiple types describing multiple interfaces.

This system provides the programmer a powerful and flexible way of organizing network-accessible objects. Because vobjects link in a uniform way, the same tools to access and inspect the vobject structure can be used for a variety of applications. For example, one tool is the command line shell mesh which allows one to browse vobject structures using familiar commands like "cd" and "ls" as well as send and receive messages interactively.

VOS is heavily event driven and allows one to register as an event listener to be notified when certain things happen, such as changes to a Vobject's list of children. This supports (and encourages) a style of programming similar to the model-view-controller (MVC) framework.

VOS has a number of other features which I will not enumerate here, for the sake of brevity.

As useful as it has turned out to be, VOS was not an end unto itself. Rather it is being developed to support a specific application which I briefly describe here.

The application is distributed, interactive multiuser 3D for the Internet. Few applications in computer science have received so much attention in the popular media, but have thus far failed to deliver in terms of an open, general purpose system. When deciding to embark upon this project three years ago, we decided to take a network-centric approach, rather than the more traditional approach focused on rendering. Three-dimensional graphics have advanced enormously in recent time, whereas much of the software infrastructure of the Internet --- application protocols like HTTP and RPC systems like CORBA --- have shown to be inappropriate for this task. Rather than work around the limitations of existing software, we decided build our own which has today evolved into the present VOS design and implementation. Although ultimately built with the purpose of describing 3D scenes, VOS is much more general purpose: one application for VOS is a network-accessible MP3 jukebox. I have also gotten email from someone intending to build a P2P network based using VOS.

We use the excellent Crystal Space 3D engine rather than writing our own, which allows us to focus on the networking and user interface aspects rather than the hairy details of 3D rendering. Presently the 3D client supports box and sphere primitives, loading 3D models from several standard file formats (such as .3DS), billboards (2d sprites that always face the camera), specifying world geometry directly using vobjects, and colored point lights. These objects can be added, removed, positioned, scaled, rotated, and otherwise modified on the fly through either direct manipulation in the 3D client, or using other tools such as the previously-mentioned mesh command line shell (which is able to do so despite having no intrinsic knowledge that the vobjects being manipulated describe a 3D scene.)

We have implemented a Virtual Webcam as an option to the 3D client that writes out an image file rather than rendering to the screen. If one visits our web page, one can see a live view of what is happening on our world server. If you were to join the world, your avatar would show up on the web cam (as well as the screens of anyone else present.) Furthermore, the webcam itself exists in the world with an avatar (a model resembling a movie camera.) The camera can be rotated and repositioned like any other 3D object, and the webcam's view will change accordingly.

The VOS reference implementation is in C++ (with additional lightweight implementation in Perl) and licensed under the GNU Lesser General Public License (LGPL). I believe that others might find the VOS platform useful for developing their own network applications. Rather than


The bloody posting preview button is broken., posted 6 May 2002 at 03:20 UTC by tetron » (Journeyer)

I tried to preview this and instead it got posted. Thank goodness my hitting "preview" again didn't end up in it getting posted twice :-) I didn't realize there was a word limit, so the last paragraph was cut off. Here's that paragraph in full:

The VOS reference implementation is in C++ (with additional lightweight implementation in Perl) and licensed under the GNU Lesser General Public License (LGPL). I believe that others might find the VOS platform useful for developing their own network applications. Rather than writing your own connection management, messaging, parsing and multiplexing infrastructure, let VOS do the legwork and you can focus on the business of, to paraphrase Eric Raymond, "scratching your itch".

Second correction, posted 6 May 2002 at 03:25 UTC by tetron » (Journeyer)

The proper link is MP3 Jukebox. Checking this stuff is what preview buttons (when they work) are good for :-)

It is interesting, posted 12 May 2002 at 18:03 UTC by Malx » (Journeyer)

I have read all introductory docs, but I could not say I grok it all :(

It is all simple untill you look at C++ example. It is strange to me.

The idea of hiding network communications from developer could be supposed good, but could lead to data loss during network problems. I thinks only application/Vobject should deside need it to resend data on connection reestablishment or not (Just think of Junk-bombing some site).

<quote> We should also implement a policy of disconnecting any peer sites that violate the protocol, by (for example) lying about their length fields </quote> Isn't it shows, that protocol is not so good? You shouldn't have duplicate data in it (data length). It whould help if you could trust it, but until then it is just useless. Still it is essential for binary protocol to be implemented.

Take a look at Jabber protocol. Isn't it good idea to use Jabber for your Vobject communication ? Even If not - look at their karma module , which is essential for bandwidth limitation/control (it is easy to DoS your service without it).

BTW. Why have you developed new protocol instead of HTTP ? It whould be very hard to convince corporate/intranet users without any Internet connection to set up special VOP-proxy server (they have only HTTP-proxy and mail gateway). Also great issue is Vobject response caching. Look at ActiveWorlds (sorry, it is win32 and it is hard to finde docs on site - use "search" - Here is some docs/api). They are developing 3D virtual reality world for 5-7(?) years. They use HTTP extensively for object/textures/avatars transfers (with http-cache support), but use own protocol for chat and avatars position updates.

You are mention "Snow crash". Take a look at Russian version of virtual reality space : "Labyrinth of reflections" by Lukyanenko Sergey [en].

And... try to write paper "Limitation and restriction of VOS" instead of "Features and possibilities". It helps you understand VOS future :)

network error hiding, posted 14 May 2002 at 04:28 UTC by lkcl » (Master)

take a look at dce/rpc. network errors can either be hidden or propagated back up to the application.

it is essential that this be done.

ignoring network errors or hiding them from the application is fine for simple programs in which to test whether the application can be distributed.

when it actually comes down to creating real and reliable applications, recovering from network errors, e.g. in transaction-based or distributed file system applications where locking is essential, becomes part of the responsibility of the application.

other than that, great! go for it!

Thanks for the comments, posted 16 May 2002 at 07:50 UTC by tetron » (Journeyer)

Malx: The C++ example is presently not very good, as it was derived from a simple app used to test features rather than to be a simple demonstration of the API. I plan on making a big push to improve documentation fairly soon.

Network errors (disconnections, timeouts, access control violations, etc) are propagated up to the application via exceptions and other event mechanisms. If something fails, the application WILL know about it.

The purpose of having an ascii protocol is a) makes things vastly easier to understand and debug and b) makes it very easy to write simple things like perl scripts. Don't underestimate how useful it is to be able to interact with a server without needing an extensive set of specialized tools (like IDL compilers...)

The reason why there is a "length" field in messages is for parsing efficiency. The parser will not be run on the again message until that number of bytes has been received. It is possible to detect the beginning and ending of the message without knowing the exact length before hand, but it is less simply efficient.

Actually, an possibly even more efficient way to do this would be to modify bison to support incremental parsing, so that it saved state and returned when it ran out of input rather than failing. I'm not sure how easy or hard this would be.

Jabber does share some superficial similarities with VOS, but Jabber is focused on text messaging and isn't a full Distributed Object System (since last I checked.) I will look into the Karma module, thanks for the suggestion.

Using HTTP as an RPC mechanism is and always will be an ugly hack. By the time one has extended HTTP to support things like persistant connections with subscriptions and updates and explicit object structures, you require specialized servers and clients and have eliminated 90% of the reason for using HTTP in the first place. I don't know how active worlds works because it is propriatary and windows-only. It is also based on a central server (or more likely a cluster of servers) for the entire "world" which runs completely counter to the decentralized design of VOS, the web and the internet in general.

For the core VOS library there is actually only one feature left to implement before it being feature complete for 1.0, and that is an authentication architechture. We're tentatively planning to use SSL for this.

Anyway, thanks again for your comments!

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!

X
Share this page