Comments on Jabber Internet-Drafts

Woops.. really messed up on those URLs. They are all in the /rfc/ directory. Please use these URLs:

XML Streams { TXT | HTML | XML }

Jabber Protocol { TXT | HTML | XML }

Thanks, and I apologize for any inconvenience.

Both drafts seem very simple, which might be a good thing, but they feel very much like a bare-bones type implementation, and I can't really see how they solve, or even address, the issues of scalability and security at all (it mentions that there are no parts of the protocol that prevent implementation of security, but that can be said about mostly any protocol, and IM systems having traditionally been riddled with enormous holes, I had expected that that would be one of the first issues a new protocol should address).

Also, the XML-as-protocol thing, while interesting as a concept, doesn't seem like a very wise decision, for two reasons, first of all that it tends to blur the layers of the protocol (payload can only be separated from protocol by the use of an XML parser that implements namespaces, as far as I can see), and the general performance hit of using XML to do protocol (especially since you really would need to do validation to have a reasonable level of security against maliciously deformed data, etc.) could prevent scalability. Maybe not for 100 users, or for a 1000, but things like these should scale to at least a million simultaneous online users, probably on several servers, and then you need to route your messages fast.

So, my question is how much protocol implementation experience the authors of these drafts actually have. There are things in here that smell of buzzword-compliance.

Radgast:

Current internal XML debates (you don't want to know) nonwithstanding, essentially every XML parser out there implements namespaces.

As to the problems of validation, I don't think this is as serious as you think. Proper element content would probably not need to be specially checked for (the regular code would reject it) and thus the only security check would be for well-formed but illegal content, probably in text format. This wouldn't be much harder than checking that sort of thing currently.

IRC provided instant messaging services for years and is a well understood piece of technology. Instead of piling up XML crap why don't we address known issues in existing protocols. That would be useful for net people and open source hackers. Instant Messaging companies do not want open protocols, so it is going to be RTP-2 all over again.

Things that IRC needs in my opinion are:

• User authentication with the server and user database on the server as a part of the standard. Guests may be permitted or may be not.
• Standard encryption.
• Better scaling, including:
  • Point to point /m without round trip through the server.
  • Server reestablishing connections when /m is directed to a user.

There are many more. By doing a new revolutionary protocol you are going to discover them all again, which is admittedly a part of the fun. So decide what you want: doing something useful for the community or exercizing in IETF procedures.

Zaitchev: I strongly disagree, and suspect you don't really know the issues. There are two reasons why what you say is incorrect in my opinion.

Firstly, IM protocols are not the same as IRC. IRC is largely channel-based many-to-many communication, with some one-to-one communication tacked on. IM is largely about the brodcasting of and subscription to presence information, as well as quick dispatching of small payloads user-to-user.

Secondly, the IRC protocol is a severely brain-damaged piece of crap. If you need examples, I can provide, but the way they bastardized the standard UN*X line protocol conventions into something that would marginally support pushed, not only pulled data (as well as the original DCC protocol being based on modem protocols, presumably because the implementors didn't know what services are provided by TCP) should be enough to make anyone not consider IRC as an alternative here. In fact, IRC could stand to be replaced with something that's better designed, but that's a separate issue.

You would also do well to remember that the large IM providers (AOL and Mirabilis) have pledged to support an IETF standard in this area, so it's important that the standard is good. I don't think the Jabber protocol cuts it, but if I had to choose between Jabber and IRC, on their relative merits for use in IM applications, it wouldn't take me long to toss IRC out.

The concerns on speed of parsing are very valid, and we've started to tackle it in some ways. Our xmlstreams system, is actually quite helpful in this area, because we don't have to parse an entire document each time data comes in, rather just a piece of the document. We are also persuing some architecture changes are taking place in the 1.1 devel series to further address massive scaling. Drop me a line to discuss those further.

Security is one of my favorite areas, and I've been working on security in Jabber since almost day 1 of when I joined the project. As you can tell what we have in there currently is marginal at best. Digest Auth, for both users, and transports, and SSL for user to server communications (very important to note that it is not server to server). We would love to have a PKI system in place so that signing and encrypted messages can swiftly be in place, but MAN is it hard. Because the architecture is distributed the wide array of man in the middle attacks is staggering. To compound the security issues more, we currently have a high trust level invested in the server itself, and there has been talk of trying to remove some of that through encrypted buddylists and other information storage, but again this is still hard. We are very hard at work, and it is my next issue to talk after our first round of scaling work is done.

Jabber has been a phenomenonal mind boggler for us, as we have begun to uncover ground totally untred by anyone. With great comments like yours we hope that we can continue to grow and expand into those grounds, and make it a better field for everyone.

First of all, I see your reasons for using XML, but I'm still not sure it's a good design choice, and I'm very unconvinced it's the only way to get "true extensibility". After all, line-based protocols provide the same level of extensibility, just embed the protocol version in the request. HTTP does this, quite sucessfully (the growth of the specification from 1.0 to 1.1 is staggering). As you're using namespaces anyway, it's not like you can grow more flexibly with XML, you'll need to provide a DTD for your protocol, and the versions will vary, which means you're going to have to support more than one DTD, and so on.

I was guessing you were doing something like parsing fragments instead of whole documents, that was the only way you could get it to be fast enough. I still see severe scaling issues with this, though (what sort of throughput are you achieving at the moment, on respectable server hardware? Have you benchmarked?) I'd be interested in hearing about what you're thinking about to increase scalability, I can't think of much that'll work, except tacking on non-XML data as a header (much the same as HTTP does).

A PKI system, or crypto at all, isn't really what I was thinking about when I mentioned security. As shown by (particularly earlier versions of) ICQ, security in IM is more about making sure who has access to your presence info, for instance. In a distributed system like Jabber, that's even harder. For instance, it's not that difficult to make sure your presence info doesn't get to a client you've blacklisted if you're running on a centralized server (not that Mirabilis didn't screw that up anyway), but when you're on a distributed network, how do you prevent compromised servers from giving that info to people who aren't supposed to have it? Tricky, tricky. PKI can be added later, first work out a model of trust for servers, etc. In fact, I believe the work Raph's been doing on trust metrics might be worth looking into for you, it's specifically designed to address the problem of a network of servers where the integrity of a given server is unknown.

Encrypted buddylists and other information storage (I assume this is stored on the server) should be trivial in most cases. Man in the middle attacks? Well, that's also kind of a buzzword. There's not much you can do to prevent them, except using a shared secret, although a lockstep protocol might be able to help you out here, at least reduce the risk. But mand in the middle attacks are pretty far-fetched even where the gain from revealing secret info is high, and I think your time is better spent looking at other areas of security.

I'd love to hear more about the proposed features, though. For now, I'll change my opinion of the Jabber protocol from "bad" to "immature". So let's see. Heh.

Firstly, IM protocols are not the same as IRC. IRC is largely channel-based many-to-many communication, with some one-to-one communication tacked on. IM is largely about the brodcasting of and subscription to presence information, as well as quick dispatching of small payloads user-to-user.

Secondly, the IRC protocol is a severely brain-damaged piece of crap. If you need examples, I can provide, but the way they bastardized the standard UN*X line protocol conventions into something that would marginally support pushed, not only pulled data (as well as the original DCC protocol being based on modem protocols, presumably because the implementors didn't know what services are provided by TCP) should be enough to make anyone not consider IRC as an alternative here. In fact, IRC could stand to be replaced with something that's better designed, but that's a separate issue.

You would also do well to remember that the large IM providers (AOL and Mirabilis) have pledged to support an IETF standard in this area, so it's important that the standard is good. I don't think the Jabber protocol cuts it, but if I had to choose between Jabber and IRC, on their relative merits for use in IM applications, it wouldn't take me long to toss IRC out.

[Pity that replies cannot be deleted or altered. One wrong keystroke and...]

By Radagast:

Firstly, IM protocols are not the same as IRC. IRC is largely channel-based many-to-many communication, with some one-to-one communication tacked on. IM is largely about the brodcasting of and subscription to presence information, as well as quick dispatching of small payloads user-to-user.

This is an artefact of the way Radagast uses IRC. Others use it for presense control. People do connect and not join any channels. So, irrelevant.

Secondly, the IRC protocol is a severely brain-damaged piece of crap. [...]

No doubt about that, but look at the issue in this way: does it work? what advantages do we receive by switching? On the last USENIX there was a presentation "What is broken in HTML and why it does not matter." I think it rings familiar.

You would also do well to remember that the large IM providers (AOL and Mirabilis) have pledged to support an IETF standard in this area,

Can I sue them if they do not? Certainly you cannot trust that.

You ask "Does [IRC] work? What do we gain by switching?" This is missing the point. We're not talking about switching anyone away from IRC. You are not the target user here. ICQ has at least 50 million registered users. AOL has what, 30 million users? This is a question of making these people use a standardized protocol. IRC is irrelevant in this context. Noone mentioned IRC. Noone's trying to obsolete IRC. This is a different discussion.

Also, if you think suing is the only way you can ensure that providers move to the standard, you haven't been paying attention. Standards-compliance is the buzzword at the moment. Look at how much flak browser makers are taking for not supporting the standards properly. If AOL and Mirabilis, who have pledged to move to the standard when it's ready, fail to do so, they'll be ridiculed, not to mention out-marketed (network economy at work, no single vendor can lock in the IM market with a proprietary protocol if everyone else uses an open protocol).

Ok, this IRC thing has gone WAY off topic... but I thought I should mention a couple of items.

Like Radagast said, Jabber isn't designed to take over IRC. That isn't the target market. IM is very different than IRC. Even AOL knows this, as they have their own "IRC" (chat rooms) as well as "IM" (Buddy List / Instant Messenger) within the same application (AOL). Yes, IRC does this.. but Radagast made the good points on that already.

The main points I wanted to mention are: Jabber does IRC. The first support we had was being able to connect with your IRC client to Jabber. Instantly you have a great Jabber client with full functionality (anyone want to talk to ICQ, AIM, Yahoo!, MSN, etc. users via your IRC client? ). Secondly, we added extended our "groupchat" (conference room / chat room) service to allow for IRC channels. Instantly, any Jabber client capable of using groupchat was able to connect to any IRC server and act just like a regular IRC user. None of these enhancements took very long to implement, a couple of things took some brainpower (mostly because of how horrible the IRC protocol is), but they were both implemented within a month or so of conception. Another cool IRC feature is being able to add IRC users to your Jabber contact list... just like any regular user. I'm not sure if this has been been implemented on the Jabber.org server, but I've seen it work and it's cool... Most of this work has been done by TSBandit.

Anyhoo.. completely off-topic... but interesting nonetheless.

I briefly talked about security in Jabber with Julian today, and I got a confirmation of something I'd suspected from things I'd heard earlier: That Jabber implements little or no security because the developers haven't managed to come up with a scheme that's not vulnerable to a man in the middle attack. If this is true, it's a shame, because it means people haven't really understood the problem. So I thought I'd post a little about it.

Basically, man in the middle attacks are impossible to prevent, if you're using public key cryptography without any external channel for confirming identity or some sort of shared secret. There are a few techniques that can be used to reduce the efficiency of a man in the middle attack, most notably interlock schemes, but they're only partly effective.

So, there's no hope? Well, it's interesting to note that hysteria about man in the middle attacks is pretty typical in people who haven't really thought about the implementation issues. Yes, MitM attacks can break your crypto scheme. But what's required to do this? For a MitM scheme to break a good security system, the attacker needs to be able to alter packages in the traffic between the two network entities in real time, and has to have been able to do this from the very first time the entities communicated. Think about it. Not eavesdropping, which is relatively trivial, but actively changing the network stream. From the very start of your communications with the other entity, and every single time you communicate with it after that.

In a practical situation, this is highly unlikely, bordering on impossible. If someone can do this, they can do the same for your communications with your mail server, web servers (including SSL), IRC... Basically, if your enemies are this powerful, you have a big problem. Not that you can't design a system that makes it possible to secure against MitM attacks if you know you have the need for it, just make a fingerprint of the signature, call the other person up (or, if your omnipotent enemies control the phone system too, meet them in person, in the middle of the Gobi, or something) and read the fingerprint. Of course, not everyone will do that, but not everyone needs that level of authentication. For most people, that level of security is unnecessary.

In short, if you're holding up development of any security because you can't do any better than the protocols used for, for instance, e-commerce, I think you're making a mistake. You're not going to come up with a better security solution than people like Schneier. You could, theoretically, say "All the world's crypto schemes are bogus because none of them completely solves the MitM attack problem, so we're not going to implement any of them, as a protest", but you'd look pretty silly doing so, and the rest of the world would go on its merry way, implementing another protocol that doesn't let random people in the same dorm eavesdrop on their steamy messages to their SO. You can do good authentication and encryption with simple application of existing technologies, and you'd make it any normal person's definition of completely secure. If you in addition make it possible for extra paranoid people to use key fingerprints to identify each other via an external channel, you're going to get a medal.

If anyone involved wants to discuss these issues further, please either post here, or mail me, joakim at helixcode dot com.

First, let me get this off my chest: #!$^@#$%^&@#&%$!

Sorry, I just spent an hour composing a decent reply to various issues and X crashed when I hit the Post button :(

The sun is rising, so I'm going to have to make it much shorter this time.

First, XML wasn't a "choice" made during the design phase, but instead it was the original goal for the project, "how can we use XML as the payload for an IM and Presence system". So, it's difficult to argue for/against XML in this sense :)

Of note, although XML seems quite complex, it's in fact very easy to parse, no more difficult than any structured text parser. There is a very simple set of rules and tokens that can be applied to a stream of characters to parse it as XML, no special buffering or heavy CPU calculations required.

We're using Expat for our parser on the server side, just like Perl, Apache, Mozilla, and many others, and it performs superbly.

Now, on to the fun section, security. I'll try to sum up the almost-article I had written:

The issues have been confused in reference to MitM and Jabber. The facts about MitM were very well stated and that's not in question, nor is Jabber attempting to create or waiting for a 100% solution (if one will ever exist). In fact, ensuring that Jabber is secure is one of the most important goals for us.

The issue boils down to who you're trusting. For authentication, accessing your roster, and message delivery, you need to trust the server. For this we use SHA1 Digest authentication by default, as well as have SSL server connections available.

Trusting the other individual you're communicating with is the real issue, and we're specifically not forcing you trust their server in order to trust them. Their server could be compromized, mis-administrated, monitored, or provide a non-secure method of access to it's clients. Essentially, we don't want to try and enforce a Jabber server as a trusted man in the middle.

What then? Of course, I also want secure-as-freaking-possible communication with other individuals via Jabber, so I've given it some thought :) The trust has to begin when the user enters the message, within the client. Just like email, clients need to support signing/encrypting messages and presence (via PGP/GPG/etc). Unfortunately most of the client authors are quite busy just making their clients work, but if anyone else would like to contribute or work on this area they are welcome to join us.

Last quick note: as mentioned, the existing two drafts are incomplete. They are only the groundwork, I'm nearly finished composing a third "Jabber Architecture" draft which pulls them together in a bigger picture. I hope to have the existing drafts updated/fixed and the new one available in the next day or two.

Hopefully I didn't forget anything important this time, and the sun is now up :(

Why would I want to run a new client that speaks a new protocol that nobody uses, and requires me to talk to a server for this new protocol that acts as a gateway to the existing and deployed protocols before I can even get anywhere?

What I want is a single client that transparently speaks many protocols, using a consistent user interface.

Jabber just seems like a completely bass ackwards way to go about it.

Incidentally, the notion that ``chat'' and ``instant messaging'' are different things is complete crap. AIM is ``IRC where every command begins with /msg and /join is missing.'' How can anyone claim that the ability to talk to N people is so fundamentally different than the ability to talk to one person that it should have a different protocol and client program? That's like saying that you should run one email program when you want to send mail to individuals, and a different program when you want to send mail to mailing lists. It's lunacy.

(Some mail archive archaeology later...)

About a year ago Jeremie asked me what I thought of the Jabber design, and this is what I wrote. He never sent me a rebuttal, but maybe it will interest you folks:

When you put the multi-protocol logic on the server, you get a few different problems:

• Installation is far harder: you either need to be able to connect to an external Jabber server on whatever port it speaks, or install and run your own server. For many people, this will be a big deal, and require all kinds of root access including tweaking the firewall. This isn't an end-user kind of deal.

• You can't use the same client to talk to your internal, inside-the- firewall IRC (or whatever) server, because the external Jabber server can't get to it.

• Instead of one point of failure, now you have two, connected in series: you're dead in the water when either the Jabber server or the "real" server is down (and "down" can mean simply routing lossage.)

• You've more than doubled your scalability problem: AOLIM, ICQ, and IRC have each already solved the hard problem of managing millions of simultanious users (the first two with deep networking and replication magic hidden far behind the scenes, the third with a network of servers each of which handles a relatively small number of users.) If Jabber is to be successful, it can only do so by scaling up to be able to handle the sum of the number of users of each of those other services. Even if you had a good design to accomplish this, and the resources to implement and deploy it, it's still incredibly wasteful. Why spend the hardware on solving this problem? It doesn't need to be solved at all.

None of these problems exist if the multi-protocol logic is in the client. In addition, I don't see what problems are solved by putting the logic in the server instead of in the client. It looks to me like it's all down-side and no up-side.

First, XML wasn't a "choice" made during the design phase, but instead it was the original goal for the project, "how can we use XML as the payload for an IM and Presence system". So, it's difficult to argue for/against XML in this sense :)

Regardless of whether XML is good, bad, or indifferent, or whether or not it's relevant to on-the-wire protocols, that's got to be the worst of all possible reasons to use it. You should use a tool because it's the right tool, not because it's a pretty tool. You should define your problem then find the best way to solve it. If you list the tools you will use in your problem definition, then the problem you're solving isn't ``how can I make chat better,'' it's ``how can I find something to use XML for.'' And that's just wanking: it may be personally rewarding and intellectually enriching, but from an engineering perspective, it's utter folly.

Again, I'm not saying XML is the wrong tool for this job. I'm saying that if the above is really the reason that you're using it, and if it turns out to have been the right tool for the job anyway, you just got lucky.

Hi again Jamie :)

I apologize for not sending this rebuttal last fall, I don't like arguing for something that isn't real yet, so I decided to make Jabber work first.

First, you're dead on with the "XML for its own sake" comment. I started Jabber using XML specifically because it was personally rewarding and intellectually enriching. I wanted to understand XML and how it would work for this type of application. I also now believe that it is an excellent tool for the job Jabber is doing, or as you put it, I "just got lucky".

The first rebuttal I'd like to make isn't even a rebuttal. When I started Jabber, I had in mind exactly what you first wrote about in unity of interface and are asking about now, a single client that can transparently manage the protocols that it's using via a single consistent user interface. I simply chose "XML over a socket" (for reasons above) as the module API. Jabber is still this, the "server" doesn't have to be run on a remote box or be used by multiple people, anyone can run the server and transports locally for their own use. It's simply a matter of naming and packaging, with a little work (which nobody has had time to do yet) the server and transports can be packaged and bundled with a client so that they work transparently as a background module API. It's not absolutely perfect or amazingly optimal, but it is still 100% of the ideal you're seeking, and can be improved on to be even closer. So, although you see Jabber as something completely different, I've always seen it as exactly what you describe, just with additional functionality ;-)

In fact, let's do it this way... suppose my original goal was an end result approaching your argument (which it was), and let's evolve it over the past two years of development (which has happened). First, I thought hard about all the chat protocols, looking at them, probing them, understanding them in every way I could. IRC, AIM (OSCAR), ICQ (yuck), SMTP, talk, etc. The most obvious fact was that these are a hot topic in development, changing often, especially the reverse engineering efforts for AIM/ICQ. So there was an outstanding requirement for this chat protocol API to be as automatic as possible in how it gets upgraded. This could be accomplished with a nice modularized system, where you're distributing signed compiled binary modules or scripts automatically to the clients. "Hmmm..." I thought, being the lazy person I am, that sounded like way to much work. Besides, the general trend of high maintenance and communication apps has been to the web, where they are 100% dynamic, automated, and managed.

Second, I'm not a GUI programmer. Some DHTML toys are the closest I've ever come to doing GUI programming, nor do I really want to come any closer. So, if I'm going to be asking people to use this API I'm building to build GUI clients, it better be as simple and straight forward as possible or it will never happen.

Having these two basic requirements (automatic upgrades and absolute simplicity) and looking at my programming background (cgi/server apps) and current area of interest (XML), I created Jabber: abstract all the chat data into a common base, encode in an XML syntax, and communicate this to the client over a socket. "Cool" I thought, I can do this! Write a daemon that a client communicates with via a socket, sending simple XML encoded data to it which is intelligently routed to various modules that handle the translation to a native protocol. The clients have zero knowledge or involvement with the complexity of dealing with native protocols or a heavy API, and the translation components can be updated willy nilly without involvement of the client. Heck, the daemon can be hosted on a server where the translation components can be upgraded for a group of people all at once transparently... wait, wouldn't this group of people be talking to each other most often? Hmm... then why not assign them an identity within this daemon and just deliver the XML data directly. Wait, it doesn't make sense for these groups of people to be using these special identities just amongst themselves on their server, why can't they share it with others that use this server?

That was the 30-second summary of the first year of Jabber development, but hopefully it gives you some insight as to how we ended up where we are today, accomplishing the same goal as you originally set out but evolved somewhat.

So, although Jabber can be used 100% in the role that you're asking, I'd hope that I'm not the only one that sees the value of utilizing it as a server. In fact, I'd like to use the email analogy that you've used. The Jabber server and protocol is serving the same role for chat that Sendmail and SMTP has served for email. You may retrieve your email via POP/IMAP, but it arrived via a SMTP server which may have received it from another SMTP server or possibly a SMTP gateway to another mail protocol. Not too long ago there were many email systems that did not talk to each other, very similar to the world of "Instant Messaging" today. I can't see running an email client that connects to my AOL, Compuserve, Work(Lotus/Exchange), and ISP accounts, or even having to have accounts on all the systems just to email those users. OK, so the analogy isn't perfect, but lets move on :)

One of the most important benefits of utilizing Jabber as a server is this: flexible client access. You can implement a client GUI on any platform that has TCP/IP, and only have to implement as much as you want to support. Clients can be simple or complex, in fact, you can do a client in shell using netcat, or type the XML by hand with telnet. Also, there is full programmable access to all the abstraction Jabber offers, and already libraries available for this purpose in C/C++/Perl/Python/Java/TCL/Lisp, and more. You can also build alternate client interfaces to the Jabber server, one popular one in development is IRC: use any IRC client and do a "/server jabber.org", the IRC GUI is used as the Jabber client (it's not perfect, but quite usable). There are telnet, HTTP, WAP, and Palm.net gateways being built in the same fashion, providing more open access to all the abstraction Jabber offers.

The second more subtle but possibly more powerful enabling aspects of a Jabber server is a new medium. By taking that step where each Jabber user has an ID and servers communicate with each other routing this XML data directly, a whole new platform and medium is born. On this medium new applications can be built, based on a common simple XML definition of messages and presence. Web-based chats, collaborative tools, log alerts, pager gateways, RSS headlines, calendar/event alerts, even non-fun things like B2B, EDI, and ecommerce at some point can make use of a system that's immediately delivering XML messages or of knowing the presence of another entity. For those groups of people using a Jabber server, they communicate most frequently with the others in the group only via Jabber, and the alternate protocol support is only used for communicating with others not yet using Jabber.

The last little note which has even surprised me, is that the larger portion of the interest in Jabber isn't specifically in it's functionality as an all-in-one chat solution, but instead that it's an open source IM/buddy-list server. This is also all predominately commercial, coming from ISPs, corporations, portals, stores, online interest groups, service providers, etc.

In the end, I'd like to think that your unity of interface ideas are fully implemented by Jabber, it just has additional functionality and even wider open access. Much of the fault for any confusion is because of our lack of clear or proper documentation on many aspects of Jabber, but that is quickly changing.