2010-02-12

This 'Big' Is Bigger Than That 'Big'

Most people who have known me for any length of time know that I have a particular fondness for IM systems.  Unlike most, I don't think about actual people chatting when I think of IM systems -- I think of it as an infrastructure which provides a set of communication primitives.


IM seen as communication primitives.

Primitives that can conveniently be used for person-to-person communication, but which can also be leveraged by software agents that need to be able to communicate asynchronously,  from behind NAT'ed connections (or non-routable mobile IPs) and without knowing or caring where the other party is.  Having a concept of agent state, however primitive, is also a useful feature.  As are various forms of group communication primitives.

If you are the curious type you have probably set up a packet sniffer to figure out what your Android telephone is doing when it boots up.  I have to say that I wasn't surprised when I saw that it connects to what appears to be an IM service at Google.  I'll leave it as an exercise to the reader to figure out what this connection is doing.


How big is big?

Earlier tonight I was looking for information on how scalable one can expect ejabberd to be.  I did some searches and skimmed through some articles, blog posts and mailing list threads.  This reminded me that people have wildly different views on what magnitude they assign to words like big and scalable.  

For a lot of uses an IM system can be said to be big if it can handle what I call an "enterprise size problem".   Enterprises typically have a few thousand users.  Larger enterprises may approach the 100k mark.  For those uses, being able to handle 10k simultaneous users on a single node is a Big Deal.

Sadly, for "internet scale problems" 10k simultaneous users isn't even a credible test setup for an IM system.   "Big" starts at 100M registered users and some double-digit percentage of those being online and chatty.  Internet scale problems are usually addressed by architectures that are distributed -- often by having lots of moderately powerful servers in a datacenter.

If you can't handle 100M users affordably and you have no reason to suspect that your system is physically able to scale to 10x that with not-too-far-off-linear increase in cost, you are are doomed.  No point in even playing.  Go home.  Take up knitting or crochet instead.

My cursory examination of what has been said about ejabberd scalability suggests that it is probably fine for the enterprise, but it sounds too expensive to scale it to Internet scale problems.   Even worse, some other XMPP implementations I read about have even worse performance -- which is hardly a surprise when you take into account that there are still weirdos who dedicate an OS-thread to handling each connection -- in 2010.  Not even IRC (which has scalability problems by the bucketload) has servers that are this inefficiently designed.

This means that if you are in the business of providing an IM-infrastructure (XMPP or otherwise), you are most likely going to have to roll your own.

The really big IM systems I've glanced at run on relatively large farms of servers.  There is generally more than one type of server node.  Some handle connections, some track state, handle the roster, some do message routing, some even have persistent queueing in case the client is currently offline etc.  The architectures vary quite a bit, but the common theme is that it isn't as simple as just writing a IM server and firing up N instances of it to handle M users.  There is some division of responsibilities since you are not solving a problem you are solving a set of problems which are very different.

Quite recently various notes on how Facebook put together an IM system have surfaced on the net.  (Perhaps notable for their polyglot approach to the problem and for introducing Erlang into the vocabulary of people who follow these things semi-consciously).  The Facebook system sounds interesting and I am impressed at how quickly they were able to deploy it.

One last rant before I go to bed:

For XMPP the protocol is an issue.  Despite what the XML camp claims, XML is slow and clumsy if you need to parse a lot of it and you have to manage cost.  People who claim otherwise usually fall into one of two camps: people who have never implemented an XML parser and people who have only ever implemented an XML parser and whose basis for comparison is other XML parsers or other inefficient forms of serialization.

I'm sorry, but if you think XML is a performant technology I'm going to have to suspect you of not being a very bright person.  Please don't bother commenting.  I am not interested.

My main beef with XMPP is that it uses XML for framing:  that is for delineating the boundaries of a single message.  This was a big mistake and I was told (by people who were close to the standards process) that it was on the verge of being rectified, but never made it into the standard for various reasons.

In practical terms this means that a performant implementation of an XMPP connection server needs to have two XML parser implementations -- one that merely figures out the framing and one that actually parses the payload or some variation over this theme (you could do some shallow parsing in the "framing parser" as well, but you get the basic idea).

The reason is that you really do not want to have a big, stateful, fully featured XML parser sucking on the end of the network socket that the user is connected to.  If at all, you want to feed the message to the big hairy all-bells-and-whistles-standards-compliant XML parser only once you know you have a complete message.

This is further complicated by the fact that for languages that do not provide a good threading-abstraction that avoids mapping threads of execution to OS threads, you usually do connection multiplexing. (Which is one of the reasons Erlang is an attractive choice for a connection server). Even when you do have a good threading abstraction, you still care about the per-connection cost.   Having 10.000 connections with a big leaky XML parser allocated to each ... well, you see the problem.

Internally, large IM systems don't use XML.  They usually employ a more efficient serialization (and few inter-node connections).  The XMPP in full XML is mostly useful for integrating with user-agents or for federation -- not between nodes in a large IM system.

Of course, there is nothing preventing you from offering more compact and efficient protocols to clients,  but this is perhaps something you would want to do only in more specialized cases.  (For instance when communicating over mobile broadband ;-)).

The important part of XMPP isn't the client-server protocol.  It is the set of primitives offered.

Oh, and if there was any doubt:  if you are building a large IM system then I think it should be based on XMPP -- which basically means you have to support the primitives of XMPP in your architecture and you have to support the standardized protocols when talking to the outside world.

No comments:

Post a Comment