XMPP is a very popular open source chat protocol, using a client server model, finding many useful applications ranging from IM (Instant Messaging), to Collaboration tasks, to M2M (Machine to Machine communication), allowing easy deployment and customization over a score of platforms. XMPP can be found silently at work inside the most popular chat clients, such as Facebook Chat, WhatsApp, and even Google Hangout (Though google is migrating to a proprietary scheme this year).
In the next few paragraphs and in future articles we will cover why it has become so prevalent worldwide and how it has been used in the development of LiveBox.
A brief history of XMPP
The XMPP Protocol (Extensible Messaging and Presence Protocol) was born in the mind of Jeremie Miller around 1998 as a replacement and improvement over the then existing protocols, which were mostly proprietary or very limited and were difficult to extend and port onto new platforms.
XMPP made its first appearance in 1999 with the initial name of Jabber and its first implementation Jabberd is still in wide use. Soon, the idea of a new open protocol attracted the attention of enthusiasts, programmers, engineers, and even the IETF (Internet Engineering Task Force) who is responsible for much of what makes the modern web possible by redacting and updating the standards documents. Acting together after years of work, in 2004 they published the first two documents describing XMPP stringently, these two documents RFC 6120 and 6121 would go on to become the core of XMPP. These described the core functionality, the main rules to be followed to implement a client and a server capable of interacting with other XMPP clients and servers. Now, 10 years later, XMPP is a flourishing Protocol with hundreds (yes, literally hundreds) of extensions (additional functionality built onto the core protocol) and millions of users worldwide.
What do we use it for?
XMPP has quickly become a core component of the LiveBox suite, and serves several functions. A small reminder first. LiveBox has a client server structure, with clients (be they Windows PCs or Macs) asking the server for remote changes to their data, or notifying the server of local updates. In the field of IT there are two main ways of doing this. An easier, but inefficient way, and a trickier, but much more rewarding way. These are polling, and pushing respectively. Very long lectures could be written about polling and push based synchronization approaches, but that goes beyond the scope of this article, I trust the following simplified description should suffice for the purpose of this article.
The Polling scheme can be explained with a simple example. Imagine having a server with 5 clients, each client polls the server every 10 seconds to check for changes. Only 2 of these clients actually have changes requiring notification once to them in a minute of running time.
This would result in about 2 useful communications per minute against 28 useless ones. This means, in our specific example only 6% of connections have any purpose, the rest are just a waste of server time and bandwidth. Bring this to an hour, and you’ll have 1680 useless connections against only 120 being meaningful. And this is after all just an hour of running time, with only 5 clients. Picture this scenario taking place on an average corporate network. Yes. Your hair should stand on end at the very thought.
In short, polling requires every client to call the server regularly in order to remain synchronized, but this results in many pointless calls when there are no changes to the data. Push based approaches instead require clients to call the server only to notify changes, and the server to broadcast these calls to interested clients. So that only relevant information is passed. This is far more complicated to implement of course, but, as we’ll see, far more rewarding.
The Push approach, this approach instead relies on slightly more intelligent servers and clients. In this scheme, clients only contact the server to notify changes of their local data, and the server only contacts interested clients to inform them of remote changes to their data. This means that when no changes are taking place, there are no connections being made between them, they rely on changes being logged and notified when possible. The practical upshot of which is that if we take the same use case as our previous example, there will be only 2 connections per minute (by the clients being informed of remote changes by the server), for a total of 120 connections per hour, with an unsurprising efficiency of 100%. All that takes is some forward thinking.
That’s where XMPP comes into our picture. Rather than keeping permanently open connections with the server, we reasoned we could make each PC and Mac LiveBox client an XMPP client and each LiveBox server would have its own XMPP server, of course the mobile Apps and LiveBox Chat where still a glint in the Project Manager’s eye back then. This way, we could make the information travel safely inside a message, straight to the client machine.
This would solve several problems, firstly, the XMPP Server would take care of routing the message to the relevant LiveBox clients, and would also track those that were currently online (No point trying to send a message to an offline client). Secondly it would guarantee low overhead, being based on XML and as most chat protocols it was designed from scratch to handle large volumes of traffic easily. Lastly, and more importantly, the protocol was open and we could plan to expand easily onto it as we expanded LiveBox’s features and functionalities.
We went straight ahead and quickly implemented a plugin to achieve this on the chosen XMPP server, in our case Openfire, a popular Java XMPP Server, another open source project. Development time was short, and it worked very well for us.
So in the beginning, we only used it to deliver notifications, some time later when we started developing chat functionalities into LiveBox XMPP proved to be the logical choice, after all, even Google was using it.
This is no longer a short introduction I fear, but I hope that I have not bored you too much, I look forward to describing some of the key strengths of XMPP and some of the challenges and achievements we had in our first year with and thanks to XMPP in the next article of this short series on the subjects of LiveBox and XMPP.
As a closing note, I would like to post some references for further reading for those interested, as I know that I skimmed over lots of topics taking them as given or requiring each their own couple of articles.
Firstly I would recommend the reading of the wikipedia article on XMPP (Wiki link here) where most of the Protocol basics can be found.
For anyone interested in a deeper analysis of the protocol and its applications I would also recommend the very good book by O’Reilly on the subject.
Of course I will welcome comments and emails, be they for corrections, suggestions, or questions regarding the subject matter. Feel free to write to: email@example.com