An open standard for instant messaging: extensible Messaging and Presence Protocol (XMPP)

(1)

Presence Protocol (XMPP)

Pin Nie

Helsinki University of Technology

Pin.Nie@hut.fi

Abstract

This paper discusses the eXtensible Messaging and Pres-ence Protocol (XMPP), which is the first effort in creating an open standard for instant messaging and presence informa-tion. XMPP was introduced by the Jabber Software Founda-tion (JSF) and formalized in the IETF. Numerous extensions called Jabber Enhancement Proposals (JEP) have evolved through subsequent work. This paper explores the XMPP specifications and technology behind XMPP, accompanied with examples of the application program. Both advantages and disadvantages are analyzed.

KEYWORDS: XMPP, Instant Messaging, presence, XML streaming

1 Introduction

Instant Messaging (IM) is a service, which enables people to communicate with each other in realtime. It supports dif-ferent communicating styles, such as unicast, multicast and group talk fashion. Quite a few protocols and frameworks that support the IM service have been created already. For example, AIM, MSN, Yahoo Messenger and ICQ are exam-ples of existing IM systems. In addition, Session Initiation Protocol (SIP) [10] based design, SIMPLE (SIP for Instant Messaging and Presence Leveraging Extensions) [14] and IM extension [9] also exploit the same functionality. How-ever, each of them has serious disadvantages. Commercial products are based on proprietary protocols, which block in-teroperability between different implementations and limit further development. SIMPLE [14] [4] is quite complex, be-cause it is built on the top of the SIP architecture. In order to overcome these problems and facilitate the use of IM service, another new solution emerges.

The eXtensible Messaging and Presence Protocol (XMPP), is a standard specified by the IETF for carrying IM service. It is an open XML protocol for near-real-time messaging, presence, and request/response services. XMPP was first proposed by Jabber open-source community and is still under the development. Later, it was formally approved and archived by the IETF in four Internet specifications [5] [6] [7] [8]. Due to its inherent good features, the prob-lems aforementioned can be solved. Furthermore, Jabber Software Foundation keeps improving this framework and producing enhancement proposals, known as JEP(s) to carry out new requirements.

The aim of this paper is to present the basic principle of XMPP and discuss its features. A few typical extensions of XMPP are covered. The paper is organized as follows. Section 2 describes the essential knowledge of XMPP and addresses fundamental actions and key elements in XMPP working process. Section 3 introduces some extensions of XMPP, which enhance the power of XMPP. Section 4 gives a short review of previous work with an overall evaluation of XMPP and summarizes the paper.

2 Basic Framework of XMPP

This section explores the essential principles in XMPP. Tech-nologies are introduced with evaluation. Core functions, se-curity issues and basic procedures are covered. Examples of the XMPP-based software are presented to illustrate typical features.

2.1 XMPP Core

The XMPP architecture is composed of three elements, XMPP client, XMPP server and gateways to foreign net-works. Fig. 1 illustrates this architecture. The XMPP server is responsible for connection management and message rout-ing. The gateway serves a bridge between different networks and has to manipulate at least two different protocols.

The underlying connection between any two XMPP el-ements is TCP. Before the content session, a TCP con-nection for XML streaming must be established first. A single TCP connection can carry multiple sessions, which are identified by their unique addresses. Similar to the addressing scheme in SMTP, the identifier for XMPP en-tity, named JID resembles the pattern of an email address followed by an URL: user@domain/resource, for example, niepin@tml.hut.fi/laptop. Application level routing policy is defined on the server side, but the specification [5] leaves the process details to the implementation. In XMPP, there is no routing discovery capability, which is specified in the SIP framework. In most cases, a direct server-to-server stream between two domains is negotiated at the initiation phase.

Three essential primitives called XML stanzas are defined in XMPP. XML stanza is the actual payload message in XML format that can be exchanged over the XML stream [1]. In addition, there are five common attributes (to, from, id, type, xml:lang) for these stanza. Three key stanzas are described below:

(2)

Figure 1: The XMPP Architecture [3]

• <message/>: it is carried out in a "store and push" mechanism through which one entity pushes informa-tion to another, for example, transferring messages be-tween two endpoints.

<body>Hello</body> </message>

• <presence/>: it is a broadcast through which multi-ple entities receive information about a given entity to which they’ve subscribed, for example, entity’s avail-ability.

• <iq/>: it is a request-response mechanism, similar to HTTP, that lets entities make requests and receive re-sponses from each other, for example: file transfer, ros-ter retrieve.

<iq type=’get’

from=’niepin@tml.hut.fi/laptop’> <query xmlns=’jabber:roster’/> </iq>

All primitive XML stanzas must reside in the <stream/> block, which stands for a XML stream. It means these XML stanzas should be treated as the content of a XML stream. All stream-level errors are considered unrecoverable and a <error/> stanza with description inside is sent by the detector. The list of error conditions is defined in the specification [5]. As a result of these characteristics, XMPP has five positive features as follows.

1. Message box: since messages are stored before being forwarded. XMPP implementations can keep messages for later delivery if the recipient is offline.

2. Service discovery: with <iq/> stanza for query, service discovery is possible. Further, DNS SRV is supported to locate network services. A detailed extension, JEP-0030 has been proposed [13].

3. Resource binding: resource location is attached in the address to distinguish different resources of a single user. There is no need for an additional protocol to han-dle this issue.

4. Internationalization: <xml:lang> common attribute en-ables internationlization.

Figure 2: Security layers for XMPP

5. Reliable communication: a TCP connection to carry out sessions offers a guarantee of the delivery, which can not be accomplished with UDP transmission.

However, due to the human-readable data format, the XMPP message size is bigger than those who use binary for-mat and leads to a heavier traffic load. Further, for some cases, TCP connection is too heavy and UDP connection produces less traffic and is more efficient, for example, ca-sual communication between two nodes.

2.2 Security Overview

Security for the communication is also specified in the core of XMPP [5]. Two mechanisms are employed to achieve confidentiality, data integrity and mutual authentication. The first protocol is the Transport Layer Security (TLS) [11]. It encrypts XML streams to ensure the confidentiality and data integrity between two entities. The second is the Simple Au-thentication and Security Layer protocol (SASL) [12], which can provide a reliable mechanism for validating the identity of an entity. Both of them are required to be supported for client-to-server and server-to-server communication. The or-der of layers for the protocols stack is showed in Fig 2.

TLS is used by XMPP to protect a channel from tamper-ing and eavesdropptamper-ing. Clients should use TLS to secure the streams prior to attempting the completion of SASL nego-tiation. It is triggered by the "STARTTLS" extension and has its own namespace "urn:ietf:params:xml:ns:xmpp-tls". SASL defines a generalized method for adding authentica-tion support to connecauthentica-tion-based protocols, and XMPP uses a generic XML namespace ("urn:ietf:params:xml:ns:xmpp-sasl") profile for SASL that conforms to the profiling re-quirements. Supported mechanisms are announced within a <mechanisms/> element. The general procedure of a suc-cessful TLS+SASL negotiation is as follows. Fig. 3 illus-trates the security setup process between the client and the server.

1. The client establishes the TCP connection to the server and initiates the XML streams.

2. The server sends a STARTTLS extension to the client, including the supported authentication mechanisms. 3. The client responds to the STARTTLS command. 4. The server informs the client that it is ok to proceed. 5. The client and the server complete the TLS setup of the

(3)

Figure 3: A successful security setup with TLS and SASL

6. Given that the success of the previous step, the client initiates a new stream to the server.

7. The server responds by sending a stream header to client along with any available stream features (SASL authentication).

8. The client picks up an appropriate authentication mech-anism.

9. The server sends a Base64-encoded challenge to the client.

10. The client responds to the challenge with the credential. 11. The server sends another challenge to the client, as the

session token.

12. The client again responds to the challenge.

13. The server informs the client of successful authentica-tion.

14. The client initiates a new stream to the server for the application-purpose communication.

In addition to these solutions applied in the core specifica-tion of XMPP [5], another specificaspecifica-tion [8] defines methods of end-to-end signing and object encryption for the XMPP. The idea is based on the XMPP-to-Common Presence and Instant Messaging (CPIM) [7] mapping, and must follow the procedure below:

1. Generate a "Message/CPIM" object.

2. Sign and/or encrypt both the headers and content of the "Message/CPIM" object.

Figure 4: A ping-pong interaction between XMPP client and server [3]

3. Put the resulting signed and/or encrypted object into an XML CDATA section contained in an <e2e/> child of a <message/> stanza.

2.3 XMPP for IM and Presence

As the name of XMPP implies, one primary purpose of the XMPP/Jabber technologies is the instant messaging and presence service. To describe extensions and the applications of the XMPP core features to this aim, another specification [6] was developed as a complement for the core document.

Fig. 4 illustrates a simple communication process at one side between the client "A" and the server. It is almost the same situation at the other side. The first two messages are to initiate a XML stream. At this step, some key parameters, such as the stream tag "id" and security mechanisms (e.g. TLS and SASL) should be negotiated between two entities. Then, the content transfer can be processed. The third mes-sage is sent by the client "A" to the remote node "B" who is in the "server.com" domain. The language is set to "en-glish". The fourth message is the response from the callee party. After the simple conversation, client "A" tears down the connection by sending a </stream:stream> to the server. The server confirms the action by replying the same XML stanza to indicate the end of the XML stream. The TCP con-nection is closed at last.

The IM session above is very simple. In reality, the scenario could be much more complicated. There are five types of instant messages in XMPP, chat (unicast), groupchat (multicast), headline (broadcast), normal and error. Every

(4)

kind is designed to fit a specific environment. Besides, the XMPP IM specifies a set of child elements (subject, body and thread) for instant messages.

Concerning presence, seven types of the state are de-fined for presence information. They are unavailable, sub-scribe, subscribed, unsubsub-scribe, unsubscribed, probe and er-ror. Presence stanza has three child elements: show, status and priority. The "show" depicts four statuses (away, chat, Do Not Disturb (dnd) and eXtended Away (xa)) that are re-flected on the IM client program. The "status" could contain any words that we set about our states, and it is showed to our friends on the contact list. Note that presence is a service that has privacy issue. It needs the entity to decide whether to subscribe it and how to apply the subscription. Some pres-ence configuration should be moved to the client side to let user do it.

The roster, meaning the contact list, can be stored at ei-ther client’s side or server’s side. Although it reduces the workload on the server if the roster is locally managed, it is recommended to store at the server side, because the roster may be lost in case the local file is destroyed or removed, for example, reinstall operating system, disk failure or accident deletion. Moreover, in the XMPP, the IQ stanza is effectively used in the roster management.

2.4 Software

According to the XMPP architecture in Fig. 1, there should be at least three kinds of software: client, server and gate-way. The first two form the base of the XMPP network. In this section, an example for each element is introduced. The criterion to choose them is based on their functionality and an "open and free" condition. Both of them provide different version for different popular operating systems (Windows, Linux and Macintosh).

The client example is the Jabber Client [15], which was developed by Jabber Community. It can enable the user talk to his friends who are using different IM systems, for exam-ple, "ICQ link", "AIM link", "Yahoo! Transport" and "MSN Transport", at the same time on a single program, depending on the capability of the connected server. Jabber Client has another browser-based version, called web messenger, which takes advantage of the HTTP protocol. One surpassing fea-ture of the browser-based web messenger is that it can still function, when normal IM clients are blocked. This situation is quite normal in many companies for security reason. Only the necessary ports are opened, while other ports are closed and no traffic is allowed. The port "80" for HTTP is usually permitted, so web messenger can work by encapsulating its traffic in HTTP packets.

The server example is the wildfire [16], which was devel-oped by JiveSoftware organization. It was called Jive Mes-senger at first. The Wildfire server was developed with Java and it applies web-based administration console. The server has cross-platform capability and can be administrated re-motely. The basic installation supports the core specification [5]. Other specifications and some extensions are sustained by several plugins of the server.

3 Extensions

The XMPP extensions are composed of two major branches. The first part is JEPs, which are produced and managed by Jabber Software Foundation (JSF). It is the biggest source of XMPP extensions and provides substantial support to the functional improvement and generic standards. The sec-ond source derives from various research work and imple-mentations, most of which focuses on the interoperability of XMPP-based applications, one key purpose of the XMPP. One example is given here. It is a XMPP-based design of IM service extended from Short Message Service (SMS).

3.1 JEPs

The JSF develops extensions to XMPP through a standard process centered on JEPs. The process is managed by the JEP editor and involves intensive discussion on the Standard-JIG mailing list, formal review and voting by the Jabber Council, and eventual inclusion in the set of approved proto-cols. A list of all JEPs and details of each proposal is avail-able on the Jabber website (www.jabber.org). So far, 175 JEPs have been published. 106 pieces of them have been approved and the rest are either rejected or obsolete.

Apart from rejected, deprecated and obsolete JEPs, there are five kinds of status for every approved enhancement to the Jabber/XMPP protocols as well as proposals that are un-der active consiun-deration. They are listed below.

1. Final

The protocol defined may be considered as a stable technology for implementation and deployment. 2. Draft

The protocol defined is encouraged and is appropriate for deployment in production system, but some changes are possible before it becomes a Final Standard. 3. Experimental

No approval guarantee for this JEP. It is only encour-aged in exploratory implementations. Production de-ployment should not be done until it advances to a status of Draft.

4. Proposed

This JEP is currently within Last Call or under consid-eration by the Jabber Council for advancement to the next stage in the JSF standards process.

5. Active

The process or activity defined has been approved. It is currently working and will do so until this JEP is dep-recated or obsolete.

In addition, there are five types of JEP(s), i.e. Historical, Humorous, Informational, Procedural and Standards Track. Among them, Informational and Standards Track are impor-tant for XMPP application developers. Historical and Pro-cedural are related with JSF community, other than XMPP extension protocols.

Five important JEPs have been finalized. They are Data Forms (JEP-0004), Jabber-RPC (JEP-0009), In-Band Reg-istration (JEP-0077), Non-SASL Authentication (JEP-0078)

(5)

and Service Discovery (JEP-0030). Besides, there are 31 draft JEPs, 31 experimental JEPs, 36 active JEPs and 3 pro-posed JEPs.

Thanks to these JEPs, many new functions and improve-ments have been added in some XMPP-based programs and software, for example, personalized avatar, file transfer and gateway interaction. Consequently, XMPP applications are becoming more powerful and popular. Note that standard er-ror responding procedure was also specified in the core and extension protocols. It can make sure the clients that does not support some extension would not behave incorrectly, when communicating with those extension-enabled clients.

3.2 Extension on Interoperability

In order to enable the XMPP-based applications to work with other protocols or networks, extensions must be provided to coordinate different protocols and connect them seamlessly. Effort has been made on this area. The XMPP gateway to commercial protocols, such as MSN, Yahoo Messenger and AIM, are available. Here a new design to combine XMPP-based IM with Short Message Service (SMS) is offered for analysis.

Similar to the IM service, SMS is another popular instant text communication. But unlike IM working in IP-based net-works, SMS serves in mobile netnet-works, e.g. GSM. To bridge the two is becoming a new promising niche because people could then enjoy the instant message service anywhere at any time. Now there is a common, unified and open standard available [2]. The infrastructure, based on XMPP is able to interconnect IM systems and SMS systems.

From the architecture showed in the Fig. 1, XMPP client can talk with an external node through the XMPP server that connects to the foreign network by the help of the gateway. In this design, the gateway acts as a translator and bridge. The underlying protocol in SMS system is Short Message Peer-to-Peer (SMPP). It is in binary format, different from the human readable text in XMPP. The gateway must be capable to perform two basic missions. One is the conver-sion between XML packets and SMPP Protocol Data Units (PDU), and the other is the session management. The first requirement includes both XML packet parsing and SMPP PDU wrapping. The second one is for keeping track of short messages within an IM session, because the SMPP does not provide session management.

In addition, there should be some way to ensure in-order delivery. Based on the analysis above, a gateway design between IM systems and SMS systems was proposed [2]. Fig. 5 shows the logic structure of the XMPP server and Fig. 6 depicts the XMPP gateway. In order to satisfy the second requirement, that is, session management, the session module of the XMPP gateway creates a pseudo XMPP client. It acts for the corresponding SMS user to execute functions, such as login, data transmission and so on. The queue mech-anism and store-and-forward mode enable orderly delivery of messages. In reality, unexpected failures, e.g. unreach-able or power off, may occur on a mobile handset from time to time. Retrial and delivery need a confirmation from the recipient as the guarantee. To solve this problem, a feature of SMPP, delivery receipt, is utilized to indicate the final

de-Figure 5: Basic functional modules of the XMPP server [2]

Figure 6: Basic functional modules of the XMPP gateway [2]

livery status for each message.

The philosophy of this design can be extended to bridge XMPP system and other foreign networks, for example, pro-prietary protocols. The details depend on the attributes of the non-XMPP protocols and the interface provided by the protocol owner.

4 Conclusion

This paper presented the XMPP framework and discussed both the specifications in the IETF and extensions in the JSF community and wild world. Advantages of the XMPP proto-cols were mentioned, along with the corresponding features. They explain why the XMPP is designed in this way and how it fits the requirements of the users. Examples of appli-cation programs were also given to gain a direct view of the XMPP-based software and its current development.

The XMPP/Jabber technology has achieved big success, especially since the IETF approval of the core protocols. It led to significant implementations, major deployments and renewed activity by open-source projects and commercial software developers. New features and capabilities are being built into the framework. Nevertheless, the XMPP/Jabber still has to face challenge from competing technologies, such as SIMPLE stated above. In addition, to interoperate with other IM systems requires more effort on the gateway devel-opment. Compared to commercial products and enterprise applications, several topics, for example, quality of service, value-added services and multimedia support demands fur-ther work. Thanks to the extensibility of XML, the above

(6)

issues can be considered and solved by adding new elements to realize desired rules in the future, with backward compat-ibility.

References

[1] Peter Saint-Andre. Streaming XML with Jab-ber/XMPP. In Proc. 2005 IEEE Internet Computing, Sep 2005, IEEE Computer Society Press.

[2] Heng-Te Chu, Wen-Shiung Chen, Yi-Hung Huang and Jeng-Yueng Chen. A novel design of instant messag-ing service extended from short message service with XMPP. Fifth IEE International Conference on 3G

Mo-bile Communication Technologies, 2004

Page(s):504-508, IEEE Computer Society Press.

[3] Mikko Laukkanen. Extensible Messag-ing and Presence Protocol (XMPP). At

https://www.cs.helsinki.fi/u/kraatika/Courses/IPsem04s/slides/xmpp.pdf

University of Helsinki, Department of Computer Sci-ence, 2004.

[4] Simone Leggio. SIP for Instant Messag-ing and Presence LeveragMessag-ing Extensions. At

https://www.cs.helsinki.fi/u/kraatika/Courses/IPsem04s/SIMPLE_Leggio.pdf

University of Helsinki, Department of Computer Sci-ence, 2004.

[5] Peter Saint-Andre. Extensible Messaging and Presence Protocol (XMPP): Core. IETF RFC 3920, Oct 2004. [6] Peter Saint-Andre. Extensible Messaging and

ence Protocol (XMPP): Instant Messaging and Pres-ence. IETF RFC 3921, Oct 2004.

[7] Peter Saint-Andre. Mapping the Extensible Messaging and Presence Protocol (XMPP) to Common Presence and Instant Messaging (CPIM). IETF RFC 3922, Oct 2004.

[8] Peter Saint-Andre. End-to-End Signing and Object En-cryption for the Extensible Messaging and Presence Protocol (XMPP). IETF RFC 3923, Oct 2004. [9] B. Campbell, J. Rosenberg, H. Schulzrinne, C.

Huitema and D. Gurle. Session Initiation Protocol (SIP) Extension for Instant Messaging. IETF RFC 3428, Dec 2002.

[10] J. Rosenberg, G. Camarillo, A. Johnston, H. Schulzrinne, J. Peterson, R. Sparks, M. Handley and E. Schooler. Session Initiation Protocol (SIP). IETF RFC 3261, June 2002.

[11] T. Dierks and C. Allen. The TLS Protocol Version 1.0. IETF RFC 2246, January 1999.

[12] J. Myers. Simple Authentication and Security Layer (SASL). IETF RFC 2222, October 1997.

[13] Joe Hildebrand, Peter Millard, Ryan Eatmon and Peter Saint-Andre. JEP-0030: Service Discovery. At

http://www.jabber.org/jeps/jep-0030.html.

[14] Robert Sparks and Hisham Khartabil. SIP for Instant Messaging and Pres-ence Leveraging Extensions (simple). At

http://www.ietf.org/html.charters/simple-charter.html.

[15] Jabber Community. Jabber Client At

http://jajc.ksn.ru/download.php?id=4.

[16] JiveSoftware Organization. XMPP server wildfire. At