A fast-moving Comet is about to impact the Internet. When it hits, it will wipe away the architecture flaws we have lived with for the past 15 years and allow a new World Wide Web to evolve.
This new Web will include applications that are instantly on and always on, applications that are truly multi-user, and applications that go far beyond today’s “click and wait” Web solutions.
Brace for Comet Impact
Comet (or Reverse AJAX) introduces a significant departure from the stale “click-and-wait” interaction we traditionally associate with Web applications, and resurrects push-style communications – the 1990s technology that was long before its time.
Comet introduces a more scalable, agile, and broadly supported approach to mixing push capabilities with the traditional REST-based communications model of the Web, one that also addresses the limitations that made the initial inception of “push” short-lived.
Simply put, push communications like Comet remove the requirement for an end user to explicitly click on a link or button to request information from a server. Instead, the server is free to send messages to clients anytime a server-side event occurs. Therefore, content no longer needs the tried and true “click” to reach you. Web pages from hereon out will simply update as new data is made available, finally laying to rest the refresh button on your browser.
Comet makes it possible to build any application that requires real-time updates, enabling true desktop-like functionality to be delivered over the Internet. Example rich Internet applications (RIA) include, but are certainly not limited to: chat applications to exchange messages on social networks; online games; stock prices from online trading platforms; tools for online collaboration; betting odds for gambling sites; news feeds; and results from sporting events.
Imagine a more interactive eBay that actually shows auction updates as bids occur. This would enable users to bid on products with the most up-to-date information, rather than having to place a bid based on stale information, only to find that you have been outbid. Financial Web sites such as E*TRADE or Yahoo Finance will also benefit from Comet by providing a scalable means for delivering real-time quotes to millions of customers.
Comet also brings true “instant play” to the world of online games, enabling gaming sites to deliver multi-player games via the browser rather than requiring a lengthy download and installation process to install clunky desktop software.
With its ability to turn the Web into an event-based medium, Comet brings the promise of a Web that is always on and always adapting, a Web that is no longer bound to page views, but dedicated to streaming continuous content that is ever more targeted and personalized, and at any time.
To quote Douglas Crockford, Yahoo, at AJAXWorld East 2008, “Comet is a correction of the initial architecture of the Web.” Comet provides us with a true event-driven model for building next-generation Web applications.
Traditional Web application dinosaurs are headed for extinction.
Comet Ripple Effects
The Comet model for communications is a drastic departure from that found in the classical Web model in which events are client-initiated. The most obvious benefit of Comet’s model is the server’s ability to send information to the browser without prompting from a client. Comet-style communication also differs from that found in the traditional AJAX model, in which browser clients poll at regular predefined intervals for data – also known as Time to Refresh (TTR).
In most cases, polling schemes blindly check for updates irrespective of state changes in the application. The result is poor resource utilization on both the client and server, since CPU-cycles and memory are needlessly allocated to prematurely or belatedly detect updates on the server. Furthermore, high polling frequencies result in increased network traffic and server demands, while low polling frequencies result in missed updates and the delivery of stale information. So achieving a refresh rate that approximates real-time updates is difficult without flooding the network.
How Comet Works
Comet applications create long-lived HTTP connections between the client and server. These connections eliminate the need for clients to poll the server and allow servers to stream messages to clients as new data becomes available.
HTTP does not provide a scheme for a server to open a connection to a browser; therefore, the creation of a long-lived connection is client-initiated.
From the client’s perspective, the data is arriving in chunks over the lifetime of the HTTP response. The client can process each chunk as it arrives, rather than wait for the entire HTTP response to complete.
The net effect of streaming data over an open connection is a reduction in the latency between the time an event occurs and a message is delivered, enabling application state to update in real-time. In addition, the overhead of HTTP headers and opening and closing connections (as seen in polling) is greatly reduced.
Some Comet implementations provide a publish-and-subscribe programming paradigm, which allows clients to subscribe or express interest in a certain type of information – commonly known as a topic or channel.
Supported Technologies
Comet is not bound to any particular set of client technologies, but refers instead to a group of techniques that facilitate bi-directional, low-latency communication between clients and servers. However Comet, like traditional
A variety of techniques exist for managing connections to a Comet server from a browser endpoint. Each technique has its own characteristics in terms of browser support as well as latency and throughput.
For example, some Comet clients may attempt to use plug-ins such as Flash or Java to open socket connections instead of using methods native to the browser. Other clients rely on more traditional approaches, using native JavaScript to opening connections via iFrames, ActiveX, or XmlHttpRequest (XHR) objects. Either connection scheme provides equivalent methods of streaming and/or efficiently polling for information.
Browser developers are also aware of the growing demand for real-time Comet streaming support and, as such, the upcoming proposals for HTML include the concept of Server-sent Events (SSE). SSE will allow clients and servers to standardize the protocols and APIs for making long-lived Comet connections from browser to server, which should usher in a new area of interoperability and support for real-time browser messaging.
Like Comet clients, Comet servers do not mandate the use of a specific technology. However, some technologies are better suited for handling Comet connections than others.
Performance and Scalability Considerations
Despite their promise, long-lived connections do pose a problem for server performance. For this reason, until now most Comet deployments today have been limited to small- or mid-sized audiences. In general, Comet servers maintain some state and, therefore, must store information about connected clients and the resource allocated to each connection. These additional demands for resources limit the number of concurrent connections that a server can maintain.
Long-Polling and Streaming
Some Comet vendors attempt to offset these costs by implementing efficient polling strategies such as long-polling. This method of polling, also known as Asynchronous-Polling, allows a long-lived connection to be maintained between the client and server for a set timeout interval, after which the server terminates the HTTP response and forces the client to reconnect. If the server sends a message before timeout, the server still terminates the HTTP response, once again forcing the client to reconnect to the server.
Messages are buffered at the server until the client reconnects, when they are immediately delivered in a complete HTTP response to the client. Therefore, in high message throughput environments, the long-polling technique degrades to traditional polling, with no wait on the server.
The long-polling strategy helps prevent starvation of resources on the server, but fails to deliver the minimum latency guaranteed by streaming data over a long-lived connection. In long-polling over HTTP/1.0, a single response is sent over each open connection before the connection is torn down and reopened. In HTTP/1.1 with keep-alive, the underlying HTTP persistent connection can be reused by the browser, but HTTP response headers accompany each message delivery to the browser.
Streaming data optimizes network traffic by eliminating the need to send redundant HTTP response headers. Streaming data over an open connection that is initialized only once incurs only the cost of sending response headers on the setup of the connection. Subsequent messages are sent over the connection without the response header, reducing the network bandwidth required to deliver the same messages, when compared with long-polling.
Few Comet implementations provide the scalability to enable streaming and instead rely on long-polling as a means to serve additional clients. Kaazing’s Enterprise Comet represents one of the few Comet offerings that can scale to meet the intensive requirements of streaming data.
Measuring Concurrency
When developers start implementing Comet functionality into their Web applications, there is a need to be cautious and alert about changes in user behavior and how it might impact the server architecture. Today, when developers and IT managers track concurrency the numbers are usually in the low hundreds, which is still a considerable number when addressing a short-lived request and response cycle. With a Comet solution concurrency takes on a new meaning because everyone is connected to the server for lengthy periods of time.
Developers and IT managers will have to confirm the size of their user community and research their user-base’s behavior. How large is it – 200,000 – 2,000,000 – 20,000,000 users? And what happens if these clients stay connected for 30 seconds, 90 seconds, 30 minutes, or 90 minutes?
When concurrency suddenly skyrockets with the usage of Comet, what options do we have to address this scalability issue?
Vertical Scalability
There are three main areas that need to be addressed to scale a Comet solution to support a production enterprise environment: vertical scalability, horizontal scalability, and message distribution.
Historically, Web servers like those offered by Apache and Microsoft have been the main solutions for delivering content over the Web. However, these servers are not designed to handle long-lived HTTP responses, but are designed to serve requests as quickly as possible, complete the response, and move on to the next request. While a long-lived connection may not be a problem in itself, traditional Web servers will not scale when serving long-lived connections to large audiences, creating demand for reinvention of the traditional Web server.
Traditional threading models use a synchronous, blocking thread model, in which each connection served requires a dedicated thread. This results in rapid degradation of server performance as the number of concurrent users increases.
To vertically scale, servers must have the ability to process asynchronous requests. Asynchronous request processing (ARP) allows servers to handle higher volumes of concurrent sessions – a characteristic of both AJAX and Comet applications – with the least number of threads possible. Twisted Python and Java Native IO (NIO) are two examples of technologies that support ARP. Non-blocking technologies such as ARP are able to release resources while waiting for another event, allowing a single thread to serve multiple connections. Therefore, the number of clients served is no longer directly proportional to the number of threads a machine can handle. Comet servers like Enterprise Comet provide vertical scalability by supporting asynchronous requests.
Horizontal Scalability
Few Comet servers provide a means beyond long-polling and ARP to cope with high volumes of concurrent users. Neither approach used in isolation or combination completely addresses the substantive hardware requirements necessary to serve many thousands of concurrent users.
Clustering schemes are also employed by application servers to meet high demands for resources. Clustering allows clients to be served by a pool of servers running on separate machines, thus distributing resource allocation. Comet clusters generally fall into two categories – embedded and distributed – that are primarily a result of the server’s integration point with the message origin, i.e., the data source.
In an embedded architecture – a Comet server running in an application server – clients are pinned directly to a specific application server instance in the cluster, and shared data is forced to be replicated among all the nodes in the cluster. This, in turn, requires high volumes of network traffic overhead.
In addition, Comet integration at the application server level lacks the benefit of separating the servicing of long-lived connections from the execution of time-sensitive application logic. Sharing a single resource to serve both of these concerns can introduce a point of contention for applications that need to respond to tens of thousands of requests within sub-second intervals.
To address this issue, some Comet implementations sit alongside an application server in a so-called distributed architecture. These implementations typically rely on traditional HTTP load balancing with Comet session affinity for clustering. This form of clustering works well to offset connection loads, but still requires a significant amount of state management in terms of message queues and client subscriptions.
Enterprise Comet enables subscription groups to be partitioned at the application server independent of long-lived connections. This leads to significant scalability benefits, especially when the frequency of publishing messages far exceeds the frequency of subscribing to groups.
Enterprise Comet does not store message queues within the confines of the application server, but instead distributes message queues across a delivery network. Each node on the network maintains a queue of messages for each browser connected to that particular node. This distribution balances the resources for state management across a larger set of resources, enabling Enterprise Comet to serve more long-lived connections and allowing it to rely on streaming as a primary method of message delivery.
The careful distribution of message queues also enables better fault tolerance, freeing up more nodes to serve orphaned clients in the case of a node failure.
Message Delivery
Volatile data can easily cause message storms, and with thousands of concurrent connections, CPU and bandwidth utilization will suffer.
In most Comet implementations every message sent has to be copied across multiple connections, possible thousands of connections. Just imagine sending one message to 100,000 users, and repeat that every 50ms!
Reliable Message Delivery
Enterprise Comet extends enterprise-messaging services to the browsers. As such, Enterprise Comet must ensure the reliable delivery of messages to clients no matter their location. Therefore, the server provides message sequencing to Comet, allowing clients to maintain a record of the last message delivered and enabling the client to re-query for messages that may have been lost in transit.
Since message queues are distributed and replicated across the Enterprise Comet network, buffers are highly fault tolerant and primed for resends in the event of a failure.
Fair Message Delivery
Another feature unique to Enterprise Comet is the introduction of fair messaging. Fair messaging enables administrators to create policies that govern the timeliness of message delivery for all clients connected to an application. All connected clients send information about the quality of their connection to the Enterprise Comet node they are served by. The aggregation of this information is used by Enterprise Comet to interpret the time to deliver a message to any client in the network. This information can then be used by administrators to govern access requirements to an application by bandwidth or time the delivery of information to a diverse set of clients connected at various rates.
Group Message Delivery
Few Comet servers provide a scalable means to broadcast information to large groups of users, e.g., channels of subscribers. This is largely due to the servers’ need to send multiple copies of a single message rather than sending a single message that is routed to multiple destinations, which in turn avoids otherwise high CPU loads.
Enterprise Comet introduces the concept of multi-way routing over HTTP to overcome redundant message sends. This strategy for message delivery enables the server to send a message over a single connection to multiple destinations without sending copies of the message over multiple connections. Enterprise Comet does this by fanning out message delivery across a network of Enterprise Comet nodes, where each point in the delivery network serves a subset of the connections served by an application.
The benefit of distributing message delivery this way extends beyond eliminating message copying to facilitating greater distribution of connections across machines. This, in turn, allows more finely tuned load balancing, resulting in better overall application performance and the ability to serve tens of thousands of clients.
Global Message Delivery
Group Messaging and multi-way routing are not only limited to the internal networks of the enterprise, but also include the delivery of messaging services to customers across the globe. Enterprise Comet’s ability to fan out messages efficiently to clients connected across its delivery network allows high-performance messaging to any client connected to the network’s edge – a place once limited to the static caching of content can now dynamically stream data with the accuracy of real time (see Figure 1).
Summary
Comet has laid the foundation for the future of Web applications and dethroned the stateless “click-and-wait” Web dinosaurs. By using Comet, developers can now build event-driven solutions deployed for the Web without needing plug-ins or downloads.