The Motivation

The Cosmo UI is a rich web-based front end for the OSAF's Cosmo calendar server. Comet can be described as something of an iteration on Ajax. Comet involves pushing data from the server to the browser, without the need for a direct request from the client. This facilitates a more active user experience, as events can be received in the client as they occur, rather than waiting for the server to be polled.

The primary purpose of this project is to integrate Comet services into the Cosmo server and UI, allowing live calendar events to be pushed back to the browser. As a user is inspecting a calendar, they should be able to see events that are being added, removed or updated, as those events are occurring. Even under Ajax, the ability to realize this behaviour is limited, necessitating periodic polling. If the polling interval is too small, large amounts of unnecessary traffic are generated. If the polling interval is too long, the user may be presented with (and potentially be working on) out-of-date data.

As useful as Comet style services are in enabling a two-way conversation between client and server, there are a number of scalability concerns. The primary cause of these is the number of long-lived HTTP connections that must remain alive. Many traditional Servlet environments are not equipped to deal with connections that are maintained beyond a simple request and response. Comet is a young technology, and the infrastructure necessary to support such enhanced communications is still making its way into production environments.

Current solutions either necessitate the use of non-blocking I/O in the server (support that is not yet universal to all Servlet contains or App Servers), or the use of a third-party process that is specifically designed to address the Comet model.

Technical Problems

There are a number of technical problems that must be overcome as part of the process of Comet-enabling Cosmo.

Server Infrastructure

The Comet communications pattern requires special consideration on the part of the Web Server in order to deal appropriate with the rather long-lived connections that exist between the client and the browser. There are two ways to approach this problem. The first is to use some sort of "snap-on" component for the web service (Twisted[1], or POE[2] seem to be the popular ones). The main problem here is that they require installation and configuration to use, making the deployment process unnecessarily complicated. Additionally, this approach restricts options in shared or restricted hosting environments.

The second approach is to use facilities built into the server or servlet container by default. Not all containers currently support the Comet communications model, so while this approach may be neater, it is also less portable. However, Tomcat (the default Cosmo container) has recently added support for Comet-style services.

Until there is a standard API that Servlet container implementors can provide, the portability of any Comet-based features will be limited. On the flip side, using an external component necessitates additional setup on the part of a user (creating an extra point of failure and necessitating further support). This really is a lose-lose situation. The way I see it, using the Tomcat features is the best approach. Although it won't be portable, the fact that Snarf uses Tomcat probably means that a large number of Cosmo users are also on the same platform.

Gathering Notifications

The first Cosmo-related task will be to add code to the framework that allows for the gathering and distribution of relevant event notifications. The expectation with Comet is that events happening in the server (such as the addition of a new appointment, or the alteration of an existing one) will be captured and pushed to any relevant clients asynchronously. Capturing these events is the first step.

There is a high-level notifications framework going into the client side [3], however, there is no such server-side facility. The current approach is to explicitly do relevant notifications in code. The snippet below is taken from a conversation I had with Bobby:

[3:35am] <bobbyrullo> we dont' really have notifications like you are hoping though
[3:35am] <bobbyrullo> I think the code looks like:
[3:35am] <bobbyrullo> createUser() { createtheUser(); sendAnEmail()}

Although I could go through all the relevant code and put in additional notifications for the appropriate events, that doesn't help when new events need to be added later. A much better approach would be a generic notifications framework for the server-side, similar in intent to the client-side one being implemented [3].

Distributing Notification

Continuing on from the previous task, as events are captured in the server, they need to be pushed out to the relevant clients. The biggest problem here is identifying which clients are interested in which data. Pushing any notification to all Comet-connected clients is a tad inefficient. There needs to be some way to filter out notifications such that the server only pushes to the client information about events it is currently concerned with.

Cosmo Web UI Enhancements

The final problem involves the client side. Once a notification has been captured and pushed to the client, the relevant view needs to be updated. My initial idea is that the high-level notification framework [3] could be used to trigger updates. However, I haven't looked much into this at all yet.

Proposed Deliverables and Target Dates

In order to stage development such that progress is incremental, I would suggest splitting the project into the following deliverables. All dates are guesstimates.

(1) Simple Comet Example Finish: 25th June

Although I have a lot of Java experience, my webapp development experience is next to nothing. Making that problem a little bit bigger is the fact that it isn't all just Java stuff. This is the first time I've used Javascript in several years (and even back then I only used it VERY briefly). As such, I have struggled through ignorance of the area and don't really have anything more than a basic understanding of how things fit together. I'd find it difficult to start work on the real deliverables because I would not even know where to get started.

As such, I felt that I would benefit from writing a very simple Cosmo service (client and server) just to get started. This helps me figure out what I'm doing and also gives me a reference point that is simple and to the point (rather than trying to integrate it with the large Cosmo codebase right off the bat).

(2) Simple Server Logger Finish: 9th July

This involves the implementation of the server-side notification gathering mechanism described above. For the time being, messages would just be pumped out to a log file or the like. Once this is complete, sourcing notifications won't be a concern.

(3) Client Notification Viewer Finish: 30th July

Extending on the previous deliverable, this will be a basic web-client that just lists logged notifications. Think of it like a Comet-enabled log viewer. This step will allow me to get right the necessary filtering and the semantics needed to both asynchronously push events to the client and read them there. Just like #1, getting the basics right before integrating it into the large library for the Cosmo Web UI first allows me to ensure I have the communications working properly before dealing with the integration.

(4) Integration into Client Finish: 20th August

Once I have data being successfully filtered and pushed to a client, the final step will be to link up those incoming messages with the appropriate notifications on the client side. This is the bit that allows the web UI to perform the updates as a result of incoming notifications.

• [1] http://twistedmatrix.com/trac/wiki/Documentation
• [2] http://poe.perl.org/?What_POE_Is
• [3] http://wiki.osafoundation.org/Projects/HighLevelCosmoUIEventsProposal