Notifications Architecture Proposal: Collections and Items

Notification Architecture Breakdown

Implementing the Notification Proposal will require the following additions to the current Cosmo Architecture.

• Adding the ability for a user to specify his or her timezone during the sign up process and the ability to change that timezone in the Account Preferences.

• Creating two new Atom projection URI's to support notification changes and forward notification queries.

• Augmentations to the Service layer to
  • based on a time range return all items in a collection that have a start time (events) or reminder time (alarms) that falls with in the range.
  • based on a time range query for a collection return notification objects that represent the changes per item with in the range.
  • Record in real time changes made to an item or a collection (via a Hibernate / Spring abstraction) to modification database tables including the previous and current values.

• Expanding the User Preferences logic to store settings for push based listeners (EMAIL, SMS, XMPP) including which collections to push, the type of notifications (forward or modifications), and how often.

• Incorporating a Job Scheduler in to the Cosmo Architecture. The Scheduler will have jobs that awake at specific intervals, determine which users want notifications, query the Service layer, and pass the results to registered listeners (EMAIL, SMS, XMPP).

• Adding new tables to the Cosmo schema and wrapping the database insertion and query logic in a Hibernate / Spring abstraction layer. The tables would persist changes to items / collections as well as additions and removals. The appropriate table columns would be indexed for fast query of modifications made in a given time range.

Cosmo Atom Publishing Protocol Additions

       http://localhost:8080/chandler/atom/collection/446aad0c-efab-11dc-eb2a-0017f2ca5c2b/?ticket=qjoom1jdd0&start=2007-10-29T18%3A05%3A23Z&end=2009-3-3T18%3A05%3A23Z
    

The start and end date query parameters are used with or without a timezone to determine what events occur with in the time range. This logic can be leveraged quite nicely for notifications as well. Specifically, all notifications happen with in a time period which is known to the requestor (Web Widgets). For example, "Show me all the changes / modifications on items in collection 'foo' that happened in the last three hours".

Two new Atom projection types are needed to support the notification logic.

Changed

The 'changed' projection type will query the Cosmo Service for all items in the collection that have been modified between the time range and return an Atom feed of entries containing diffs as XHTML in the content attributes. The start and end query parameters are required for this projection type. Each entry will have a link to the item it references. A possible exchange might look like this:

<<< REQUEST >>>
GET /chandler/atom/collection/96e0364e-db7d-4250-98a2-99ef4f402c0b/changed/?ticket=qjoom1jdd0&start=2008-3-3T15%3A05%3A23Z&end=2008-3-3T18%3A05%3A23Z HTTP/1.1

<<< RESPONSE >>>
HTTP/1.1 200 OK
ETag: "qC34DI11NJLl9/lCZRf+rlZBy04="
Last-Modified: Thu, 12 Jul 2007 18:53:58 GMT
Content-Type: application/atom+xml

<?xml version='1.0' encoding='UTF-8'?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:xml="http://www.w3.org/XML/1998/namespace" xml:base="https://hub.chandlerproject.org:443/atom/">
  <id>urn:uuid:96e0364e-db7d-4250-98a2-99ef4f402c0b</id>
  <title type="text">foo</title>
  <updated>2008-03-02T13:39:20.736Z</updated>
  <generator uri="http://cosmo.osafoundation.org/" version="0.12.0">Chandler Server</generator>
  <author>
    <name>briankirsch</name>
    <uri>user/briankirsch</uri>
  </author>
  <link rel="self" type="application/atom+xml" href="collection/96e0364e-db7d-4250-98a2-99ef4f402c0b/full/eim-json?ticket=qjoom1jdd0" />
  <link rel="edit" type="application/xhtml+xml" href="collection/96e0364e-db7d-4250-98a2-99ef4f402c0b" />
  <entry xmlns:app="http://www.w3.org/2007/app">
    <id>urn:uuid:87473b70-c154-11dc-f901-cdc1f4805</id>
    <title type="text">New Item Title</title>
    <updated>2008-03-02T13:39:20.715Z</updated>
    <app:edited>2008-03-02T13:39:20.715Z</app:edited>
    <published>2008-01-12T21:26:35.315Z</published>
    <content type="text/xhtml">  
      <div class="changedprops">
        <div class="prop">
        <div class="id">title</div>
        <div class="oldvalue">Old Item Title</div>
        <div class="newvalue">New Item Title</div>
      </div>
      <div class="changedstamps">
        <div class="stamp">
          <div class="id">EventStamp</div>
          <div class="action">REMOVED</div>
      </div>
    </content>
    <link rel="self" type="application/atom+xml" href="item/87473b70-c154-11dc-f901-cdc1f4805/full/eim-json?ticket=qjoom1jdd0" />
   </entry>
  </feed>

As with all other projection types recurring events will also include additional links to the master event and other information.

An additional option could be to include a human readable description in a summary element of the entry. for example:

<summary type="text/plain">
gr24 changed the start time of Janitor's Pipe Dream from 3/12/2008 3:00PM to 4:00PM PST
</summary>

Range

This leaves two options:

1. Augment the current projection logic to also return items with a reminderTime in the time range.
2. Create a new projection type which I will call 'range' for a lack of a better term at the moment.

Like the 'changed' projection type 'range' would require start and end time range query parameters.

A possible exchange might look like this:

<<< REQUEST >>>
GET /chandler/atom/collection/96e0364e-db7d-4250-98a2-99ef4f402c0b/range/?ticket=qjoom1jdd0&start=2008-3-3T15%3A05%3A23Z&end=2008-3-3T18%3A05%3A23Z HTTP/1.1

<<< RESPONSE >>>
HTTP/1.1 200 OK
ETag: "qC34DI11NJLl9/lCZRf+rlZBy04="
Last-Modified: Thu, 12 Jul 2007 18:53:58 GMT
Content-Type: application/atom+xml

?xml version='1.0' encoding='UTF-8'?>
<feed xmlns="http://www.w3.org/2005/Atom" xmlns:xml="http://www.w3.org/XML/1998/namespace" xml:base="http://localhost:8080/chandler/atom/">
  <id>urn:uuid:446aad0c-efab-11dc-eb2a-0017f2ca5c2b</id>
  <title type="text">Test</title>
  <updated>2008-03-11T21:04:13.039Z</updated>
  <generator uri="http://cosmo.osafoundation.org/" version="0.13-SNAPSHOT">Chandler Server</generator>
  <author>
    <name>TestUser</name>
    <uri>user/TestUser</uri>
  </author>
  <link rel="self" type="application/atom+xml" href="collection/446aad0c-efab-11dc-eb2a-0017f2ca5c2b/full/eim-json?ticket=qjoom1jdd0" />
  <link rel="edit" type="application/xhtml+xml" href="collection/446aad0c-efab-11dc-eb2a-0017f2ca5c2b?ticket=qjoom1jdd0" />
  <entry xmlns:app="http://www.w3.org/2007/app">
    <id>urn:uuid:be558000-efac-11dc-ae7d-acc9a4f410b3</id>
    <title type="text">test 2</title>
    <updated>2008-03-11T20:50:35.000Z</updated>
    <app:edited>2008-03-11T20:50:35.000Z</app:edited>
    <published>2008-03-11T20:50:12.414Z</published>
    <content type="text/plain">{"uuid":"be558000-efac-11dc-ae7d-acc9a4f410b3","records":{"item":{"ns":"http://osafoundation.org/eim/item/0","key":{"uuid":["text","be558000-efac-11dc-ae7d-acc9a4f410b3"]},"fields":{"title":["text","test 2"],"triage":["text","200 0.00 1"],"hasBeenSent":["integer","0"],"needsReply":["integer","0"],"createdOn":["decimal","1205268612"]}},"modby":{"ns":"http://osafoundation.org/eim/modifiedBy/0","key":{"uuid":["text","be558000-efac-11dc-ae7d-acc9a4f410b3"],"userid":["text","TestUser"],"timestamp":["decimal","1205268634"],"action":["integer","100"]}},"note":{"ns":"http://osafoundation.org/eim/note/0","key":{"uuid":["text","be558000-efac-11dc-ae7d-acc9a4f410b3"]},"fields":{"body":["clob",""],"icalUid":["text","be558000-efac-11dc-ae7d-acc9a4f410b3"]}},"event":{"ns":"http://osafoundation.org/eim/event/0","key":{"uuid":["text","be558000-efac-11dc-ae7d-acc9a4f410b3"]},"fields":{"dtstart":["text",";VALUE=DATE:20080313"],"duration":["text","P1D"],"location":["text",""],"rrule":["text",null],"exrule":["text",null],"rdate":["text",null],"exdate":["text",null],"status":["text","CONFIRMED"]}},"displayAlarm":{"ns":"http://osafoundation.org/eim/displayAlarm/0","key":{"uuid":["text","be558000-efac-11dc-ae7d-acc9a4f410b3"]},"fields":{"description":["text",null],"trigger":["text",null],"duration":["text",null],"repeat":["integer",null]}}}}</content>
    <link rel="self" type="application/atom+xml" href="item/be558000-efac-11dc-ae7d-acc9a4f410b3/full/eim-json?ticket=qjoom1jdd0" />
    <link rel="edit" type="application/atom+xml" href="item/be558000-efac-11dc-ae7d-acc9a4f410b3?ticket=qjoom1jdd0" />
  </entry>
</feed>

Comments

As with the current Cosmo Web UI, all Widget clients are responsible for managing and developing their own localizations based on the data returned.

Cosmo Service Layer

StandardContentService^?

* findItemsInRange: will find all items with a start time or reminder time (alarm) that falls with in the specific rangeStart and rangeEnd.

   public Set<ContentItem> findItemsInRange(CollectionItem collection, DateTime rangeStart, DateTime rangeEnd, boolean expandRecurringEvents)  

• findModifiedItems: Returns a Set of ItemModification^? objects for the given collection that falls with in the specific rangeStart and rangeEnd. An ItemModification^? will be a new Java type that will contain a Pointer (uuid) to or instance of the item as well as a modification log (list). Each entry in the log will include the attribute changed and the old and new value. Modifications are transactional in that one user creates one or more modifications (changed start time, edited title etc) on an item per ItemModification^? object. Each entry in the ItemModification^? Set will contain all the item diff information needed by the caller to the format the appropriate output (Atom, EMAIL, SMS, and IM).

   public Set<ItemModification> findModifiedItems(CollectionItem collection, DateTime rangeStart, DateTime rangeEnd, boolean expandRecurringEvents) 

Modification Persistence

• The StandardContentService^? methods such as updateItem and createCollection can be augmented to also record the modification data via Hibernate / Spring to the persistence (database) layer.

• A new listener class that handles the persistence of modifications will register with the Spring Framework to tie in to the fireBefore event. For each relevant action such as updateItem, the listener will record the modification data to the persistence layer.

User Preferences

The current User Preferences model of key value pairs is probably a bit to rudimentary too store what types of notifications to send (forward / modification), on which collections, at what intervals, and to what type of clients (EMAIL, SMS, IM). This type of data can be modeled with key value pairs but it is perhaps better served with a richer data structure.

Any Web Widget can query for forward / modification notifications with a valid ticket or username and password. Thus no Web Widget notification information needs to be stored in the User Preferences.

For each Push based notification mechanism the type of notifications and the interval need to be persisted. There will also be custom data for each push mechanism such as an email address or XMPP userid.

Notifications Manager Job Scheduler

The Notification Scheduler architecture is the most challenging aspect of the proposal since there are many to many relationships established between the the type of notifications and the consumers of those notifications (EMAIL, SMS, IM). As such there are number of ways that the infrastructure can be implemented while being careful not to over-architect the framework.

The best choice for a Java based Job Scheduler is the Quartz Timer package. It has robust schedule management and also integrates nicely with the Spring Framework. A tutorial on Quartz / Spring integration can be found here.

If timezones were not a factor the best way to implement the architecture would be with three job types: Hourly, Daily, and Weekly. However, the definition of when to actual send Daily and Weekly reports from the server is complicated by the fact that users expect this information at different times based on their timezone preferences. Since an Hourly scheduled job needs to be run anyways it makes sense to tie in to this and just have one job type not three.

As such, one approach may be a two dimensional array with each entry representing an hour with in the week. Dimension one would be the day of week 0 - 6 and dimension two would be the hours in the day 0 - 23. The grid would be based on the servers localtime. So that would be 24 * 7 = 168 buckets. Each bucket would contain a list of actions to perform. Building this time based array would be a bit of a pain but once set up it is very simple for a single scheduler job to wake up each hour see what operations need to be performed then delegate the operations to workers and sleep till the next hour. The time based array would be constructed on server startup and would need a locking mechanism so that real time changes to user notification preferences can be made while the server is running. The first day of week and at what hour to send daily reports logic based on a users timezone preferences would need be converted to localtime and then placed in the array.

The User Preferences would contain information regarding what collections to send notifications on, the type of notifications, and what output formats to send the notifications (EMAIL, SMS, IM). This information would be translated to a Java representation of the data used by the Notification Manager.

So each bucket with a value would contain one of more Notification Data objects that might look this in pseudo-code:

Class Notification:
  User ID
  Collection UUID (or ALL for all co)

  // Operation is used to build the startTime and endTime range
 Operation (HOURLY | DAILY | WEEKY)

  // Operation Type tells which method in the service layer to call findItemsInRange or findModifiedItems
  OperationType (FORWARD | MODIFICATION)

  // registered callback for all interested listeners
  // which may include one or all of the following:
  // EMAIL, SMS, IM
  listeners

Another option would be to develop a new persisted (stored in the database) type that would augment the User Preferences logic to save all the notification information. There then would be no need for conversion between key value pair User Preferences and a Java representation of the actual data.

The new notification events database table might look something like this:

user id  BIGINT(20) - foreign key
collection id BIGINT(20) - foreign key

// Bit representation of FORWARD(1) or MODIFICATION(2) 
operation type Bit(1)

// Bit representation of HOURLY(1) , DAILY(2), or WEEKLY(3)
operation Bit(1)

// The day of week 0 - 6. -1 if a DAILY or HOURLY operation
dayOfWeek INT 

// The hour in the day that the notification should be executed 0 - 23
hourOfDay  INT 

// A comma separated list of strings representing listeners (EMAIL, SMS, IM).
// The string name can be used to retrieve an interface to an instance via 
// a ContextManager. For simplicity I have modeled the listenerKeys as a
// comma separated list even though there are better ways to do this.
listenerKeys LONGTEXT

This would greatly simplify the Notification logic and also would eliminate the need for a locking mechanism since the changes to Notifications would just be updates, inserts, and deletes in the database table. The Notification Hourly job would simply do a query (via a Hibernate abstraction of course) for all Notifications that fall with in that day of week and that hour of day.

The SQL query might look something like this assuming it was run on the 3rd day of week at hour 14 in pseudo-code:

select * from notificationEvents where operation == HOURLY or operation == DAILY and hourOfDay == 14 or operation == WEEKLY
and hourOfDay == 14 and dayOfWeek == 3

On waking the hourly scheduled job would collect all of the actions to take during that period such as "Send a hourly modification notification for User X via EMAIL", "Send a forward weekly summary notification to User Y via SMS", etc. then either process each individually or have say 10 worker threads that could handle notification requests concurrently.

There is certainly more design work that needs to be done on the Notifications Manager but this is first pass to get things started.

Database Storage of Notification Modification

With the current Cosmo architecture, it only possible to determine that an item has been changed by looking at the modified date attribute. Discovering what exactly has been modified is a challenge and determining actions such as item deletion are virtually impossible.

Adding new tables to the Cosmo schema to support modification data persistence will make determining what has been modified much easier.

The two tables would be abstracted in a Hibernate / Spring layer. The main table would be the modification table which would have a transaction id, the timestamp of the modification, the item / collection modified, who modified it (user), and pointers (foreign keys) to what was modified including previous and current values. The modification table would be indexed by timestamp and collection for fast searches.

The second table would be a modification values table and would contain the actual modifications made. Since more than one modification can occur in a single edit / update such as changing both the start and end times or un-stamping the item as an event and stamping the item as a message, a one to many relationship needs to exist between the modification and modifications values tables. One modification transaction for an item / collection can have many individual changes. The table would need to be generic enough to capture events like addition, removal, and deletion of items as well stamping.

The four current use cases the tables need to persist data regarding are:

• When somebody created a new item, what kind of item, it's title and when.
• When somebody adds, removes or deletes an item from the collection, what kind of item it is, it's title and when.
• When somebody edited an item, what kind of edit, what kind of item, it's title and when.
• When an item pops to NOW, what kind of item, it's title and when

Doing a SQL join on the item table, the modification table, and the modification values table for a given transaction id / timestamp would provide all the data and more required to build the ItemModification^? diffs in the Service layer.

The addition of these tables opens up new possibilities in Cosmo. A number of reports and other valuable data gathering tools can be run to discover information such as "show me the most modified collection on the server in the last hour" or the "give me the total number of items created in the last day for user X".

These two new tables will at some point need to be purged for modifications that are older than X where X is a timestamp to prevent the database size from increasing exponentially. When to purge will depend on the number of users and the frequency of activity and can be done in a manual process by the sys admin as required.

In addition, the reminderTime attribute used to determine if an alarm falls in a specific time range should be indexed for greater performance.

Other Stuff

Performance

There are two areas where performance can suffer with greater user load. The main issue of concern would be a large number of Web Widgets polling via Atom and slowing down the server. The interval each Web Widget chooses to wait between checks as well as the use of etags for client side caching can have a major effect on performance.

The other area would be the storage of the item / collection changes in persistence layer (database). The Service layer on update events will need to determine what has changed between the current item and the persisted version and then save that information in the modification tables. This will increase the amount of time it will take to complete these operations and could slow down performance.

Timezones

Recommendations

• Do not support real time notifications for the first pass. The notification logic is based on scheduled jobs and thus additional architecture and coding would be required to support real time notifications. At least in the short term, I believe that the actual want by users for real time notifications is pretty minimal and not worth the additional developer time. Web Widgets by definition since they use a pull query model can't support real time notifications anyway.

• For IM only support XMPP / Jabber for the first pass. XMPP has the most adoption of any of the IM protocols and thus will reach the widest audience with the least amount of work. There are also a number of XMPP Open Source client libraries written in Java including Smack.

Open Issues

• What time period (start /end) do Web Widgets use for modification notifications (1 hour, 2 hours, 3 hours)?
• How often do the Web Widgets query the server for notifications (5 minutes, 15 minutes)?

-- BrianKirsch - 14 Mar 2008