Cosmo 0.6 Sharing Proposal

The purpose of this topic is to present ideas for how Cosmo 0.6 is going to implement sharing with Chandler. There is a lot of work involved, and this is an attempt to break that work up into a series of phases. Each phase may or may not relate to a separate release, depending on the estimates to implement the features in each phase. Cosmo 0.6 will implement everything in Phase 1.

Phase 1

Goals

• Implement "morse code" sharing protocol based on EIM, providing basic create/update/delete/synch methods for shared collections. Phase 1 will not include item sharing.
• Support sharing of basic Chandler item types (Events, Notes, Tasks, Messages)
• Continue to provide CalDAV/feed support for managing calendar collections, and WebDAV support for general file sharing.

The main goal in Phase 1 is to define and implement a new sharing protocol based on the ExternalInformationModel. The protocol will include support for sharing collections that include Events, Notes, Tasks, and Messages. There will be no support for sharing individual items, and no support for sharing arbitrary EIM records. The other goal is to continue to support additional protocols for the management of special types of collections, mainly CalDAV. We want to continue to support clients other than Chandler (Cosmo UI, CalDAV clients).

Limitations

Phase 1 will include a few limitations:

• EIM Records types are limited to ones that cosmo understands

Any EIM record type that cosmo does not understand will be ignored. Chandler should not send record types that it knows cosmo will not support.

Sharing Protocol

A collection is shared in Chandler by exporting everything in the collection to EIM records. These records are then serialized to an XML format (TBD) and sent to Cosmo using a simple RESTful protocol using HTTP. The protocol will provide the following actions:

• create collection (HTTP PUT)
• retrieve collection (HTTP GET)
• update collection (send only diffs) (HTTP POST)
• retrieve updates to collection (sync) (HTTP GET)
• remove collection (HTTP DELETE)

Sharing calendar collections for Phase 1 will involve using the new sharing protocol (as opposed to WebDAV + CalDAV in 0.5).

A publish of a calendar collection would look something like:

1. issue a PUT to create shared collection
2. export collection to EIM records
3. serialize EIM records into XML, sending all records in a single publish call (POST) to Cosmo using new sharing protocol

Synchronizing a calendar collection would look like:

1. retrieve updates to collection using synchronization call to Cosmo using new sharing protocol (GET), passing synchronization token from last sync.
2. publish updates using sigle publish call (POST) using new sharing protocol

Implementation Notes

Sharing Protocol Details

Data is transmitted using an XML representation of the EIM records. We will define a new mimetype:

Mimetype	Description
cosmo/eim+xml	eim records

For example:

<records xmlns="http://schemas.osafoundation.org/sharingformat/1"
    xmlns:con="http://schemas.osafoundation.org/pim/contentitem"
    xmlns:tag="http://schemas.osafoundation.org/pim/contentitem/tags"
    xmlns:cta="http://schemas.osafoundation.org/pim/contact" >

    <con:record>
        <uuid>1</a> <!-- In this example I used simple uuid values for readability, 
           but real uuids will be in RFC 4122 form, e.g. 611fcf54-296e-11db-b36c-bc8a258a92d5 -->
        <con:title>Example note</con:title>
        <con:body mimetype="text/plain" encoding="utf-8">VGgZ2V2tlbg==</con:body> <!-- Lob -->
        <con:createdOn>2006-08-08 9:50:58.432510 US/Pacific</con:createdOn>
        <con:lastModifiedBy>4</con:lastModifiedBy>
    </con:record>

    <con:record>
        <uuid>2</a>
        <con:title>Work</con:title>
        <con:createdOn>2006-08-08 9:50:58.432510 US/Pacific</con:createdOn>
        <con:lastModifiedBy>4</con:lastModifiedBy>
    </con:record>

    <con:record>
        <uuid>3</a>
        <con:title>Work</con:title>
        <con:createdOn>2006-08-08 9:50:58.432510 US/Pacific</con:createdOn>
        <con:lastModifiedBy>4</con:lastModifiedBy>
    </con:record>

    <con:record>
        <uuid>4</a>
        <con:createdOn>2006-08-08 9:50:58.432510 US/Pacific</con:createdOn>
        <con:lastModifiedBy>4</con:lastModifiedBy>
    </con:record>

    <tag:record>
        <tag:item>1</tag:item>
        <tag:target>2</tag:target>
    </tag:record>

    <tag:record>
        <tag:item>1</tag:item>
        <tag:target>3</tag:target>
    </tag:record>

    <cta:record>
        <uuid>4</a>
        <cta:emailAddress>morgen@example.com</cta:emailAddress>
        <cta:firstName>Morgen</cta:firstName>
        <cta:lastName>Sagen</cta:lastName>
    </cta:record>


</records>

Question: How do we represent deletions?

To represent a deletion, we need the record type and the key(s) of the deleted record.

Retrieve collection records:

URL	http://server:port/cosmo/mcp/collection/[uid]
Method	GET
Querystring	synctoken=	synchronization token
	updatetoken=	update token
Returns	200 OK & XML (cosmo/eim+xml) + new sync token
	401 Unauthorized
	403 Forbidden
	404 Not found

Create collection:

URL	http://server:port/cosmo/mcp/collection/[uid]
Method	PUT
Querystring	parentuid=	uid of parent collection, if not present, home collection is parent
Request Body	None
Returns	201 Created
	401 Unauthorized
	403 Forbidden
	404 Not found
	405 Method Not Allowed

Update collection:

URL	http://server:port/cosmo/mcp/collection/[uid]
Method	POST
Request Body	XML (cosmo/eim+xml)
Returns	200 OK + update token
	401 Unauthorized
	403 Forbidden
	404 Not found
	415 Unsupported Media Type

Delete collection:

URL	http://server:port/cosmo/mcp/collection/[uid]
Method	DELETE
Returns	204 No Content
	401 Unauthorized
	403 Forbidden
	404 Not found

Authentication/Authorization

Currently, sharing authentication and authorization is implemented with tickets. A read and read/write ticket are created after a publish, and these tickets are used by others to obtain access to the shared collections.

Question: Do we continue to use tickets with the new sharing protocol? If so, then the above GET/POST methods would require a ticket parameter. How are tickets created using the new protocol? MKTICKET? Seems like tickets are a DAV-centric thing.

Data Model

For Phase 1, Cosmo will understand only EIM record types that correspond to calendar events, tasks, notes, and messages. Cosmo will understand and translate these record types to and from icalendar data in order to interoperate with other protocols such as CalDAV. All other EIM record types will simply be stored as Attributes in the CollectionItem.

In order to accommodate for EIM records, there needs to be a change in how item attributes are modeled. Currently, an Item has a Map of Attributes, indexed by attribute name. EIM records have namespaces, so we need to add namespace support. Instead of a Map of attributes, each Item should have a Set set of attribute sets. An AttributeSet has a namespace, and a set of Attributes. This maps nicely to an EIM record, so think of an AttributeSet as an EIM record. An item can then contain a set of EIM records.

For recognized EIM records, these records will be stored in individual CalendarItems. All non-recognized EIM records will be stored in the CollectionItem.

Synchronization

Part of the new sharing protocol includes a synchronization method. The idea is that Chandler should only retrieve changes to EIM records. Currently, Chandler performs synchronization using ETags on WebDAV resources. This requires doing a PROPFIND on a collection to retrieve the ETags for every resource. This can result in downloading over 300K of data for the office calendar, even if there have been no changes. Synchronization using the new sharing protocol works a little different. Instead of using ETags, a synchronization token is used. After a subscribe or synchronization of a collection, a token is returned to the client. The client should use this synchronization token on subsequent synchronization calls to only retrieve data that has changed since the synchronization token was returned.

After each update to a collection, an update token is also returned. This token is used during the sync process to ignore updates that were made by the client. Chandler clients should send this token along with the sync token in order to assure they only receive updated records since the last sync + update.

Publish sequence:

Synchronize sequence:

In order to implement this type of synchronization, Cosmo will have to keep track of all changes made to EIM records (AttributeSets). There will be some overhead and extra space required, but it will streamline the synchronization process significantly. To keep track of changes, Cosmo will need an activity log that is updated on every addition/modification/deletion of an EIM record. This can be done with a generic ActivityLog table that looks something like:

This generic log can keep track of any number of user activity, and in the future Cosmo can contain nice reports using this data.

Estimates

Note, estimates are very conservative and each task includes time for documentation and unit/integration tests. Some tasks may involve ramp-up time (if we decide to use RESTlets for example):

Task	Estimate
Implement AttributeSet, migrate existing code	1 week
Implement ActivityLog (objects, hibernate mappings, dao, service)	1 week
Implement MorseCode protocol
--servlet (or restlet)	2 weeks
--morse code base services	2 weeks
--morse code sync service	2 weeks
--add EIM bindings to CalendarDao	1 week
Total	9 weeks

-- RandyLetness - 30 Oct 2006