Cosmo 0.6 Architecture

Model Building Blocks

Item

All content in Cosmo inherits from the abstract Item class. Item defines common attributes such as uid (unique identifier), name (used in DAV URI), displayName (item description), and common associations such as owner (user that created item), parent (collection that item belongs to), attributes, tickets, and stamps.

Collections are modeled as items (CollectionItem) with an additional association (children). Cosmo 0.6 supports items belonging to a single collection.

The subclasses of Item are ContentItem, NoteItem, HomeCollectionItem, and CollectionItem. ContentItem represents a binary piece of content in Cosmo. ContentItem includes attributes such as contentLength, contentEncoding, contentLanguage, and contentType. All files created through DAV are stored as ContentItem internally. NoteItem inherits from ContentItem, and is the base item for all pim (personal information manager) items. For 0.6 this includes calendar event items, but in future version it will include tasks and messages. Calendar event items are represented as NoteItem stamped with EventStamp (stamping explained below).

CollectionItem is an Item that contains a Set of child items. HomeCollectionItem is a special CollectionItem that represents a user's home collection. It is created when a user is added and is for the most part immutable. HomeCollectionItem is a root collection, it has no parent collection.

Implementation Notes Items are mapped to the item table using a table per class hierarchy in Hibernate. This means all Item subclasses are mapped to the same table, and use a discriminator column (itemtype) to denote the item type for the row.

Attribute

Item has a Map of Attribute objects, indexed by a QName. An Attribute can be associated to a single Item. QName is a composite key that represents a qualified name (namespace and local name). There are several Attribute implementations (String, Binary, Integer, etc) that are used to store different types of values. The polymorphic mapping allows any number of any mix of =Attribute=s to be associated to an item. It also allows new Attribute implementations to be created without any changes to Item.

Implementation Notes Attributes are mapped to the attribute table using a table per class hierarchy in Hibernate. All attribute implementations are stored in the same table, and use a discriminator column (attributetype) to denote the attribute type for the row. There is a bi-directional association between Item and Attribute, with Item containing the inverse mapping. This means that Attribute is responsible for managing the association. Save, update and delete operations are cascaded to Attriubte from Item.

Stamp

Item has a Set of Stamp objects. A stamp can be associated to a single Item. A stamp associates a related set of properties/attributes/methods to an Item. This is different from Attribute, which generally represents a single value. For example EventStamp, contains properties and apis relating to a calendar event. A calendar event is modeled in Cosmo as a NoteItem that contains an EventStamp in its set of stamps. CalendarCollectionStamp is another stamp implementation. CalendarCollecitonStamp is used to indicate that a CollectionItem is a calendar collection.

Like Attribute, the polymorphic mapping allows any number and mix of stamps to be associated to an item. It also allows new Stamp implementations to be created without any changes to Item, or Item DAO apis and implementations.

Implementation Notes Stamps are mapped to the stamp table using a table per class hierarchy in Hibernate. This means all item implementations are stored in the same table, and use a discriminator column (stamptype) to denote the stamp type for the row. EventStamp ,=MessageStamp=, and CalendarCollectionStamp are mapped to a secondary tables (event_stamp, message_stamp, calendar_stamp) that is joined to the primary stamp table. Note that not all Stamp implementations require a new table. Instead, a Stamp implementation could store data in the underlying =Item='s attributes. For example CalendarCollectionStamp stores a set of strings as a MultiValueStringAttribute in its underlying Item. One benefit (or drawback depending on how you look at it) of this is that when a stamp is removed, the underlying attributes remain associated to the item, allowing the item to be "stamped" and "unstamped" without losing the stamp data. There is a bi-directional association between Item and Stamp, with Item containing the inverse mapping. Like Attribute, stamp creation, modification, and deletion is cascaded by Item.

Ticket

Item has a set of associated Ticket objects. A ticket can be associated to a single Item. A ticket is an authorization object that grants some level of access to an item and its descendents based on a unique ticket key. If a client has this key, then the client can gain access to the item and its descendants with the ticket key.

Implementation Notes Tickets are mapped to the tickets table. Ticket privileges are mapped to the ticket_privilege table. There is a bi-directional association between Item and Ticket, with Item containing the inverse mapping.

Calendar Items

As stated before, NoteItem provides the basis for a calendar item in Cosmo. Cosmo 0.6 only supports calendar events, which are stored as NoteItem with an EventStamp. The parent of a calendar event item is always a CollectionItem stamped with a CalendarCollectionStamp. The EventStamp contains an ical4j Calendar object, which is an iCalendar component as defined in RFC 2445. The iCalendar component conatins at least one VEVENT component (master event).

Note that this allows Cosmo to support any event submitted using an external protocol such as CalDAV because the internal representation of an event is iCalendar. Cosmo preserves all valid iCalendar data.

Indexing Calendar Items

In order to support CalDAV and other internal query apis, Cosmo relies on internal indexes of calendar items. The indexing occurs at item creation and modification. There are two indexes: one for indexing the iCalendar properties (CalendarPropetyIndex), and one for indexing the timeranges of and event (CalendarTimerangeIndex).

To index the iCalendar properties, the iCalendar data is parsed into a "flat" form. This means the icalendar property:

DTSTART;TZID=US-Eastern:20070221T070000

gets indexed as:

key	value
icalendar:vcalendar-vevent_dtstart_tzid	US-Eastern
icalendar:vcalendar-vevent_dtstart	20070221T070000

This allows Cosmo to support CalDAV calendar query filters that contain property and parameter filters.

Cosmo does not index ATTACH properties and the maximum length for the property key is 255 characters and property value is 20,000 characters. Any property name that does not fit will not be indexed and any property value that does not fit will be truncated in the index. This means on the first 20,000 characters are searchable for a property value.

The calendar property indexes are stored in the cal_property_index table. Each index is linked to a single Item and EventStamp.

To index the time ranges of a calendar object, the time range or ranges for a recurring event are calculated. A time range index is created for each occurrence of an event. If event start/end times include a timezone, then the time range is calculated in UTC. Otherwise the time range is calculated in floating time. A "isFloating" flag is set to true for floating events and allows time range queries to handle both floating and non-floating times. This means that a recurring event will have multiple indexes. The drawback to this is that Cosmo doesn't really support recurring events with no end date. Cosmo 0.6 will only calculate and index occurrences up until a maximum date (2009). You can't jump ahead to 2020 and expect to see your weekly meeting that you scheduled with no end date.

The reason Cosmo indexes events like this is that it makes time range queries extremely quick. The expansion is done once up front at creation time and searches can be done with simple SQL queries with no crazy expansion logic. The drawbacks are obvious and we will have to come up a better solution in future releases.

Example of indexes created for event:

BEGIN:VCALENDAR
PRODID:-//Open Source Applications Foundation//NONSGML Cosmo Sharing Server//EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
UID:22fdf94d-9d45-4d31-bbfc-8cc42cb6f348
DESCRIPTION:description
SUMMARY:myevent
LOCATION:
STATUS:CONFIRMED
DTSTART:20070201T153000
DTEND:20070201T163000
RRULE:FREQ=DAILY;UNTIL=20070208
END:VEVENT
END:VCALENDAR

indexes created:

startdate	enddate	isFloating
20070201T163000	20070201T153000	true
20070202T163000	20070202T153000	true
20070203T163000	20070203T153000	true
20070204T163000	20070204T153000	true
20070205T163000	20070205T153000	true
20070206T163000	20070206T153000	true
20070207T163000	20070207T153000	true
20070208T163000	20070208T153000	true

The timerange indexes are stored in the cal_timerange_index table. Each index is linked to a single Item and EventStamp.

EventStamp contains collections for both index objects (CalendarPropertyIndex and CalendarTimerangeIndex) and relies on Hibernate's cascading lifecycle feature to manage them. Both timerange and property indexes are cascade deleted by the database when their associated EventStamp is removed.

CalendarDaoImpl, the Hibernate implementation of CalendarDao, utilzes a CalendarFilterTranslator to apply a CalendarFilter to a calendar collection and return a set of matching items. CalendarFilter is modeled after the CalDAV calendar filter. The default implementation is SQLCalendarFilterTranslator and generates a native SQL query that is run inside a Hibernate session to return a set of items. The reason a native query is used instead of HQL or Hibernate Criteria is because in order to support complex calendar filters that CalDAV allows, a UINON query performed best and Hiberanate HQL does not support UNION as of version 3.2.

Persistence

Cosmo 0.6 uses Hibernate annotations (http://www.hibernate.org/397.html) on all the model classes (org.osaf.cosmo.model.*) to map Cosmo objects to database tables. This frees us from the use of separate XML config files (as used in Cosmo 0.5).

All package level Hibernate configuration annotations are defined in org.osaf.cosmo.model.package-info.java. This includes named queries and custom types.

Cosmo utlizes Hiberate's second level and query caches. The cache implementation used is Ehcache, which can be configured using echache.xml.

Cosmo 0.6 has been developed and tested on Derby 10.2.1.6, MySQL^? 5 (InnoDB^?), and PostgreSQL^? 8.2 databases. Cosmo relies on Hibernate's auto schema generation to create the database schema at startup (see DbInitializer.java). Hibernate uses the annotations defined on all the model objects to automatically generate and execute the ddl statements that define the Cosmo schema. Because one of the goals for Cosmo is to support multiple databases, the database schema is not as optimized as is could be. We have to support the least common denominator, which means no custom triggers, stored procedures, or anything that is database specific that isn't handled by Hibernate's dialect abstraction.

DAO Layer

The DAO layer in Cosmo provides the basic CRUD operations for the model objects. The main DAOs in Cosmo are UserDao and ContentDao. UserDao handles all User CRUD operations and ContentDao handles the ContentItem CRUD operations. There is also a CalendarDao that provides special calendar object query apis. The only consumer of the DAO layer is the Service layer. That is, the only code that uses DAO objects is the Service layer.

Implementation Notes Cosmo provides a Hibernate based implementations for the DAOs. These implementations extend from Spring's HibernateDaoSupport class.

Service Layer

The service layer consists of UserService and ContentService and is what is exposed to the application. The transaction boundary in Cosmo is at the service layer, meaning each service call is a transaction. A service can utilize several DAOs. One example is UserService.createUser(), which needs to create a new User and create a new HomeCollection for that User.

Implementation Notes Cosmo uses Spring's declarative transaction management (TransactionProxyFactoryBean) for the service layer. There is no transaction code in any of the Cosmo service/dao implementation classes. Instead, transactional behavior is defined in Spring's applicationContext.xml. For Cosmo, this means each method called on a service require a transaction to be present, and will create a new transaction if one doesn't exist. Cosmo relies on transactions for atomicity. If a service method is called that updates multiple database tables and fails halfway through, the transaction is rolled back instead of leaving Cosmo in an invalid state.

Application Framework

Cosmo utilizes the Spring framework extensively to instantiate and configure the majority of the service/dao layer objects. An example is the Hibernate SessionFactory (no hibernate.cfg.xml file as everything is configured in Spring's applicationContext.xml).

Cosmo utilizes Spring's transactional framework. This includes the use of Spring's HibernateTransactionManager and TransactionProxyFactoryBean that wraps transactions around Cosmo service apis.

-- RandyLetness - 30 Jan 2007