OSAF Hosted Service Presentation to 2006-06-29 OSAF staff meeting

Slide 1: Contents

• Hosted Service as Project
• Roadmap
• Scaling
• Hosted Service Engineering: Load distribution
• Hosted Service Engineering: Email integration
• Support and outages
• What should we be doing Right Now?

Slide 2: Hosted Service as Project

• We're starting an official Hosted Service project at OSAF.
  • service-dev, projects wiki, bugzilla, staff meeting updates, PPD integration
  • Name TBD
• The Hosted Service provides always-available network-accessible services that support the calendaring and PIM goals of users of OSAF software products.
• We're shooting to sync with overall OSAF Alpha, Beta, 1.0 schedule.
• The service is a member of the ecosystem. Unique position:
  • Available 24x7: Web UIs, mashups, scripted integration, mobile
  • All services are network protocol accessible: HTTP, WebDAV, CalDAV, CMP, Atom, JSON
  • Centralized / federated architecture makes some things easier: synchronization, email, feeds, group coordination, asynchronous operations

Slide 3: Roadmap

• The bulk of the Hosted Service features are provided by Cosmo and Scooby.
• Some service-specific code will be written: email robots, batch jobs, etc

• Alpha: Scooby 0.3, Cosmo 0.5, and ...
  • Experimental email
• Beta: Scooby 0.4/0.5, Cosmo 0.6/0.7, Chandler 0.7final, and ...
  • More email
  • Probably 1 or 2 nifty things not yet conceived. Perhaps mobile.
• 1.0: Scooby, Cosmo, Chandler, and ...
  • Revenue; interop; additional target users

• Daydreaming about nifty hosted service features is easy; but limited by timeline, people, "upstream" software (Cosmo, Scooby, Chandler), more. Mindset is to focus on ecosystem support and Beta reality.
• Some roadmap visualizations at HostedServiceOpsOverview

Slide 4: Scaling

• Operations (servers, bandwidth, customer service, staff) all scale with growth.
• Proposing "average number of weekly Chandler users" as core metric.

• Load = Target user load + Casual-users-driven-by-target-users load
• Target user load = Chandler background sync
• Casual user load = Scooby-driven Cosmo read/write ops + Feeds
• Background sync likely to dominate load

• Right now, a 1300-event calendar takes 90 seconds on an unloaded 3Ghz server.
• 1,000 users doing background sync every 30 minutes will generate about 25 syncs/minutes.
• Based on rough estimates, on a $15k server with current scaling, we could support about 200 large-calendar users doing background syncs (or $75/user). Regular users, estimate 1,000 users per server.

Slide 5: Hosted Service Engineering: Load distribution

• Current deployment architecture
  • Apache/mod_proxy for HTTP indirection (SSL, multiple hosts, throttling, etc)
  • Tomcat hosting Cosmo + Scooby + Jackrabbit (storage engine) + Derby (Pure-Java RDBMS)
  • 1 Java process and 1 Apache process on single server
• Most promising directions
  • Method profiling, SQL query auditing
  • Replace in-process Derby database with separate MySQL. Later put MySQL on separate box.
• Future work: Multiple Scoobies to 1 Cosmo
• Blue-sky: EJB, multiple independent clusters

Slide 6: Hosted Service Engineering: Email integration

• Proof of concepts planned
  • Email submission of new events to Cosmo calendar
  • Full send-receive email accounts

• Likely architectures
  • Postfix: highly-modular open-source incoming mail daemon
  • Programatic control over recipient mapping (database or scripted lookup)
  • Python batch jobs

• Spam sucks - large operational issue. Who wants spam in their calendar and todo lists?

Slide 7: Support and outages

• Service problems are subtle, often with multiple products involved.
• KEI IT will provide helpdesk/network operations (NOC) services.
• Outage events:
  • Send outage issues to: service911@osafoundation.org.
  • Multiple people will get paged 24x7.
  • OSAF "strike team" will get copies of new issues and status updates.
  • Notice will go out when an IT team member has acknowledged and taken the issue.
  • Web page will be available which provides list of current problems
• Service level:
  • 20 minutes during business hours (9-5:30pm)
  • Good-faith best-effort during non-business hours. Likely < 1 hour during evening.
  • Will grow expectations as users acquired. First colo 24x7, then in-house 24x7.

Slide 8: What should we be doing Right Now?

• Project basics: list, wiki change, bugzilla
• Performance of server operations
• Vetting and supporting load distribution models to support scaling
• Prototyping email operations and integration
• Building automation and monitoring systems
• Planning alpha and beta functionality and development
• Buying couple of dedicated servers to provide more runway

Slide 9: Questions

• Questions, comments, and ideas welcomed!
• Also subscribe to service-dev@lists.osafoundation.org