Platform Team Meeting 27 June 2006

Agenda

• Question of the week: How often do you want to have this meeting?
• Alpha3
  • background sync and quitting the app
• Alpha4
  • installers and dump/reload (Sheila as guest)
  • stamping?
• branching criteria
• Chandler-dev summaries

Notes

• Question of the week: How often do you want to have this meeting?
  • Grant: fortnightly!
  • PJE: ditto
• Installer
  • Sheila: looking for feedback on what the open issues are
  • Heikki
    • we don't have an installer on the mac, so the installer maybe shouldn't do migration
    • either in chandler itself or something that chandler launches
  • Grant
    • what is hard: 3 different ways we can need to upgrade: berkeleydb, andi's repo layer, our code
    • may or may not be the right thing to declare the first two problems we won't solve
  • PJE
    • assumption that one thing is being upgraded: not true if we have other parcels
    • Question: how to handle new parcels -- tie into upgrading 0.6 --> 0.7 -- this set of parcels from version 1 to another, when supporting other parcels is going to parcels --> chandler is a bunch of different parcels --> all of these are changing, maybe that gives us some shortcuts --> upgrading chandler
    • application itself? strictly by the installer?
    • one question: what is the user facing design of upgrade process, scoping that to include how does the user obtain and install plugins for upgrades to the software vs installation package in the first place -- upgrade or enhance from within chandler user operating environment
  • Heikki
    • when you upgrade firefox, upgrade then start for first time, checks version compatibility information, disables those marked not supported for current version -- pop up, check your extensions (menu) will show status of installed extensions. enabled/disabled because they are incompatible, some have upgrade available
  • auto check for updates, plug in management type facility, save the users data before the upgrade process begins -- new version needs to run an old version of chandler -- what is the user facing
  • Heikki: users running chandler does automatic checks to see if user is available?
  • uninstall current version -- enumerate the ways people could try and do things
  • mac installer? program that runs one time and then turns itself off -- do I need to run an installer?
  • Grant: people will just go change the name of the app?
  • Bear: what are other apps that do this? directory name you can give your app -- pristine copy? some trick to learn more about. Upgrade installer -- pull out chandler installer -- spawn that. User will download, two stage installer step -- otherwise you run into hairy mess.
  • PJE: another radical possibility -- what if we stored our data in a stable external format to begin with. Most email apps store their data in inbox format for a reason.
  • Bear: tug of war -- really limited.
  • PJE: additional indexes are built up. don't discard repository, just rebuild the repository when we upgrade. what is the stable format? upgrades controlled at lower level.
  • Bear: repository format is already stable -- haven't updated what gets ripped to disk. How python layer interprets changes.
  • PJE: storage layer, data model layer, domain model layer. Suggesting, push data model to an externalized format. storage layer is the only thing we have that is external. two layers on top, push another one of them out. redundant externalized format, upgrades can be transparent in mac arena.
  • PJE: upgrading the data format (domain model or another layer), better off limiting that, making changes effectively in your API, better to isolate that change from the physical representation.
  • Bear: mirror store, new chandler starts up for data migration, all code required to understand old data is mirrored, can do some type of conversion process. no different from needing to leave your old data installed. code sitting with data in different location from app.
  • PJE: don't we need old version of libraries?
  • Bear: store 2nd and 3rd layers alongside data -- reference special code to give old data and run through conversion process.
  • PJE: doesn't address schema evolution issue -- object relational mapping popular -- layer that allows you to duck schema evolution by having a translation layer between efficient or stable data representation and what you want in your program. We lack that translation layer. schemaAPI is a repository API, not a translation layer. Spec for one side of that mapping. Underlying thing -- this is how that data is physically stored. When we evolve our schema, don't evolve our format. Not really disrupt the existing data or schema -- only how we access or process.
  • When do we make this backup? Whenever you create a data directory? Today -- copy of python c library -- could be a distribution step. This is the manifest used to create the schema evolution api conversion directory. Doubling our disk footprint. Need copies around. Might be able to get away with copying non-gui parts of the app.
  • Heikki: you'd have to copy everything but wx. 50% of the python code. modules may import wx.
  • Grant: upgrade only work at the level of Kind, Attribute, Type.
  • Pylucene, berkeleydb. Substantial footprint.
  • Redundancy in the data instead of redundancy in the program. Backend selfdescribing and stable OR external representation for primary application data coupled with indexing facility that is redundant, so that it can be recreated. (mbox format, but also have database and indexes, in a pinch you can rebuild).
  • How does this help us for parcels with data that are not those types.
  • Data is always in it -- never a dump.
  • Tightly coupled parcel to another library? No purpose of an external format, by externalized format, standards defined or otherwise limited to well defined simple data structures. Not something that evolves, enduring format. Part of having a valid parcel is taking responsibility for dealing with the standard format. Well understood problems within this sphere. Need to be able to read your own sharing format.
  • 3rd party parcels blowing off old formats -- external data format versioned, parcels register callback for processing particular version, system knows that the data is not loadable. The problem is isomorphic. APIs write to this format. Right now, our platform maps transparently and you have no control over the mapping. Transparent object persistence is a problem not a solution -- the transparency shifts the problem to sharing, performance burden. You always have to write this code, frees you up to do other things.
  • Only place where this is really in scope: sharing. Here's the format you want to do for sharing. Schema API as translation layer for this format. How much API change would we be talking about? PJE: not where the changes are likely to be. The real depth of API activity: we expose refcollection indexes at application code level. Not something that can be changed transparently.
    • refcollection indexes
    • implementation details to gain performance -- not merely hints but things that you have to use -- we set up data structures and maintain them to make it fast
    • monitor and notifications
  • As we look at information model for sharing format -- consider possibility of explicit transformation, might gain us wins in a number of areas -- performance issues
  • UI coupling to the repository -- more stuff for future of the platform discussions

• • Installer changes can be quick, or need to investigate pre-install conversion process
  • Sheila's next step is to detail that more
  • Format discussion going
  • When do we have to support upgrading the data? Alpha4 - Alpha5: manual solutions are still possible
  • When is it a requirement to be seamless?

• Cut down on gratuitous library upgrades -- stricter the closer we get to 1.0. PJE: upgrade more quickly rather than slow the pace of change -- we don't do ourselves a favor trying to freeze a moment in time. Ted: we're already so far ahead of the platform, chance of using platform libraries close to 0. PJE: I don't see that as a problem. Ted: At some point we'll stop and be an application that people use. If you ship a real product you have to stabilize it.

Status

Katie

• Progress
  • Worked with Jared on hosted service roadmap -- he'll present to staff this week
  • Logged various bugs at IRC QA session Friday
  • chandler dev summary
  • "brand fitness audit" for Pieter and branding team
• Plan
  • continue focusing on alpha3
  • priorities writeup and discussions on cosmo/scooby lists
  • get installer discussion going with sheila on chandler-dev
  • work with Ted and PJE on developer platform strategy

Andi

• http://kumu.osafoundation.org/~vajda/logs.cgi

Morgen

• Progress
  • Fixed the problem with TestSharing^?.py (functional test) was failing under debug mode
  • When the user hits the sync all button, any outstanding edits in the attribute editors are applied before the commit takes place
  • Worked with Jared to get more data on cosmo-demo-related timeouts
  • Fixed a bug where subscribing to monolithic .ics files broke when bgsync landed
  • Fixed a bug where deleting a collection can cause background syncs to fail from now on (bug 6135)
  • Restore shares dialog now first checks to see if sharing account is set up
  • Hooked up menu items to the new bg sync code, instead of old dialog
• Plan
  • Finish alpha3 bugs

Ted

• Progress
  • Bug:1745
  • Licensing files
  • dev platform strategy thoughts
  • brand fitness audit
  • desktop IRC QA session
  • web workflows design session
  • hiring/recruiting
• Plan
  • Bug:1745
  • dev platform strategy thoughts
  • SoC^? intern reports
  • Cosmo/Scooby OSCON talk
  • hiring/interviewing
  • branding plan

Heikki

• Progress
  • Fixed a handful of Chandler regression bugs
  • Prepaing for adding intel mac tinderboxes
  • Updated 0.7 performance targets
  • Winding down on alpha3
• Plans
  • Drive alpha3 bug count to under 10 so we can branch
  • Ship M2Crypto 0.16

Grant

• Progress
  • 6149 Typo in WebDav.py prevents certificate panel from showing
  • Fixed, pending review: 6151 unstamping email-ness removes from calendar or task list
  • Figured out the issue underlying 6040.
  • Fixed, after discussions with JohnAnderson, a problem in the functional tests that was causing them to fail for some developers.
  • Filed a couple of bugs (6124, 6126) in the background sync bug-a-thon.
  • Looked into issues related to recurrence in 0.7α3
  • Finished up first phase of recurrence-related work; still need to track down some sharing-related issues.
  • Ongoing assistance to ErnestoRivera (SoC intern) and Travis Vachon.

• Plan
  • 0.7α3 bug fixes
  • SoC and/or intern support.