Preview Tracking Proposal

Based on all the discussions we have had so far about Preview tracking and potential risks, I put together a set of decisions and next actions to discuss during the next Preview meeting.

Recap of concerns

• Worry about feature creep and not enough discipline and focus.
• Goldplating features unecessarily.
• Emotional wear of a continually slipping release date.
• Unpredictable end for Preview with a changing and unknown bug count.
• Concern over the large number of features landing in Chandler Desktop at the last minute and the QA time required.
• Not enough QA resources to get accurate coverage of the app in order to assess the bug baseline - bugs keep trickling in.
• Performance and the unknown amount of time to get it to the level we need for Preview.
• Finish line is not very clear particularly around expectations.
• We don't all agree on the user goals. Day to day adoption vs trying it out and giving us feedback but never using the app again.
• Feeling by some that we need to cut drastically to get to a date.
• Feeling by others that there is nothing to cut.
• Not enough non- OSAF dogfooders participating in the QA.
• Expectations and pressure for Preview - pressure from the book.

Proposal with immediate next actions

This does not address all of the concerns but the following are suggestions for how we can proceed in the short-term to help manage the Preview release process, mitigate risks and give everyone enough information to understand where we are in achieving our goals.

• Focus on what we can gain confidence about right now - determine an accurate date for Chandler Desktop feature freeze (March 26)
• There will be a period after desktop feature freeze where everyone focusses on performance work (3 weeks). We realize that we may have to fix some bugs as well if they block dogfooders and there will be dependencies with Cosmo 0.6.1 interop work.
• We will do a best guess estimate based on today's desktop bug count as to the time required to fix bugs ( June 16th) realizing that after Chandler feature complete and we have had a chance to do some level of testing to get a baseline on bugs and performance, we may have to adjust this. There will also be associated bug triage and punt activities on an ongoing basis.
• Use Philippe's write up on milestones and exit criteria to see how we are doing.
• Draft the complete set of workflows we are supporting for Preview and track which ones are complete, which have bugs and prioritization. This will help clarify the finish line better and manage feature creep in general and help punt bugs.
• From now until desktop feature freeze we will use the workflows to manage what is reaching feature complete and when.
• At this time, we will not track the same milestones and exit criteria (as the desktop) for Cosmo. Tracking Cosmo and 0.6.1 will be handled via the workflows since 0.6.1 is a workflow driven milestone anyhow - get something dogfoodable.
• Revisit whether or not we should track Cosmo 0.7 with milestones and exit criteria after 0.6.1.
• Post Chandler Desktop feature freeze we will track the path to Chandler Desktop code complete using several artifacts
  • Milestone definitions and exit criteria
  • Bug graph
  • Separate tracking mechanism for performance which gives visibility into the strategy - outstanding bugs, performance metrics, resources etc (3 week performance period allows us to do this easier).
  • Workflows and level of "buggyness" or "brokeness" - design assessment of features.
• Agree on the criteria to prioritize bugs so that we can better narrow down the list we have to fix for Preview. The proposal on the table is...
  • P1 Playing around with the app, trying out the features
  • P2 Stuff that blocks day to day usage
  • P3 Stuff that confuses people
  • P4/5 Stuff that makes it more likely for people to stick with Chandler (conveniences)
• Mimi and Sheila will go through all the bugs and categorize into these buckets - use a whiteboard status.
• Could make a decision to automatically defer anything that fits into P4/P5 pro-actively.
• We can further tag bugs with additional criteria ie: performance, crash, dataloss etc to track what types of bugs we are finding.
• Documentation milestones we will only track at a high-level in the Preview countdown meeting. PPD can choose to track progress at a more micro level internally if they want.
• Continue to track hosted service goals at the level they are now as part of the Preview countdown unless Jared would like to see more detailed milestones.
• Agree on the user goals. Proposal...
  • Get people outside of OSAF to dogfood Chandler the way Mitch, Sheila, Priss and Mimi have been dogfooding:
    • Using Sharing
    • Using the Dashboard
    • Using the Calendar
    • Using Edit/Update

-- SheilaMooney - 04 Mar 2007