0.4 "post-mortem" 1 November 2004

Agenda or topic list

Review of ZeroPointThreeDebrief20040224 -- "0.3+" indicates a "more of" from 0.3, and "0.3-" a "less of".

Planning

• Making release expectations clear, write them down (0.3+)
• Set goals earlier in the release, make goals clear (0.3+)
• Make expectation changes clear when it happens, write it down and publish it (0.3+)
• Planning for the release early on (0.3+)
• Gold plating the end of the release (0.3-)
• Meaningless milestones (0.3-)
• Orphaned projects/snakes (manage snakes as projects, set expectations and priorities, track progress) (0.3-)

Design

• First stab at specifications (and testing to specifications)
• Demos of some sort from the design team (informal, short) (0.3+)

Development processes

• Increased code reviews
• Tracking tasks by milestone (0.3+)
• Stabilize repository early in release (0.3+)
• Get more than one person to review design (either before coding, or early on enough to refactor according to feedback (0.3+)
• Write up design and get feedback -- from the dev list and other teams, etc (0.3+)
• Better cross platform development environment for developers (0.3+)
• Lxr tool (0.3+)
• Code building up in a private environment (0.3-)
• Sprinting at the end (more consistent marathon pace throughout) (0.3-)

Testing

• Testing to specs
• Platforms supported
• Regular testing, during office hours, plan during the release (0.3+)

Bug fixing, reporting and triage

• Bug council (0.3+):
  • Short/effective bug council meeting
  • clearly define blocking bugs criteria, don't spend time at bug council meeting defining it
  • clarity about attendees for bug council meeting
  • wiki page for bug council with important links, attendees, policies, ...
  • look at bug list before the bug council meeting, link available for the table that we use
• Level of robustness expected of release
• More useful list of bugzilla components (0.3+)

Build/Release

• branch during the release process to not slow development (0.3+)
• public display of the build state (0.3+)

Documentation

• Topic coverage
• process of writing/reviewing
• Timing relative to release
• Standard formats for documentation (to reduce time worrying about format, better info management) (0.3+)
• Upfront planning of documentation for release (0.3+)
• Feedback on existing docs, to know what's useful (0.3+)
• One person to own docs, notice holes, consistent style (0.3+)

Community

• Office hours (0.3-)
• Use of 'dev' list
• Use of consistent project pages for status and goals
• Integration of community working group into the process (0.3+)
• Strategic decisions about efforts put towards the community (0.3+)

General

• Forum for intergroup/interoffice/intraproject communications (0.3+)
• Health of project dashboard (0.3+)
• Investment in making tools better (bugzilla, wiki, etc) (0.3+)
• Hard to find info on wiki, bugzilla (0.3-)
• Marcomm plan for each release ahead of time (0.3+)

Actual Notes from Meeting

Planning/requirements

More of: slicing across large set of functionality

• projects can be deep or broad... deep projects don't provide as much apparent progress as it would if the project was a slice across a larger set of functionality

More of: more explicit about our philosophy about how polished our UIs can be, what requirements are and aren't

More of: goals around performance, set expectations (user centric as well as dev centric)

• should have a goal ahead of time for when we'll stabilize repository

Less of: too long a release

• Things changed too much from beginning to end of 0.4 and one can't plan for that long.

Observation: development is a learning process that is hard to plan for.

Accomplishment: we did remain truer to goals in 0.4 than in 0.3 (more people to achieve goals, more of a known universe to work within)

Design/specs

More of: relating implementation work back into design

• closing earlier in release on what features actually do (what simplifications need to be made
• for testing: our designs were more forward-looking than our implementations so designs couldn't be used directly for testing. Need more functional specs about what's in/out.
• need meetings near the end of dot release cycle to confirm how features really worked (e.g. sharing meeting)
• feed implementation experience back into spec more
• rigid specs reduce our flexibility in long cycles.
• continue to iterate design

More of: design and engineering working face to face

• verbal interactions to go along with specs
• we did some design/engineering meetings but not many followups --> need more of
• more earlier on as well
• make sure that design knows what is really hard/easy (not make assumptions)
• regularly scheduled meetings (like the detail view status) in case issues come up

More of: user feedback on our design (usability testing?)

Less of: duplicative wiki pages without status

• many, many wiki pages to track and hard to know source of truth (chrome, widges, DnD, detail view)
• multiple project pages produce a fragmented todo list from a developer's eye point of view

Accomplishment:

• we did a lot more written communication than in 0.3, as seen in more Wiki pages and peoples' Journal pages.
• more organized design material in 0.4 than before

Development processes

More of: insistance that once something is working it must remain working

Less of: repository changes that require deleting repository

• possibly a feature to avoid throwing away data when repository changes
• possibly export/import, possibly upgrade

Accomplishment: we didn't gold-plate the end of 0.4 as much as 0.3 -- didn't set the bar higher and higher in bug council

More of: our pace was pretty steady but it would be nice to have more features wrapped up earlier to be able to test them (not all finish right during feature complete)

More of: stabilizing repository early in release (just like after 0.3);

More of: track down panic errors before working on data longevity

More of: code reviews

• code reviews have been very useful to stimulate architecture discussion
• 3-person code reviews better than 10-person code reviews
• both in-person and email code reviews have seemed useful

Accomplishment: we did better at not changing repository API in last few weeks of release, than in 0.3

Testing

More of: unit tests running constantly adding to repository (should ultimately run for months without error)

More of: performance unit tests

Accomplishment: we did much more QA

Bugzilla, bug council

More of: bug council should invite developers in to discuss their current bugs

More of: clear what our list of bugs for release are -- need one bug list to all look at - one link to follow

Build/release

More of: could have tinderbox track performance

More of: supported/tested platforms such as Windows 2000

More of: keep an eye on the portability of our library choices and other decisions

More of: release communication in last stage

• what was going to happen and what was expected of people -- meaning of code freeze, etc.
• one location for all release deliverables rather than pages here and there
• need to track things other than code (features, bugs) before the very end of the release (non-code deliverables wiki page existed but not widely known of)

More of: looking at tinderbox (and not breaking build)

Accomplishment: we saw nice improvements in build process over course of 0.4

Community

More of: help from volunteers doing other platforms and maintaining Chandler

• make clear our policy for getting volunteers help us test/maintain platform support
• solicit help from outsiders on choosing libraries and related decisions
• need to set expectations with community about challenges supporting different platforms -- not zero-effort!

More of: developer dogfood progress

• Some progress made but more needed

Accomplishment: we are getting better at getting community goals into day-to-day efforts, and expect to continue to have community integration

Observation: we plan to make it easier for outside developers to contribute by having a framework in place to add to

Mixed more/less: IRC chats

• IRC topics where we did all the talking felt strained, possibly less useful
• QA IRC chats generally productive
• IRC topics where we talked to each other were still useful (e.g. source code repository) even without so much community involvement
• we should have community make requests about what they want to hear more of (in office hours, etc)
• In the very long run we'll move towards constant office hours rather than scheduled

Documentation

More of: guidance getting up and started on Linux platform

Accomplishment: documentation was much better in 0.4 -- more of it, better quality (and more features to document!)

General/communication

More of: tracking meetings in central location (iCal) to keep group awareness up and allow ad-hoc attendance

More of: up-to-date status of where we are (e.g. feature design, development, feature freeze, code freeze), in prominent location (DevelopmentHome)

More of: long term goals on project and WG pages

More of: targets for external events where we meet specific goals of stuff to show (more interest from community generated by that)

Accomplishment: better at managing/tracking snakes in 0.4 and reduced population

Observation: doing integration work periodically is key

More of + accomplishments: Improving Wiki usage

• Wiki page names still hard to remember/guess/use
• we have come a long way in looks/usability of wiki (sidebars, menus)
• rss feeds great (even better if had full page text
• still hard to know which of seven pages is the best one for a topic
• wiki searches hard to use, alphabetical search results aren't useful
• hard to join project and find where things were and what's current; and follow up on a page only to have people say that the page wasn't current
• more of removing things from wiki when obsolete - more tracking past stuff in clear archives (Journal) rather than leaving historical detritus in main pages
• more jumping-off spots in wiki (point person can gather links to other pages and explain them)
• more stuff annotated by date so you can see what's not current
• more staff notes pages (incredibly useful)

More of: communicating with each other

• tools for knowing how/when to do ad-hoc interactions? E.g. ...
• more maintaining IM status so we know when to interrupt each other (or not to)
• Keep the conversations going about how to be effective and work together best
• we may have scaled back too far on big meetings with lots of give and take (due to past experience with rat-holes) - re-instantiate those somewhat
• more of: contact information to include Skype accounts

More of: input into things even if not directly responsible

Followup meetings needed:

• Supporting multiple platforms and community help around that
• Wiki issues

-- LisaDusseault - 27 Oct 2004