Scope and Major release planning

Developing a large and innovative new software product is a serious and long-term undertaking. Planning any work among a dozen developers is difficult; describing product work out to a five-year time frame would better be described as imagining or suggesting a possible course and potential goals, than as a plan or roadmap. Still, we attempt to plan distant work at an appropriate granularity to guide our resource decisions and early work and to communicate with other stakeholders, and plan above all on revising the plan constantly as we do the actual work.

The first step is to have some idea what work is in scope for the product and what work is explicitly out of scope. This is absolutely essential to keep on focus. Since we've been working on Chandler for some time, we believe we have a fairly good understanding of scope. Still, scope issues continue to arise (particularly with contributors and advisory committees) and we should document explicitly what's out of scope.

The next step is to determine some rough phases. Humans are good at triage when the number of buckets is small so our highest-level sorting is very rough indeed: three planned major releases plus a bucket for "later/maybe". For Chandler these are Kibble, Westwood and Pasadena. See the roadmap to learn how we partition our high-level plans into K/W/P.

When will OSAF deliver Kibble, Westwood or Pasadena? We don't know -- we can't, too many factors change, and even unchanging factors are sometimes unknowable until we tackle actual implementation. Like most software development efforts, we are trying new things and we may in fact be trying too many new things at once.

Recap: Planning major releases beyond the current one is extremely hazy. However we can be specific about what is out of scope for Chandler and for each major release and this will be a help.

Point releases

A point release like "0.6" consists a cycle of work involving roughly a design period, a development period, a QA and bug-fix period, and a final documentation and verification phase including the actual release. At OSAF, point releases have taken from four to eight months apiece. Naturally we try to parallelize some of the work, beginning development before design is complete, beginning QA before development is complete, and beginning the next point release design before the old release is quite out the door. However, the work is not completely parallelizable because a certain amount of input from all teams is required at each stage.

The list of features for a major release the scale of Kibble or Westwood is enormous and we are incapable of determining a project plan for an entire major release in go. The only approach is to triage, triage again, then start work. We triage major releases into point releases. Kibble has had many point releases: six so far and two more envisioned if we hit no more major snags. We can't plan all of those at once, we can only plan one of them at once. We pick a set of goals for a point release (including developing a theme or set of tenets to organize the goals undertandably) and start breaking them down into projects.

In addition to describing themes and projects, we attempt to describe the expected maturity level. Innovative feature work cannot leapfrog to the state of being polished and usable in one point release: we need a cycle of use (if only by ourselves) and feedback in order to make a user interface feature polished and usable. Our tentative classification of maturity level:

• Embryonic: The basic ability to accomplish a task is there but the user interface cannot be considered to be designed or coherent. It may have been roughed in by a developer without the involvement of design simply in order to expose early underlying functionality. Embryonic user interface elements may be disabled or hidden unless the user takes explicit steps. Developers and QA are the only ones expected to use embryonic features.

• Initial: Functionality is more extensive but there are major flaws in how the functionality is exposed -- we may have taken short-cuts rather than implement to design. In other words the design hasn't yet been nearly accomplished. Again, the feature might be disabled by default.

• Plausible: The feature is not complete but a user with imagination can see how it's intended to work according to our design. At this point we have a hope of getting useful and relevant feedback from users about how this could work for them although we know there are still bugs and other problems, possibly significant missing functionality that prevents real-world use.

• Usable feature set: Now the feature is more polished and the feature set nears completion. We're not sure that the feature is actually usable but we believe that all the pieces are there. We definitely need feedback now about which design choices or bugs impede actual usability.

• Usable at OSAF: Sometimes we can develop a feature to the point where we can use it internally but more work is still required to make it broadly usable -- for example, easier account configuration and real broad interoperability.

• Usable externally: Reaching this point is our ideal for every feature, naturally, but we expect that getting there will typically require a few rounds of point releases, QA, personal experience and user feedback.

The lifecycle of usable features takes longer for more innovative features and it's important for somebody tracking Chandler's progress to realize this. Sometimes we may have to do major design rework to make a feature actually usable. We're not sure -- we haven't gotten there yet -- but a conservative plan would include time to completely rework a small fraction of our innovative user interface ideas, and to do significant rework on a larger fraction. A wild guess at those fractions might be 10% to 20% for complete rework and 20% to 50% for moderate rework. Note that kind of rework adds a project overhead of easily a third more time than if we miraculously managed to be right every time we designed an innovative way of doing things.

Recap: Major releases are organized into point releases to solidify the project and create feedback loops. Point releases are planned as a set of consistent-granularity projects organized around a theme. User interface projects should have their maturity level described and communicated.

From Projects to Tasks

A reasonable project is one to two months of effort for a single developer. We can break a feature into a set of projects at this granularity, provided the feature is well-understood. We also develop an idea of prerequisite projects and necessary sequencing choices. Often when breaking a feature into projects we quickly assign the projects to a series of point releases, knowing the projects won't all get done at once. However we focus on understanding the point release at hand, knowing the plan for future point releases will change before we get there anyway.

We keep planning at the level of granularity of entire projects all the way until we begin design work for a point release -- and sometimes even later. We typically need specifications written in order to gain an appropriately detailed shared understanding of what the project is expected to achieve. Specification writing takes place at the end of the previous point release and during the design phase. Developers read and contribute to specs (and sometimes write them) and the spec can't be done until the developer is able to scope their work into a set of tasks.

Breaking a project into tasks thus takes many hours of work by program management, design and development, often with review by a rather wide group. Even once the specification is rather fixed, a developer may need to spend some time researching and testing functionality before the task list can be drafted. Because of the high level of investment it takes to task out a project, we do not attempt to do this seriously until the developer is actually ready to start the project (it's a tenet of software development that speculative work is often wasted, a variation on the YAGNI principle applied to scoping instead of actual development). A developer can easily spend a couple weeks tasking out a project and if that project gets deferred, cancelled or significantly altered in conception, that work is almost entirely wasted.

Naturally scoping out the work for a project can affect the specification, the requirements, and future project plans. In fact, it is difficult to reliably even list the projects we think we're going to do for a point release until the previous point release is actually feature complete. Until we reach feature complete, we're never quite sure what might slip or end up being unusable due to unforeseen missing pieces.

Recap: Our detailed task planning and time estimate event horizon is only 1-2 months, equivalent to looking ahead one project per developer.

Completing tasks on schedule

We know that software projects at even the richest, hardest-driven software companies are prone to slipping. Much of this isn't visible to the outside world -- inside secretive and competitive software companies, each group's management adds slip space to the schedule. Even then, project teams routinely cut features in order to either make the schedule or at least break it less. Many software companies delay promising new releases until the release is nearly complete (and even then leave room for slipping) and most are naturally reluctant to provide feature lists for planned releases until the release is actually feature complete or even later.

Another model is that taken in many open source projects where work is relatively unplanned. Rather than divide features into buckets and lists projects and tasks and try to predict how long each will take, many open source projects simply have contributors take on goals and start working on them. Releases may happen on schedule, in which case a release simply includes whatever set of work happens to be sufficiently done in time for the release. Alternatively releases might happpen when some important piece of work is done (feature driven).

At OSAF, we have not allowed ourselves the secrecy of internal development schedules or the inscrutability of "it happens when it happens" plans. Understand that this means we undertake a planning task that is known to fail often and when we do slip we will slip publicly. OSAF observers should understand the risks in advance planning.

Estimating task time requirements is harder than it appears. At OSAF we attempt to do this collaboratively (mostly between a manager or tech lead and the developer who will actually do the work), to document estimates, and to review them against progress. We hope that this practice will improve our ability to make accurate estimates. At the moment our collective task estimates often vary from actual time spent by a factor of two -- most often in the wrong direction.

Interaction between features or developers makes task estimates more unpredictable. We attempt to minimize this with good communication, good review of specifications and task plans by any involved parties, and by designing interfaces which minimize interaction by defining clear contracts and fulfilling them. We also make very strong efforts to avoid adding work to the schedule in mid-development.

Recap: We haven't got task planning down to a science yet but are paying attention to improving it.