Contrary to some common misperceptions, agile is a highly disciplined and rigorous framework for developing software. Like any other disciplined undertaking, agile requires solid planning. Agile planning is however fundamentally different from traditional waterfall planning. The difference being that agile planning is adaptive. The principles behind agile planning are: 1) plans should only be as detailed as the depth of information currently available, and 2) planning must be continuously adjusted as improved project knowledge becomes available. Agile teams should be prepared to make planning adjustments as information about requirements, design constraints and the team’s velocity improves.
Agile planning occurs at a number of levels: Product Backlog > Product Roadmap > Release Backlog > Iteration Backlog.
Figure 1. Agile Planning
Product Backlog. This is the list of every conceivable requirement, feature request, enhancement, and wish-list item for the product. This list is derived from a variety of sources, including, product management, sales and marketing, customers, engineering, customer support, and virtually anyone who feels they have a stake in the product. Some organizations may have instituted a “Requirements Management” process, where requirements are processed through a formal workflow, with state-transitions like: submitted > accepted/rejected > approved, for reviewing and then accepting or rejecting proposed requirements. This results in a ‘groomed’ product backlog. Regardless of the degree of formality with which this is done, product management owns the backlog and is responsible for its grooming and prioritization.
Product Roadmap. For reasons of budget, engineering resources, market windows, and other constraints, not all product features can (or should) be delivered in a single release. Therefore a plan is required showing how features in the product backlog will be brought to market over a series of product releases. This high-level plan is usually referred to as the product roadmap. The product roadmap shows how major new product capabilities get rolled out over time, with the time increments usually at the calendar quarter level of granularity. The release content for each release is usually stated in terms of major product functions, not user stories.
Figure 2. A Product Roadmap
The product roadmap will be validated by discussions with customers on the one hand to get the priorities right, and internal stakeholders on the other hand to confirm the technical and budgetary feasibility of delivering in the advertized timeframes. The product backlog is constantly revised and updated as customer and market requirements continue to evolve.
Release Planning. Release planning is the most challenging part of the agile planning framework. The literature is full of confusing and contradictory advice. So I will attempt to be very precise about what release planning means and how it should be done.
The product backlog should contain a list of customer-facing “features”, where a feature describes a user-observable function of the product. Features are described in the language of sales & marketing staff, product managers and business executives, and are the things we would find in the marketing brochure or product data sheet for the product. Since major product features can rarely be delivered in a single iteration, they are usually decomposed into multiple user stories, spanning multiple iterations. In agile terminology, features are “Epics”. One example of an epic, and its constituent user stories could be the following User Administration epic:
Figure 3. Epics decomposed into User Stories
One of the fundamental differences between agile and traditional software planning is that with agile, the schedule and resources are fixed, and the scope is the one variable to be estimated. In addition to this we typically plan releases around a regular release tempo with, say, a release every 90 days (the releases do not actually have to go out the door). So the planning challenge for the team is simplified: what is the set of features that can be delivered in the next release cycle. It is explicitly understood that the scope the team estimates is going to be just that – an estimate. The planning exercise is helped by the fact that the product feature backlog is prioritized, and the scoping takes this into account. The estimation process does not consist of breaking each user story down into its constituent tasks and estimating effort for each task – that level of planning only occurs at the iteration level. The estimates derived in release-level planning are based on 2 things: a) estimating the relative sizes of each story (in story-points or other equivalent sizing measure, and b) knowledge of the team’s velocity – the number of story-points the team can deliver per iteration. The result of this exercise is a plan that may be presented as follows:
Figure 4. Release Plan
The broader team as a whole must commit to this plan (planning is not a game between the engineers and product manager or the marketing team). The team will next have to allocate individual user stories to each iteration in the release based on known production capacity per iteration (if the team does not have a calibrated velocity, there are some other steps that can be taken to get things moving. This will be covered in another article).
The entire planning process should take 1-2 days depending on the size of the release. This assumes appropriate preparation has been done: Product Owners – groomed and prioritized backlog; Team – Features/Epics broken down into User Stories.
Iteration Planning. Let’s summarize where we are so far. When release planning is complete we should have the following:
- Epics (product features) decomposed into User Stories
- Size estimates done for each story (Fibonacci, T-shirt sizes, etc)
- Stories allocated to iterations based on iteration length and team velocity
The next step is to define the development and testing tasks required to complete each story. To do this you need to have a clear definition of ‘Done’ for user stories. Here is an example:
- Code completed and checked in to the SCM system
- System builds without error
- Product daily or continuous regression completed without error
- Static analysis has no warnings
- Unit test complete and automated
- Code review complete
- Zero outstanding defects against the user story
- Product owner has accepted the story
We go to these lengths because the goal is to get each user story to a shippable state, and not to accumulate re-work. That’s what being agile means. Some people are quite shocked when they hear this. Let me put it another way:
As the team approaches the start of the next iteration, the next level of planning must be tackled. Iteration planning gets us down to a level of detail where each user story is broken into the development and other tasks that must be completed to get the story to ‘Done’. Let’s go back to our earlier example from release planning, and add the required tasks to implement the first user story:
Figure 5. User Stories decomposed into tasks
For each task, we need an estimate (in hours, using 2 days as a guideline for the upper limit for a task), and we need an owner. When the above chart has been completed for all user stories, and we can re-confirm that all of the work will fit within the iteration boundaries, then we have an iteration plan. For tracking purposes, we can track the overall release by burning down story-points, and we can track the progress of an iteration by burning down hours (More on this in a subsequent article).