I thought I would add a few words of clarification on the subject of ‘predictability’. It is now widely understood in the agile community that detailed planning for say, a 12-month project, is futile. How many of us have constructed those nice Microsoft Project pert or Gantt charts, showing sequences of interconnected tasks, only to throw them away a few days later as they rapidly became useless. This is what Agile is not: Up-front specifications followed by development of those specifications in a series of iterations. Agile acknowledges that the world changes rapidly and the game is about building and adapting. A realistic planning horizon is more like 90 days, in which the team will do say, 3 iterations. In parallel with this. the product backlog is being continuously groomed, refined, added to and re-prioritized. There is not necessarily any conflict with ISO/TL-9000 here. Define your process as comprising an up-front plan for delivering , say, 90-days worth of feature development, at the end of which the team adaptively plans for the next 90-days with an updated backlog. Overall, a typical 12-month release, is adaptively planned in 90-day time-boxes, within which shorter time-boxes are used to deliver working increments of functionality.
So now you are up and running with scrum, doing sprints, and delivering valuable functionality at frequent intervals. Furthermore, you team is constantly learning and improving by identifying and eliminating project impediments, and by diligently closing out the actions arising from the sprint retrospectives. Your customers and management teams are cheering and applauding! You may be at or close to, a framework of concurrent planning, development and validation as discussed in the previous article, and represented by the following diagram:
Of course to get a product to market, you need to do many things outside the boundaries of this diagram: contract negotiation, sales and support processes, and there is much in the ISO/TL 9000 standards devoted to these areas of running a business. You should continue to operate these processes as you have previously done, per the definitions in your quality management system. The question now is: how do you run a truly agile “Product Realization” system, while satisfying the requirements of the standards?
Let’s take a look at the key requirements for Product Realization for ISO/TL 9000 as defined in chapter 7 of both standards. For ISO9001 there are 20 individual requirements, and TL 9000 adds an additional 60 requirements (or “adders”) to this list. For our purposes we are going to ignore the sections on Purchasing (7.4) and Control of monitoring and measuring equipment (7.6). This reduces the number of requirements to 11 and 40 for ISO9001 and TL9000 respectively. The sections of both standards we need to concern ourselves about are as follows:
- 7.1 Planning of product realization
- 7.2 Customer Related processes
- 7.2.1 Determination of requirements related to the product
- 7.2.2 Review of requirements related to the product
- 7.2.3 Customer communication
- 7.3 Design and development
- 7.3.1 Design and development planning
- 7.3.2 Design and development inputs
- 7.3.3 Design and development outputs
- 7.3.4 Design and development review
- 7.3.5 Design and development verification
- 7.3.6 Design and development validation
- 7.3.7 Control of design and development changes
Now let’s take a look the typical kinds of artifacts associated with these requirements. These artifacts are the things that an auditor will want to see to verify that you are indeed compliant with the standard. (I’m going to confine this to ISO 9001 for now).
|Requirement||ISO Artifacts||Scrum Artifacts|
|7.1 Planning of product realization||Defined Lifecycle Model|
|7.2.1 Determination of requirements related to the product||Product Requirements Document (PRD)||Product Backlog|
|7.2.2 Review of requirements related to the product||Requirements Review Record|
|7.2.3 Customer communication||Product information, Notification about problems, Problem escalation procedures|
|7.3.1 Design and development planning||Project Plans, Test Plans, Traceability Matrix||Release plans, sprint plans, User Story estimates|
|7.3.2 Design and development inputs||Engineering specifications||User Stories|
|7.3.3 Design and development outputs||System architecture specs, details designs, source code, user documentation|
|7.3.4 Design and development review||Records of design and code reviews, and actions taken|
|7.3.5 Design and development verification||Records of design verification results and follow up actions to address issues|
|7.3.6 Design and development validation||Records of design validation results and follow up actions to address issues|
|7.3.7 Control of design and development changes||Records showing review and approval of all changes|
As we can see, there are very significant gaps in the artifacts required by the scrum process compared with those mandated by ISO 9001, or TL 9000. That is not to say that scrum teams are not actually doing most of what is required. The issue is that in the absence of any ISO-compliant document or record control processes, whatever artifacts do get generated may not meet the standards required by ISO or TL. So now the question becomes how do we generate the required records without overburdening the scrum teams, and taking us right back to where we started, in direct opposition to one of the fundamental principles of the Agile Manifesto!
The ideal solution is to have the required records and data generated automatically as the teams execute their sprints and releases. So now this gets us to the question of tools, and what will really help a team to meet the requirements of standards like ISO 9001 and TL 9000 while staying agile. Here is a list of categories of tools, ordered by the degree of control they provide over project data and records:
- Sticky Notes on a Whiteboard
- Bug Tracking Tools
- Web-Based Agile Project Management Tools
- Web-based ALM Tools with Agile Project Management Capabilities
The best overall class of tool will be one that includes the following capabilities:
- Web-based: available to distributed teams
- Web-based: data available to management and executives, and those beyond the engineering function
- Document control (approval and versioning of key artifacts)
- Record control: secure storage of project data (review reports, test results, change records)
- Requirements Management
- Test Management
- Project Planning (Releases and sprints)
- Project status tracking and reporting (including burndown charts)
- Bug Tracking (or integration with major commercial tools like JIRA)
In the previous article we discussed how transitioning to agile while satisfying the requirements of quality management system standards was a challenge confronting both mature companies and startups. In this second article we will present an evolutionary framework that accomplishes this goal at minimum risk.
Agile Stage 1
First, let’s consider a generic phase/gate model, typical of waterfall organizations (click on the image to see a larger version).
Let’s call this “Agile Stage 1″. I’m calling it Agile Stage 1 because I am assuming that the starting point is a solid process baseline that produces software releases in a consistent and predictable way. You can not build a successful agile framework on top of a sloppy set of development practices. In fact, this is likely to make organizational performance worse.
The central goal of agile adoption is to eliminate or minimize sources of delay in the software production process. In the above process some of the major sources of delay are in the handoffs, internal work queues, and re-work. One of the most obvious items for elimination should be the handoff between development and validation. Why can’t these be done concurrently? Let’s call this Agile Stage 2.
Agile Stage 2
To achieve concurrency, and to do it effectively, requires a number of prerequisites to be in place. These include:
- Continuous integration
- Automated regression testing, and
- Code in a potentially shippable state at the end of each sprint
The validation team begins testing as soon as the first User Story is declared ‘done’ by development. (The triggers and user story state transitions associated with this require good tools. More on this in the third article). Having a potentially shippable increment at the end of each iteration requires continuous integration and testing of all code check-ins. This also implies that all bugs are fixed as soon as they are reported – no accumulating of bug backlogs and associated re-work). The front-end of the process is largely unchanged, and continues to satisfy the requirements of ISO/TL9000. Let’s call this “Agile Stage 2″.
Please not that is not the simple encapsulation of a sprint-based development phase by traditional waterfall. The adoption of continuous integration with automated regression testing of each build, and a demonstrable ship-ready increment at the end of each sprint is a serious step forward on the road to agility. Yes, we still have the overheads associated with the Definition and Launch phases, but these are arguably necessary in any case where you have a large cross-functional organization, engaged in multiple programs, with mature customer organizations that have very formal procurement processes.
Agile Stage 3
Are we there yet? Stage 2 represents major progress, but we still have some fundamentals to tackle. What we are pushing for next is concurrent Planning, Development and Validation, that is, a merging of 3 classical waterfall phases into a single framework. This step requires a broader circle of individuals beyond the boundaries of the engineering function to be fully on-board, and thus requires some fundamental changes in both technical and organizational behavior. One note of caution is worth emphasizing: abandon the “PRD” at your peril! One of the issues constantly raised at sprint retrospectives is a complaint about not having solidly defined user stories at the start of a sprint. This causes the team to make frequent calls back with the product owner for clarifications and more details, which of course adds up to a significant source of delay, the very thing we are trying to eliminate. You cannot replace the waterfall “Planning” phase with a single, time-boxed activity called “Prepare Product Backlog”. The product backlog preparation will be an ongoing process that runs concurrently with sprint execution, and delivers a set of “Sprint-Ready” user stories into the start of each sprint.
Agile Stage 3 could be represented by the following diagram:
Finally, having successfully arrived at something like Stage 3, how do we conform to the requirements of ISO/TL-9000? This will be the subject of the next article: Achieving agility while staying in control, Part 3.
I know, I know. Performing iterations for the development phase only of a waterfall project is not exactly agile, but please bear with me while I give a little more context. Most large successful companies (say the Fortune 500) have been around for a while, and at some point on their road to success have decided to standardize their product development methodologies and lifecycle models. Senior managers responsible for a large portfolio of projects need consistent and accurate reporting of status. Even functions outside of engineering like sales and customer support need this information to be able to communicate release dates with customers and to gear up to support new product releases. Some organizations go even further and seek independent corroboration that their product realization processes have achieved sufficient maturity, and that they are repeatable and provide a basic degree of predictability and quality. Pursuing certification to one of the “Quality Management System” standards is a typical way of accomplishing this. In some cases standards like ISO9001 and TL 9000 are mandated by customers as part of their overall supplier agreements. These standards require companies to define, among other things, their lifecycle development models, and companies that undertake large development projects have typically done this via some form of “Phase/Gate” process.
Now even though many of the requirements of these standards are in no way in conflict with basic agile principles, or good engineering practices, the agile community are going to have a hard time accepting the expectations of auditors around how some of these requirements should be implemented. Let’s take a look at a few examples:
- Determination of requirements related to the product
- Review of requirements related to the product
- Requirements traceability
- Design and development planning
- Test planning
All of the above are basic product realization requirements from the ISO9001 and TL9000 standards. An auditor expects that you have defined how these are done, and also that “records” are available as evidence that you are actually carrying out what you defined in each of your projects. In a typical “waterfall” project the above items are reflected in the following project artifacts:
- Product Requirements Document (e.g. “PRD”)
- Requirements review report
- Requirements traceability matrix
- Project Plan
- Test Plan
|Product Requirements (“PRD”)||Product Backlog|
|Requirements review report||N/A|
This comparison may seem a bit harsh as many scrum teams may in fact do much of what is listed in the left hand column. However, there is no recognized “standard” for scrum (perhaps intentionally so), whereas the items on the left are defined in considerable detail (especially in the case of TL 9000). Even the typical scrum product backlog is going to have significant challenges surviving an audit. A product backlog comprised of sticky notes on a whiteboard is going to result in a major non-conformance due to lack of conformance to the fundamentals of document and record control.
The Transition Challenge
OK, so you are a ISO/TL-certified Fortune-500 company, and you want to become Agile. Or, you are a start-up, and have just inked a deal with a major service provider, but you need to sign a supplier agreement that has a clause requiring you to become ISO- or TL-9000 certified within the next 12 months. What’s the plan?