When I was first introduced to agile some 10 years ago, I was coaching teams in software engineering best practices like code inspections, formal requirements management and design reviews, all of which I had absorbed after many years in large SEI/CMM organizations, and several telecommunications equipment vendors who were either ISO 9001 or TL9000 certified. The mantra at these organizations was “On-time, On-Scope and On-Quality”, and this quest was pursued vigorously using a vast range of practices (and people). Predictability, determinism and perfect control were the key objectives. Initiatives like Six Sigma, and “Zero Defects” (Infinite Sigma!) were the frequent outputs of annual process improvement rituals. ISO and TL auditors were brought in to verify that everyone in the organization was following defined procedures, and the resulting certificates were framed and proudly displayed in corporate lobbies for visitors and customers to be impressed and comforted. Despite all of this, it seemed to me that substantially little changed from year to year, and software continued to be delivered late, with missing features, and unacceptable quality.
I watched the emergence of Agile from a distance, and despite being skeptical of it’s claims, and even outright suspicious of the motives of it’s promoters, I was intrigued. I continued to observe it’s increasing popularity and growing momentum, but still something was holding me back and I could not buy in. I was attending a process improvement workshop at a company where I worked, and sat through a presentation on ‘agile defect tracking’ using Kanban. We were shown a Kanban board with defects recorded on post-its, and columns with the usual ‘To-Do’, ‘In Process’ and ‘Done’. When the presenter finished describing the mechanics of this agile technique, he got an enthusiastic round of applause. The demonstration was for a fairly small project which had to process about 50 software defects throughout the test phase. I asked a question about how the process could be scaled up for larger projects, with say, 1,000+ software bugs, and with the team distributed across multiple geographies and time-zones – a fairly typical situation in many large companies today. I did not get a convincing answer. This presentation reinforced my thoughts about what I was reading in the agile literature – successful application was based on small, co-located teams, doing fairly simple projects (like building shopping carts). Nonetheless the popularity of agile software development continued to grow, and soon teams within companies were seizing the initiative and adopting agile in a bottom-up revolution against the traditional software development establishment. My own company at the time employed a traditional phase-gate development model, where work was passed from one group of functional specialists to the next: product managers > architects > developers > testers, with each gate transition being sanctioned at a ‘Gate Review’ presided over by senior managers. It was the development team who had seized the initiative in this case, but all they had really done was introduce iterations into their development process, and at the same time started using the scrum terminology of user stories, sprints and burn-downs. This was a small step, but one that left the broader product development ecosystem completely intact, and, did little to improve organizational throughput or flexibility.
It is very well known that introducing change in any organization is a serious challenge, and that without the buy-in and active support from senior leadership, major change is virtually guaranteed to fail. In any software company, or any technology company whose products have significant software content, the people in positions of leadership, have invariably grown up with the waterfall and it’s variants, usually accompanied by a phase-gate framework for governance. These people are in leadership positions because they have a successful track record of delivering products to market. They know how to get projects executed, largely on-time, on-scope and with reasonable quality. They have seen it all and have the scars to prove it. They are business focused and do not have the time or patience to indulge the latest process fad. Above all, they know that turning a large organization inside out is a risk that could seriously hurt their company and their careers. Getting the development team bought into doing iterations is probably the easiest part of an agile transformation program in any large company.
Meanwhile, not long after the ‘Agile Defect Tracking’ demo, I had moved to a start-up company, one that had embraced agile practices end-to-end. This company had less than 100 employees, distributed across 3 locations, in Boston, Philadelphia and Beijing. The founders were strong supporters of agile practices, mandating it’s use as a company policy. But it was about much more than doing iterations. A continuous integration system was put in place at the outset, and coupled with a concurrent and highly automated testing process, releases of new functionality were delivered at a steady tempo to customers every 6 weeks. The customers loved the tremendous flexibility of being able to try things out in their lab, suggest changes, and have them provided 6 weeks later. Within a year, the company was acquired by a major communications equipment vendor. The start-up was soon thereafter required to adopt the acquiring company’s phase-gate model and other traditional development practices. However, they not only successfully retained all of their agile practices, but soon became a reference model for agile development throughout the company.
Who This Book Is For
Like any major change or improvement initiative, organizational leadership is a prerequisite for successful agile adoption. However, in order to set organizational direction and goals (or at least to support agile goals proposed by others), executives must invest the time to really understand Agile. There is much to learn, and the world is drowning in an ocean of seminars, literature and advice on agile methods. There are however a small set of basic principles that apply, and when understood, can help leaders provide guidance for implementation activities. Key topics for consideration include:
- Why are short, time-boxed iterations, important?
- What is velocity-based planning and estimating?
- How do we leverage an agile product development framework to accelerate delivery of value and quality to customers?
- How do we use agile to strengthen and deepen the customer relationship?
This book is intended to help senior managers and executives who are struggling with questions related to agile adoption. The aim of the book is to describe the underlying principles behind agile practices, to provide recommendations for making an orderly, incremental transition to an agile framework, and to show how to leverage a newly implemented agile framework for business advantage.