Author Archives: admin

Preface

 

Science, my lad, is made up of mistakes, but they are mistakes which it is useful to make, because they lead little by little to the truth.”

– Jules Verne, A Journey to the Center of the Earth

 

“Our own attitude is that we are chartered with discovering the best way of doing everything, and that we should regard every process employed in the business as purely experimental”

– Henry Ford

 

The truth is that there are no ultimate principles, upon which the whole of knowledge can be built once and forever as on a rock. But there are an infinity of analogues, which help us along, and give us a feeling of power over the chaos when we perceive them. The field is infinite and herein lies the chance for originality. Here there are some new things under the sun.”

– T.E. Hulme

 

 

 

 

Fundamentals of Agile and Scrum

Foundational Agile Practices

In February 2001, a group of software developers met to discuss lightweight development methods. Out of these discussions they identified four key values for agile software development. This has since become known as the Agile Manifesto:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

Supplementing the manifesto, twelve principles were defined that elaborate further on what it means to be agile:

  1. Customer satisfaction by rapid delivery of useful software
  2. Welcome changing requirements, even late in development
  3. Working software is delivered frequently (weeks rather than months)
  4. Close, daily cooperation between business people and developers
  5. Projects are built around motivated individuals, who should be trusted
  6. Face-to-face conversation is the best form of communication (co-location)
  7. Working software is the principal measure of progress
  8. Sustainable development, able to maintain a constant pace
  9. Continuous attention to technical excellence and good design
  10. Simplicity—the art of maximizing the amount of work not done—is essential
  11. Self-organizing teams
  12. Regular adaptation to changing circumstances

Notice that nowhere in either the manifesto or the principles, are specific practices like scrum or eXtreme Programming mentioned.  Both the manifesto and the twelve principles are methodology-agnostic. The message is that it does not matter what specific practices you are using so long as they are consistent with the principles. In fact principle 12: regular adaptation to changing circumstances, or inspect and adapt, is an exhortation not to rigidly follow any specific practice, but to constantly be experimenting with the goal of continually improving your performance.

Today, many organizations practice scrum – the de-facto framework for doing iterations. But doing iterations alone does not necessarily produce deployable code as an output, unless backed up by 2 prerequisite sets of practices, namely Continuous Integration and Test Automation.

Take for example a communications product that resides at the core of a high-speed network. This product not only executes multiple routing protocols, and needs to be highly configurable and manageable, but also interacts with multiple external devices and products, against all of which it must be tested to ensure interoperability and tolerance to many types of adverse events in the network. In order to do this, thousands of test cases must be executed, including functional, performance, stress and interoperability test cases, which traditionally may have taken months to complete. In an ideal world, a full regression would be performed on every change submitted to the build system. At minimum, a very substantial subset of the regression suite needs to be done at least once before the sprint is declared done. A very high level of test automation is required to support this requirement.

But there is another requirement. Traditional construction has involved multiple developers working on private code branches. Changes from each developer are coded/unit tested on their own private branch and then checked into a trunk line from where the next build is produced. New builds must be ‘sanity tested’ for integrity, and then made available to the development team for verification of their changes before the system test team takes over. The process of getting a new build to system test may take several days, and many organizations working in this mode may typically make one new build per week. This is the ‘big batch’ approach to software construction which simply is not consistent with delivering production-ready code in 2-week iterations. Changes from developers must be submitted continuously, and these changes must be of very high quality to ensure minimal re-work, and minimal accumulation of an open bug backlog.

Whereas scrum and be taught and implemented in a fairly short time frame, building an effective CI system, and automating large legacy test suites may require significant financial investment and time to get into place.

In summary, agile software development must comprise elements of:

  • Iterative development e.g. Scrum
  • Continuous Integration
  • Test Automation

Scrum Basics

In traditional, so-called waterfall software development, the starting point for project planning is a product requirements document (a “PRD”). All of the requirements must be defined, reviewed and approved up-front before planning or development begins. Additionally, the development team may produce a software functional specification (“SFS”) that takes each requirement in the PRD and elaborates it into more detail, adding technical requirements and constraints. Both of these documents are then used as the basis of planning subsequent development activities required to deliver the content specified in the PRD. In order to derive a development plan and schedule, every task required to construct and test every feature must be estimated. Once development has been completed, there is a handoff of partially tested features to a test or “QA” organization, followed by several weeks or months of testing and bug-fixing. The length of this phase will be highly dependent on the quality of the code at the point of handoff. Finally, at some point, a decision to release the product to the market must be made, and this is where the usual tradeoffs between schedule, features and quality are fought over. The delivered software is usually accompanied by a set of ‘release notes’, listing all of the features that were actually delivered, plus all of the bugs that the team did not have time to fix.

Generic Waterfall Framework

Generic Waterfall Framework

Software development is referred to as an ‘empirical process’ by advocates of scrum. The output of almost every stage of software development is impossible to predict with precision, and each step produces defects that must be re-worked. This so-called ‘plan-driven’ paradigm has been demonstrated time and again not to be amenable to the level of precision expected from other forms of production such as manufacturing and construction. The throughput of a production line can be specified with a high level of precision and associated quality can be defined at very high levels, such as ‘six sigma’ (less than 4 defects per million units). It is difficult to imagine any large software project coming remotely close to this standard. Agile is a response to this problem.

We do not want to spend much time here listing the disadvantages of traditional waterfall development, however, we should note that the waterfall and its variants have delivered many successful software products over the past several decades, despite all of the known limitations and attendant budget overruns, quality problems, and stressed out development teams. Further, I have worked on many successful waterfall programs that did indeed deliver on-time with high quality. One of the key ingredients of success was building in sufficient instrumentation into the project and getting visibility into emerging risks sufficiently early that sensible tradeoffs could be made to ensure that major priorities could be met. Let us also note that there are many project activities beyond the boundaries of the development team that need to occur for the release of a commercially successful software product. Product development teams in large companies operate within a broader organizational ecosystem that includes multiple other functions which must work together to define, develop, market and support new products. Teams that want to adopt agile development practices must recognize that this cannot be accomplished in a vacuum, and must address how to achieve an effective integration with other inter-dependent functions. Furthermore, optimization of a single function (the software development team) alone will not be enough to deliver the maximum possible benefit to the customer, and a broader approach will be needed to fully realize the advantages of agile and lean.

 Key Process Differences

  • Requirements are not defined in detail up-front. Requirements are continuously elaborated into technical details throughout the project. The goal is not to nail down all requirements in detail up-front, but to expect and accommodate changes as the project evolves. From a business perspective this means that the cost of change is much lower than with waterfall.
  • Release-level planning is based on relative sizing of requirements, together with knowledge of the team’s production capacity, or velocity – not on the time and effort for development tasks
  • Detailed task-level estimating is only performed at the beginning of each iteration.
  • There are no hand-offs between developers and testers, testing runs concurrently with design and coding.
  • Each iteration is completed to an agreed definition of ‘done’, which always includes the requirement that all bugs are fixed within the iteration. The output of an iteration is production-ready code.
  • Iterations are ‘time-boxed’. A 2-week iteration ends in 2 weeks whether all of the planned work is completed or not. This is required to maintain an accurate velocity assessment. The team’s velocity is used as the basis of release-level planning.
  • The team meets daily to synchronize with each other and to highlight any issues that are impeding progress. This process maintains team alignment and drives accountability, while minimizing the need for written reporting and redundant communication. The Daily Standup should take place at the same time and place each day and last no more than 15 minutes. Three basic questions are addressed by each member of the team: What have I accomplished since yesterday, what do I plan to accomplish today, and, what impediments or roadblocks are getting in my way. The session is not intended to be a problem-solving one, and any items that require a more in-depth discussion should be taken outside the meeting. In addition to the three basic questions, the team must also synchronize on their project data – remaining story points, open defect counts and so on, confirming that they are on track to deliver all committed user stories. If the team is co-located and have a dedicated meeting room, then this data can be tracked on a whiteboard. If the team is distributed then the data can be tracked using a spreadsheet or web-based scrum tool and then shared using one of the many collaboration technologies now available.
  • At the end of each iteration, the team demonstrates the results of their work to the product owner and other stakeholders. This provides opportunity for feedback and any course corrections.
  • At the end of each iteration a retrospective review is carried out by the team to identify factors impacting the team’s performance, and to establish opportunities for improvement. This process could be as simple as having everyone on the team answer a three part question: What should we stop doing, what should we start doing and what should we continue doing? This is how the team ‘inspects and adapts’ to continuously improve their processes.

Continuous Improvement

Sustaining an agile transformation requires the transition to a continuous improvement mindset. Absent this, agile adoption, or scrum, (or any other change initiative like Lean, TQM, Six Sigma…) has a high probability of stalling, or even complete failure. Further, continuous improvement must be applied horizontally across the entire organization and not just within functional silos like the software development department. In scrum, the retrospective is intended to provide teams with the opportunity to reflect on their work, and try new approaches in order to improve. This, in principle sounds good, but in practice I have witnessed many retrospectives that were entirely superficial and where no serious attempt was made to dig into systemic issues that had the potential to deliver quantifiable improvements. Meaningful improvement requires data, and the old adage that you can’t improve what you can’t (or won’t) measure is very true. Improvement also requires that teams are willing to recognize problems and confront them. Hiding or ignoring problems is the easy way out, but leads ultimately to failure. This is why I encourage teams to show up at retrospectives armed with data from the sprint just completed. I prefer to see a data-driven retrospective rather than hear comments about what team members feel went well or not well.

Key Role Differences

  • Product Owner: The product owner owns and manages the product backlog.  In addition to traditional product management responsibilities, product owners are an integral part of the team, and are involved in all aspects of the process from release and iteration planning, to daily stand-up meetings, acceptance of completed user stories, and participation in demos and retrospectives.
  • Scrum Master: The scrum master is not the project manager in the traditional sense, but rather fulfills the role of scrum process expert, guiding and coaching the team in all aspects of scrum. The scrum master may lead the daily stand-ups, and may also take the lead in working to remove any impediments identified by the team. In principle any member of the team can assume this role, but basic facilitation skills are essential for success.
  • Team: The team has all of the functions and expertise necessary to deliver a fully shippable increment.

The following diagram summarizes the elements of the scrum framework:

Elements of Scrum

Elements of Scrum

There is a common misperception that scrum is an invitation to abandon process – perhaps through poor interpretation of the Agile Manifesto. An effective implementation requires a level of discipline that some may not fully appreciate, and which most teams will take time to adjust to.

Basic Scrum Framework

Basic Scrum Framework

The generic scrum framework is quite simple and can be learned and put into practice very quickly by small, co-located teams. Minimal tooling is required for planning and tracking releases and iterations (scrumboards in the team standup room, or spreadsheets). A full-time product owner embedded within the team, and available for planning and attending daily stand-ups, means that requirements can be defined and translated into user stories without a lot formality, and changes can be discussed and agreed quickly.

There is one universal framework but no single universal process that can be specified in detail. The situation for each team is unique in terms of product, technology and organization right down to the individuals on the team. Each team is tasked with continuously evaluating the results of their efforts and for identifying opportunities to improve.

Ideal Iteration Length – A survey

Recently I put the question of the rationale for a max sprint length of 30 days to one of my LinkedIn groups. Here are the responses:

  • The idea is that anything over 30 days is too large to effectively break down and estimate properly and for everyone to keep that model in their head. It also keeps you focused on the quick evolution of the team to learn through regular small mistakes, instead of remembering what went wrong months ago.
  • The idea is to fail fast and more than 1 month is not fast enough
  • Short sprints and small user stories provides the ability to test early, deliver usable functionality incrementally in small batch sizes, with lower risk.
  • Delivering working software in every sprint over 30 days is called a project. If you keep in mind that the content of the sprint is frozen and that a scrum team should be around 7+-2 people you freeze 7-9 man months or more of work in advance without allowing the customer or product owner to have a say or see a working product in between. This is a huge investment with high risk of error. It also prevents you from re-evaluating your way of working often and to review your failures and learn from them continuously.
  • At the beginning, the requirements for that Sprint have been frozen. So the question of Sprint length has also to do with the ability of the Product Owner and Scrum Master to keep these requirements untouched by external influences and to negotiate new requirements onto the Product Backlog for review on the next planning meeting. If you are getting a lot of ‘this just can’t wait’, then shorter Sprints are better.
  • You lose the pressure if you start a sprint longer than 30 days. I’ve experienced a 10 days sprint as the best. Try a discussion about changes with the team after you’ve estimated all tasks.
  • 10 days is the right amount. In addition, this helps with user exceptions for urgent changes or issues. It’s an easier sell if I tell them this sprint is set (or even the next one). At the most they typically only have to wait 3 weeks for us to get started on something new for them.
  • A shorter sprint duration reduces risk. I would add that shorter sprints force more frequent synchronization and convergence of the different work streams occurring within a project. If there is any incompatibility between, say, modules of code it would be discovered sooner (and presumably be fixed sooner) thus reducing the risk of having a much bigger incompatibility problem that might arise from longer sprints. It’s better to fail fast because big problems discovered later than sooner are a recipe for blowing the schedule.
  • We have tried two week sprints and three weeks sprint and the feeling was that three weeks was the perfect length. When we did two week sprints it felt like we were always starting or ending and the teams stress level was too high.
  • This also comes down to how the human brain works. If you have a term paper due at the end of the semester, you tend to start working on it at the end of the semester. If you had an outline due three weeks into the semester, you’d start working on your Term Paper two weeks into the semester. Slicing into chunks (sprints) helps to prevent all the work piling up at the end. This goes straight to the Agile Principle of “promote sustainable development.”
  • My Agile coach always recommends starting with one week sprints for new Scrum projects. When you are just starting, how you use Scrum is as much or more important than what is being built. With one week sprints you have a shorter time between learning cycles to adjust things like estimating, how many story points the team takes on and so on.
  • There are three reasons. Two are explicit in Scrum the other isn’t mentioned, but is one of the foundations on which Scrum is based. The explicit ones are feedback and enforced view of reality. The second is removing delays.
    • You want quick feedback. Longer than 30 days does not force you to get it. But feedback exists in many forms. There is the time from getting a request until delivering it. Time from starting a story until it is complete. This is where lessons are learned quickly. I personally don’t like to fail fast, I prefer to learn fast. One does not need to write code to discover the customer doesn’t want it, one needs to write a little where the customer is clear, show it to them, and enable the customer to learn a little bit more. But, if you are going to fail, do it quickly.
    • By enforced view of reality, I mean that things will show up to make it difficult to deliver value in 30 days. For example, customers may not want to talk to you that often, build processes may be inefficient, testing and coding may be too separated from each other. By having a time-box in which your work is intended to get completed, those things that work against you will become more visible and painful. These are impediments to your work actually, and instead of avoiding them by making the sprint length longer, you actually need to shorten the sprint to expose these even more. It is also harder to ignore them – if you haven’t tested your code by sprint’s end – it’ll be obvious and irrefutable. Both of these reasons actually tend to call for even shorter than 30 day sprints. Most successful Scrum teams I know of use 1-2 week sprints. In fact, I’d go so far as saying teams with 3-4 week sprints are showing a “smell” of not being engaged enough in solving their problems (not always true, but often true).
    • The third reason, which in many ways is the biggest one, is removal of delays. Since about 2004, I have been claiming that Scrum is a weak implementation of Lean principles. I say “weak” because Scrum does not deal with optimizing the whole and it leaves out a lot of lean-education and management. But one of the key tenets of Lean is to remove delays so value can be delivered quickly. This is critical because delays between steps of work literally create more work. The time-boxing of Scrum requires teams to complete their work quickly. This all reduces delays and therefore avoids incurring work that doesn’t need to happen.
    • The combination of feedback, awareness and removal of delays drives us to have shorter feedback loops until the overhead of the sprint overcomes the value achieved by them. For most teams this will be 1-2 weeks. Some teams that discover they don’t need the discipline of the time box will abandon it completely and move to Kanban. I might add that a simple value stream analysis will show most that “the shorter Sprint the better”. Scrum contains no technique or method for optimizing end-to-end, and it should not. The retrospective might uncover such a problem, but I generally advise to use Lean thinking to address end-to-end optimization explicitly.
  • 30 days is also fairly typical ‘management reporting interval’. A Sprint longer than 30 days means that management may not get a ‘status update’ for two months.
  • With experienced teams and a well-defined product backlog, a 30 day sprint may be fine (not my preference). But when the teams are newly formed, new to Scrum or when the product backlog is very dynamic, it’s better, as someone pointed out, to fail earlier and adapt sooner.
  • A two-week sprint is my preference. Just long enough to develop some rhythm and velocity, but not so long that you risk going down the wrong road for a month.
  • 30 days matched traditional development teams that were new to scrum, or where older technology was not nimble enough for rapid development for all the mentioned reasons especially quick review and feedback. Even with a 30 day sprint cycle, I have usually obtained feedback in shorter cycles before being fully accustomed to scrum. Maybe as technology and teams get more progressive we will see shorter sprint cycles.
  • All above answers are great. I will amend that by having frequent and not too far away reviews that show what was built (the increment) then you are being transparent and providing the visibility to the stakeholders, so everything that was mentioned (risk, done, value) are all observed and demonstrable. Also by repeating the cycle and having a time for inspect and adapt you can become agile. If you do them more than 30 days the people may not remember what happened and will not effectively adapt. BTW 30 days is too much for us, my experience that most teams use a 2 week (10 days) sprints.
  • Start with sprints as short as possible… one week or two week sprints. Do not overpromise the product owners but in first place under-promise as you need to learn on what you can deliver on shippable products. If one week or two week sprints are not possible to deliver shippable products you can extend the dime of a sprint to for example 3 weeks or 4 weeks.
  • A 2 week sprint has effectively just 9 days to build and test deliverables as you also need to reserve time for backlog grooming, sprint planning, sprint review and retrospectives. When you start your first sprint just under-promise.

The Lean Roots of Agile

Agile software development has its roots in the lean manufacturing paradigm developed at Toyota – the Toyota Production System (TPS). Lean manufacturing centers on creating more value with less work. One central principle of lean manufacturing is to relentlessly work on identifying and eliminating all sources of waste from the manufacturing process, and also to eliminate all obstacles that impede the smooth flow of production – usually done using a lean technique known as Value Stream Mapping.  ‘Waste’ can take many forms but it basically refers to anything that does not add value from the perspective of your customer. Taichi Ohno, an executive at Toyota, identified seven categories of waste:

  • Defects
  • Overproduction
  • Inventories (in process, or finished goods)
  • Unnecessary processing
  • Unnecessary movement of people
  • Unnecessary transport of goods
  • Waiting

Central to the lean philosophy is making improvements in real time: identify a problem, measure it, analyze it, fix it and apply it immediately on the factory floor (or in the next iteration) – don’t wait to read about it in a report later.

It is fairly easy to come up with software examples for each of the above categories of waste (example: reduce ‘Waiting’ by eliminating hand-offs, or reduce ‘Overproduction’ by cutting out all unnecessary features). Lean principles will thus sound very familiar to agile practitioners. In their book, Lean Thinking, the authors Womack and Jones, identified five principles of “lean thinking.”

  1. Value (Definition of): Specify value from the customer perspective.
  2. Value Stream: Analyze your processes and eliminate anything that does not add value for the customer.
  3. Flow: Make process steps flow continuously with minimal queues, delays or handoffs between process steps.
  4. Pull: Build only in response to specific requests from customers. In other words organizations should not push products or services to customers. Order from suppliers only the material or services required to supply specific customer requests.
  5. Perfection: Continuously and relentlessly pursue perfection by reducing waste, time, cost and defects.

There is no clear written description of Toyota’s practices, however there is a lot of written material about TPS  – most notably  The Machine That Changed the World, by Womack, Jones and Moos – now a management classic.

Several authors have boiled down the definition of Lean into Two Pillars:

  • The practice of continuous improvement
  • The power of respect for people

Continuous Improvement means the relentless elimination of waste in all it’s forms, and, the identification and removal of anything that disrupts the continuous flow of the process.

“Respect for people” is a pretty vague term, but at Toyota it means something very specific, including providing people with the training and tools of improvement, and motivating them to apply these tools every day. At Toyota they would say: “We build people before we build cars”.

To summarize TPS even further in a single sentence: Toyota practices continuous improvement through people.

Optimum Batch Size

From the late 1940s Toyota was experimenting with batch sizes and in particular with die changes associated with stamping out steel metal parts for cars. During this time the important discovery was made that it actually costs less per part to make small batches of stamped parts than – unlike their American competitors – to produce enormous lots on a large scale. There were two reasons for this: small batches eliminate the need to carry huge inventories of parts required for mass production, and making small batches means that any defects are discovered much earlier. The implications of the latter issue were enormous – and have a direct correlation with agile software development – it made those responsible for stamping much more concerned with quality, and it eliminated re-work and the waste associated with defective parts.

For more on the history of Lean Production, I refer to readers to The Machine That Changed the World, the essential reading reference on this topic. In the book: The Principles of Product Development Flow, Donald Reinertsen sets out a dozen ‘principles’ that promote the merits of smaller batch sizes. In the context of software development, Waterfall executes a software release in a single batch, whereas scrum breaks up the requirements for the release (the Release Backlog) into a set of smaller batches, and implements these in a sequence of Sprints. The question we have is: what is the optimum size for a sprint?

Much research has been done on quantifying the optimum batch size in the context of manufacturing logistics. The calculation of ‘optimum’ batch size – usually called Economic Batch Quantity (EBQ) – is based on the relationship with demand, machine set-up cost, and inventory carrying cost per item.

Specifically:

EBQ = Sqrt(2*Demand*Setup Cost/Inventory Holding Cost per Unit)

Where the inventory holding cost is the overhead cost associated with each item, and generally increases with the size of the batch, and the setup cost per item generally decreases as a function of batch size. This can be illustrated in the following chart:

Optimum Batch Size

Optimum Batch Size

For manufacturing operations the impact of large batch sizes on cost and throughput may be fairly obvious: large batches increase work-in-process inventory levels, materials lead-times, and finished good stock levels. But more importantly, large batches drive more variability, and more variability means more re-work, more cost and delays.

We will look at exactly what this means for software, but before that, what we really care about is maximizing throughput, i.e. how do we maximize the number of User Stories (or Story-Points) delivered per sprint – known as the velocity in scrum? The throughput chart is simply the inverse of the cost chart above, and will be an inverted U-Curve.

Let’s separate the production costs of User Stories into direct costs – those that contribute directly to the definition, design, coding and testing of each story, and indirect costs – those that represent overhead, or non-value adding activities.

Direct Costs

Direct Cost per User Story

Direct Costs per User Story

Indirect Costs

Indirect Costs Per User Story

Indirect Costs Per User Story

The fixed costs are associated with the fundamental development tasks that must be carried out to deliver the set of User Stories in the iteration: design, coding and testing. Building 100 user stories in a batch as opposed to building 1 at a time yields obvious economies of scale. (Think of test automation as an example). However, as the size of an iteration goes up, so too does the amount of indirect cost, examples of which include dealing with more complex branch merges, fixing broken builds, scrubbing long lists of bugs, plus all of the data collection and status reporting that goes with this – more control and more project management overhead. Requirements re-work and bug-scrubbing could be considered as WIP to make an analogy with manufacturing. The simple fact of having to deal with an iteration of larger scope and complexity drives up the amount of overhead one encounters in a project. Thus the overall cost/story vs. iteration size graph will be quite similar to that in Figure 1.

The larger the batch, the fewer opportunities there are to inspect and adapt, and the more we are exposed to WIP and re-work. It is up to each team to find their own sweet spot for the size of an iteration. In scrum, we talk about getting to an ideal ‘velocity’, or the optimum number of Story Points per Sprint. Teams should relentlessly pursue this objective by searching for and eliminating all sources of waste and delay in their development process.

References, Recommended Reading

  1. James P. Womack, Daniel T. Jones and Daniel Roos, 2007. The Machine That Changed The World: The Story of Lean Production – Toyota’s Secret Weapon in the Global Car Wars. Free Press.
  2. James P. Womack and Daniel T. Jones, 2003, Lean Thinking – Banish Waste and Create Wealth in Your Corporation, 2nd Edition. Free Press.
  3. George Koenigsaecker, 2013. Leading the Lean Enterprise Transformation, 2nd Edition, CRC Books.
  4. Philip Kirby, 2015. The Process Mind: New Thoughtware for Designing Your Business on Purpose, CRC Press.
  5. Donald G. Reinertsen, 2009. The Principles of Product Development Flow, Celeritas Publishing.

 

Organizational Alignment

Agile software development has its roots in lean manufacturing originally devised by Toyota in the Toyota Production System (TPS). There are many definitions of Lean, but most of them describe it as a transformation to a culture of continuous improvement that seeks to relentlessly eliminate waste from everything the organization does to deliver products or services to customers, while at the same time working to put in place a continuous process flow for the delivery of value to the customer. There is an important caveat however that needs to be highlighted: sustainable performance improvement cannot be achieved without addressing the entire business workflow – Lean is about working across the entire horizontal business workflow – not optimization of functional silos.

 

Production-Ready Code at All Times

The entire agile testing philosophy is based on the goal of having ship-ready code at the end of each iteration, where ‘ship-ready code’ means: 100% feature tested, 100% system/performance/stress tested, and zero open bugs.

This sounds like a very tall order, requiring a fundamentally different approach to the traditional practice of development hand-offs (typically of half-baked code), to the test organization. There is simply not enough time for this approach to work if we accept the above-stated objective.

Feature and system testing must happen concurrently with development, and for this to work, the development team must be making available nothing but clean code to the testers.

Agile vs. Waterfall

Agile vs. Waterfall

This chart underscores the fundamental difference between agile and waterfall in terms of the test and bug-fix approach. In agile there are no hand-offs, no phases and no large backlogs of bugs to fix as a release nears the end of its development life-cycle. Each iteration strives to deliver an increment of bug-free, production-ready code as the output of each iteration.

Let’s break this down to see what is required for this to work in practice. Agile testing starts as soon as the first User Story is declared done (not at the end of the sprint!). But for this approach to have any chance of success, re-work must be minimized. By re-work we mean the traditional test and bug-fixing cycle, characteristic of waterfall development, that starts with the hand-off from development to the test organization. Thus, the definition of done must include:

  • Code clean compiles with all static analysis warnings removed
  • Code reviewed, with all review issues resolved
  • Story has been unit tested, and ideally units tests are automated
  • Test coverage based on unit testing meets some minimum threshold
  • Code and automated unit tests checked into build system, and system builds and passes all unit tests
  • Build passes all predefined build tests

Next, the test team verifies the user story based on its defined acceptance criteria. The majority of stories should be passing at this point. The manufacturing analogy is the production ‘yield’, and we should be striving for the highest possible yield, say > 90%. If the yield is low (and the corresponding re-work high), then we need to dig into the reasons for this, identify root causes, and apply corrective actions to drive the yield higher. Clearly, this will not happen overnight, and may require multiple iterations, if not releases, to get there. There are a couple of additional prerequisites that go along with getting to a high first-pass success rate:

  • A continuous integration environment with a high degree of automation of both the unit test and build sanity level
  • A high degree of system test automation
  • A continuous improvement mindset where the team routinely dissects test failures and institutes actions to push the bar higher for first-pass test success.

 Continuous Integration

One of the fundamental goals of agile development is to have deployable code at the end of every iteration. Working backwards from that challenge implies that a number of technical practices need to be in place. These technical practices need to support the organization’s definition of ‘done’ at both the story and sprint level. For example:

User Story Done Criteria:

  • Story designed/coded/unit tested
  • Unit tests automated (Why? See below)
  • Tested code checked in and built without errors:
    • Static analysis tests run and passed
    • Automated unit tests run and passed
    • (Unit) test coverage measured and meets acceptable threshold
  • Independent validation of user story by QA team
  • User story acceptance criteria met
  • Zero open bugs

Sprint Done Criteria

  • All user stories done
  • All system tests executed and passed
  • All performance/stress tests executed and passed
  • All regression tests executed and passed
  • Zero open bugs

How on earth are we expected to accomplish all of this in an iteration lasting a maximum of 2-4 weeks? To make all of this happen, a number of practices must be in place:

  • There is no ‘hand-off’ from the developers to the testers. Story acceptance testing runs concurrently with development. The QA team can begin testing as soon as the first user story has been delivered cleanly through the build system.
  • Re-work must be absolutely minimized. There is simply is no time for the classical back-and-forth between QA and development. The vast majority of user stories must work first time. This can only be accomplished by rigorous unit testing.
  • System-level regression and performance testing must be running continuously throughout the iteration
  • Test cases for new user stories must be automated. This requires resources and planning.
  • All changed code must be checked in, built and tested as frequently as possible The goal is to re-build the system upon every change.
  • Fixing of broken builds must be given the highest priority.

When all of the above is in place we have something referred to as ‘Continuous Integration’. A typical continuous integration configuration is summarized in the following diagram.

Continuous Integration

Continuous Integration

In this system we have setup a CI system such as Hudson – an open source CI tool. Hudson integrates with other CI-related tools from multiple vendors, such as:

  • SCM Systems: Perforce, Git
  • Build Tools: Maven, Ant
  • Unit Testing Frameworks: Junit, XUnit, Selenium
  • Code Coverage Tools: Clover, Cobertura

Hudson orchestrates all of the individual sub-systems of the CI system, and can run any additional tools that have been integrated. Here is a step-by-step summary of how the system works:

  1. Developers check code changes into the SCM system
  2. Hudson constantly polls the SCM system, and initiates a build when new check-ins are detected. Automated units tests, static analysis tests and build sanity tests are run on the new build
  3.  Successful builds are copied to an internal release server, from where they can be
  4. loaded into the QA test environment. The QA automated regression and performance tests are run
  5. Test results are reported back to the team

Knowing that every change made to an evolving code-base resulted in a correctly built and defect-free image is invaluable to a development team. Inevitably, defects do get created from time to time. However, identifying and correcting these early means that the team will not be confronted with the risk of a large defect backlog near the end of a release cycle, and can be confident in delivering a high quality release on-time.

Setting up a continuous integration system is not a huge investment, and a rudimentary system can be set up fairly quickly, and then enhanced over time. The payback for early detection and elimination of integration problems and software defects dramatically outweighs the costs. Having the confidence that they are building on a solid foundation frees up development teams to devote their energies into adding new features as opposed to debugging and correcting mistakes in new code.

Continuous Testing

So far we have discussed a framework for a continuous integration system that includes executing a suite of automated unit tests on every build and using the results of that testing to determine whether the build is of sufficient quality to proceed with further development activities. Ultimately though, we need to have a test environment that assures us that at the end of every iteration we have a production quality product increment.

If we go back to our simple VOD system example from the last chapter we may realize at this point that we could be facing some significant challenges. The goal is to deliver an increment of defect-free functionality with each iteration. To accomplish this requires:

  • Mature software engineering practices that produce clean, reliable code at the unit and individual user story level.
  • An automated unit test framework that grows with every code check-in and can be run as part of a build validation process.
  • Build metrics such as test coverage that help ensure that the growing code base is comprehensively tested.
  • The ability to create and execute a set of new feature test cases at system level within the boundaries of a single iteration.
  • A suite of automated regression test cases that can be run continuously or at least on the final build of an iteration to ensure that work in creating new functionality has not broken existing system behavior, performance  or stability.

For systems of even moderate complexity and those being engineered by multiple teams, this may be neither a practical or economically feasible proposition. In this case the question now becomes what are the priorities and what tradeoffs must be made. In the VOD system example we might have an independent System Integration and Test (SIT) team to takes the output of each development iteration and subjects it to a combination of feature and regression testing. In this structure the output of a development iteration gives us and increment which is:

  • 100% unit tested, unit tests automated
  • 100% user story tested at the subsystem level

The increment is then picked up by the SIT team who subjects it to:

  • 100% new feature validation, end-to-end on a fully integrated system
  • X% regression test coverage, where X represents the highest priority, highest risk aspects of the system.
System Integration and Test

System Integration and Test

The result: A ‘High Quality’ increment as the output of each SIT cycle. If the teams are using say, a 3-week iteration cycle, this gets us a high quality increment every 3 weeks. If the team cannot produce something they can call ‘High Quality’ with every iteration then they should consider adjusting the iteration length accordingly. If these intermediate increments are being delivered to customer labs for trial of new functionality then this can still be done at relatively low risk. The final SIT cycle can be extended to provide 100% regression test coverage. If a decision is made to deliver one of the increments before the end of the planned release, this can be done by extending the SIT cycle for that increment to provide full test coverage. Over time, the SIT should be working aggressively (backed up of course by appropriate resourcing and funding from their management teams) to maximize the automation of their regression test suites. Some categories of testing, by their nature, require lengthy test cycles, for example, stability testing or testing of high availability system configurations. Other types of tests, for example those that require direct human involvement like video quality validation, are not easily or inexpensively automated, Nonetheless delivering a high quality product increment is a goal that should be in reach of most organizations.

 

 

 

 

 

Tracking and Reporting

The project is underway and I need a feature-level view of where we are. Burn-down charts for user stories are great but do not give me the business-level picture. Burn-down charts calibrated in hours are dangerous – you can burn-down lots of hours of effort but accomplish nothing! Completed hours of effort get counted, story-points for unfinished user stories should not be counted. This is why project tasks masquerading as user stories should be ruthlessly rooted out.

My team signed up to deliver 10 major features to a customer, and I want to know 2 basic things:

  1. How much of the feature development work has been completed?
  2. Are we on the right trajectory to complete the project on time?

The first thing I want to see is a big picture status summary for feature development status. Something like:

Feature Completion Status

Feature Completion Status

Note that the data used to build this chart has been filtered clean of all non-customer-facing “user stories”. This chart represents delivered “business value” and not just work done.

The basic takeaway from this chart is that:

  • 6 out of 10 features are 100% complete
  • 3 are partially done, and
  • 1 is not yet started

Next, I want to drill down into the partially completed features to understand what risks we might be facing. For this, I need a chart along the following lines:

Feature Status By User Story

Feature Status By User Story

In this chart we have a breakdown of each feature into it’s constituent stories, and the status of each of those stories, where B = Backlog, D = Defined, P = In-Progress, C = Completed, and A = Accepted. Additionally we have the total planned story points for the feature, the number accepted and the resulting percent complete (Accepted Points/Planned Points). Feature Six immediately stands out because it looks like no planning has been done for this feature, and thus we may want to explore that further to determine the risk of having to exclude it from the release, or taking some other remedial actions to get it back on track.

Now let’s take a look at a release burndown chart to see what that might tell us. I have rarely seen a a nice clean textbook burndown chart on a real project. They typically might look like the following:

Release Burndown

Release Burndown

Key Points:

  • Project Start = March 18
  • Target Finish = July 01
  • Initial scope: ~ 1400 story points
  • Additional scope to the tune of an additional 600 points added through 1st week in June
  • Team has completed ~ 1000 story points, approx. 350 per month
  • Team has approx 1000 points remaining, ~ 3 months. This implies a mid-September finish
  • Team options: move release date to mid-September, or, reduce scope by approx. 600 points

Of course, action should have been taken much earlier to either contain the scope additions or acknowledge a later finish date. The overall point is that the release burndown chart is a powerful tool to help teams forecast progress and the likelihood of finishing on time. It can also be used to make clear the implications of scope changes on schedule.

Here is another way to assess progress:

Scope Completion Trend

Scope Completion Trend

This chart paints a more positive picture:

  • Consistent progress is being made from month to month
  • Work-In-Progress (WIP) is stable
  • Teams appears on track to complete all work within about 1 more month.

It is very much worth the effort to pull this kind of data together so that team’s have visibility of overall project progress on a daily basis.

 

Iteration Planning

Let’s summarize where we are so far. When release planning is complete we should have the following in place:

  • A ranked set of product features
  • Features elaborated into User Stories. (At least enough stories to feed the first few sprints)
  • Size estimates done for each User Story (in Story Points)

The next step is to define the development and testing tasks required to complete each story. To do this you need to have a clear definition of ‘Done’ for user stories. Here is an example:

  • Code completed and checked in to the SCM system
  • System builds without error
  • Product daily or continuous regression completed without error
  • Static analysis has no warnings
  • Unit testing complete and automated
  • Code review complete
  • Zero outstanding defects against the user story
  • Product owner has accepted the story

We go to these lengths because the goal is to get each user story to a shippable state, and not to accumulate a pile of partially completed work leading to re-work or ‘technical debt’ in agile jargon. That’s what being agile means. People new to agile development are frequently surprised when they learn about the degree of rigor required to deliver a user story – done means done – ready to ship.

Iteration planning gets us down to a level of detail where each user story is broken into the development and other tasks that must be completed to get the story to ‘Done’. Let’s look at an example, and add the required tasks to implement a user story which is part of a User Administration feature:

User_Story_0100: Users can self-register

    • Task_8001: Create new page with registration form
    • Task_8002: Create user class to support all required data
    • Task_8003: Create methods to collect user data and insert into database
    • Task_8004: Create database schema to support user details
    • Task_8005: Code review of all new and modified code
    • Task_8006: Create and execute unit tests for the story
    • Task_8007: Add unit tests to the unit test automation library
    • Task_8008: Create acceptance tests for the story
    • Task_8009: Execute acceptance tests
    • Task_8010: Acceptance test automation

For each task, we need an estimate (in hours, using 2 days as a guideline for the upper limit for a task), and we need an owner. When this data has been determined for all user stories, and we can re-confirm that all of the work will fit within the iteration boundaries, then we have an iteration plan. For tracking purposes, we can track the overall release by burning down story-points, and we can track the progress of an iteration by burning down either story points or hours.

The output of an iteration planning session might look something like:

Iteration Planning

Iteration Planning

Arguably, once a team has established a stable velocity, the effort-hours task-level estimates could be dispensed with, that is, once a team becomes good at estimating in story points the additional step of estimating tasks in hours may no longer be necessary. The breakdown of stories into tasks at the iteration level is still required so that the work can be defined and allocated. But all that really matters is that the team has a high probability of delivering all required stories within the sprint boundary.

Let’s say we have decided to tackle 6 of the so far defined 11 user stories from the release backlog in the first iteration. These stories amount to 18 points, definitely within the team’s established velocity of 20 points. Our updated plan for the next iteration might be represented as follows:

Iteration Plan

Iteration Plan

Representing the iteration plan this way helps us to maintain good visibility of the relevance of these stories to the features they  are derived from. While this iteration is being executed, work continues on refining and elaborating on the remaining features in the release. It is a good strategy though to try and completely finish individual features before proceeding to new ones. This helps minimize feature-level WIP, and supports the efforts of system- or integration test teams to completely validate entire features end-to-end.

 

Velocity-Based Iteration Planning

The output of the release planning step is a prioritized list of stories plus size estimates in story points. Velocity-based planning proceeds in 2 steps:

  1. Select user stories in priority order from the release plan and assign them to the iteration. Continue assigning stories until the total story points assigned matches the team’s velocity.
  2. Decompose each user story into the tasks required to develop and validate it, paying attention to the tasks required by your definition of ‘done’. In short, include every task needed to get a user story to a shippable state. Team members sign up for the tasks.

This approach assumes that the team has a good track record of accurately sizing stories and scoping iterations to match their velocity.

Scaling with Multiple Teams

In the case where we have a more complex system under development, requiring a large level of engineering effort, then we will need to distribute the work between multiple teams. There are many ways to do this, but let’s look at a simple example. Consider a Video On-Demand (VOD) system that resides at the ‘head-end’ of a cable TV operators network, and gives subscribers the ability to browse, purchase and stream stored TV shows and movies to their TV’s, smartphones, tablets and PC’s. A simplified view of the architecture of this system could be represented as:

A VOD System

A VOD System

The system has 3 major subsystems:

  1. Content Ingest Subsystem
    • Ingest content from content providers and store on storage arrays
    • Create metadata (Title, Genre, Year Released, Running Time, Production Studio, etc) for each ingested asset – required for content catalog creation.
    • Transcode content  – convert content for delivery to multiple device types
    • Encrypt content – provide protection of content against unauthorized use.
  2. Content Publishing Subsystem
    • Create catalogs of video products that users can browse and select for viewing
    • Ensure users only see content they are entitled to
  3. Content Delivery Subsystem
    • Deliver content in different formats to various user devices
    • Use different delivery protocols depending on user devices and available network bandwidth

The most straightforward approach would be to have one team working on each subsystem.  This is is not a bad arrangement because each of the three subsystems operates almost independently of the others, sharing data via a database. Furthermore, each subsystem can be tested fairly independently, for example it is fairly easy to see how validation of the accuracy of asset metadata creation could be tested on the ingest subsystem without any dependency on the Content Publisher or Content Delivery subsystems.

Going back to the requirements management discussion, with this example part of the process to get features to the ‘Defined’ state would be to identify which system components needed work to deliver the overall feature. When done for all features in a release, the output of this exercise might me summarized as follows:

Requirements Allocation to System Components

Requirements Allocation to System Components

With multiple teams, our revised iteration plan is going to look something like:

Iteration Planning with Multiple Teams

Iteration Planning with Multiple Teams

Note that when planning iterations, it is important to get the highest ranked features built and demo’d as early as possible – at least the ‘core’ stories for those features (leave the bells, whistles and exception-handling to a subsequent iteration). This provides the opportunity to get early feedback not only on what has been built, but also on potential re-prioritization or further refinement of the remaining features.

 

 

Adaptive Release Planning

Agile release planning can be approached in a number of ways depending on the nature of the organization’s business. If the organization is developing software to a customer contract then the contract is the backlog. The team will typically have little flexibility with scope and will likely be held to delivering a specific feature set at a specified cost within a defined schedule. In this case the team may need to estimate the entire feature set up-front. However, contracts can be structured to deliver subsets of the total scope using, say, quarterly releases. Most software businesses are however roadmap-driven, that is, product capabilities are set as goals in a multi-year plan, with periodic product releases planned to deliver subsets of the roadmap to market. In this kind of environment, planning would be carried out at three levels – Roadmap, Release and Iteration. One way of looking at the product roadmap is to consider it to be a high-level plan for how the product backlog gets delivered over say, the next twelve-month period. Release plans are derived from the overall product backlog, and iteration plans are derived from release backlogs. The following diagram illustrates this.

Agile Planning Levels

Agile Planning Levels

The Product Backlog is a release-independent list of every conceivable requirement, feature request, enhancement, and wish-list item for the product. This list is derived from a variety of sources, including, product management, sales and marketing, customers, engineering, customer support, and virtually anyone who feels they have a stake in the product. For reasons of budget, engineering resources, market windows, and other constraints, not all product features can (or should) be delivered in a single release. Therefore a plan is required showing how features in the product backlog will be brought to market over a series of product releases. This high-level plan is usually referred to as the product roadmap. The product roadmap shows how major new product capabilities get rolled out over time. The product roadmap is the product backlog turned on its side. The time increments for each release are ideally based on a regular tempo or fixed timebox – just like scrum iterations – with perhaps a release every calendar quarter. The release content for each release is usually stated in terms of major product functions, not user stories.

The product roadmap will be validated by discussions with customers on the one hand to get the priorities right, and internal stakeholders on the other hand to confirm the technical and budgetary feasibility of delivering in the timeframes targeted. The product backlog is constantly revised and updated as customer and market requirements continue to evolve.

The starting point for release planning is a set of features proposed for the release. Initially, these may simply be high-level statements about the general intent of each feature, however it is important that the features be ranked in order of business importance. Next, beginning with the highest ranked feature, we work to flesh out the details. This step is a collaboration between the product owner, the product architect and the development team (at least those individuals from the team that can participate effectively in this exercise). This exercise could be done by getting everyone together in a room for 1 or 2 days, with whiteboards and post-its, or, it could be done more asynchronously using tools that support collaboration. In the case where we have a highly complex system and each feature requires a detailed analysis and subsequent review, then this is going to take longer. The overall goal is to get each feature into what we described earlier as the ‘Defined’ or ‘Development Ready’ state – one from which development teams can start creating user stories and test cases. Do we need to wait until all features are ‘Defined’ before proceeding with user story creation? Of course not. User story creation should begin as soon as the first feature is defined.

Agile planning is designed to be adaptive. The principles behind this approach are:

Adaptive Planning & Grooming

Adaptive Planning & Grooming

  • Plans should only be as detailed as the depth of information currently available
  • Planning must be continuously adjusted as project knowledge improves.

1. Get Features Defined

Release Planning 1

Release Backlog in Evolving Stages of Refinement

As a prerequisite, begin the process of getting features properly defined as described in the last chapter. ‘Defined’ (or, ‘Development Ready’) means that there is sufficient information for development teams to begin creating user stories and test cases.

Creating user stories from poorly defined requirements amounts to waste, and teams must adhere to the discipline of getting requirements accurately stated before proceeding with development. Unlike classical waterfall, development can begin as soon as enough features are defined to feed the first one or two iterations.

Refinement of the release backlog to get features into a defined state and to re-rank items as circumstances change, is a process that should be done continuously throughout the release.

2. Create User Stories

Create User Stories

Grooming the Backlog

For those features that are sufficiently defined, break them down into iteration-sized user stories, beginning with the highest-ranked features.  User stories should be small enough to be deliverable in a single iteration. Thus, each story needs acceptance criteria. As input to the release scoping process, stories need to be sized . The use of ‘story points’ – a technique for establishing the relative sizes of each story – has become a popular practice for this step. (For more on story points see below). When user stories for all features have been defined and sized, we are ready to set the initial scope for the release. Note, for large programs it may be unrealistic to plan every feature into this level of detail in a single step – that’s the waterfall approach. However, for fixed-price contracts, a team may need to come up with an initial, non-binding, estimate that is firmed up over time. One approach may be to only commit say, 90-days (6 * 2-week iterations) of functionality in a single cycle, and deliver all of the required features via a number of these 90-day cycles.

3. Continue Release Backlog Grooming

Remember that in contrast with classical waterfall development, agile development can begin as soon as we have enough features defined to feed the first one or two iterations. As the release proceeds more features become fully elaborated into user stories, and become available for sprint planning. Also, sometimes feature priorities may shift and this needs to be reflected in the plan.

Backlog Grooming or, Backlog Refinement, is a process to get the top of the release backlog into shape for sprint planning. This means having features broken out into user stories with size estimates for each story. Although not identified as one of the core activities in scrum, it is acknowledged to be a key process to ensure that the flow of a project proceeds smoothly without delays or interruptions. Hence this should be an ongoing activity throughout a project with perhaps one meeting per week dedicated to it. Participants should include the product owner and team. The output of the process may include a re-ranking of features and stories, addition of new features/stories, and removal of ones no longer considered essential to the program. Features are split out into user stories, and user stories elaborated into sufficient detail, including acceptance criteria, so that the development team can proceed with sprint planning and execution without delay.

Release-Level Scope Setting.

It is important to note that adaptive planning does not mean that cost and schedule remain an open question for the duration of the project. It is expected that by then, the team will have completed analysis of all requirements within the first few iterations. In the case where we have fixed-price contracts, the contract should be structured to include an early milestone where requirements are agreed by both customer and supplier, and baselined. Mechanisms for changing requirements beyond this baseline, should be defined in the contract.

Release Scope Setting

Release Scope Setting

Scope setting example (assume 2-week iterations):

  • Project start: Jan 01
  • Project finish: Jun 30
  • Duration: 26 weeks
  • Iterations: 13 (26/2)
  • Team Velocity: 20 (points per iteration)
  • Maximum deliverable points: 260 (13 X 20)

In this example, and assuming the team’s velocity is fairly accurate, and that the features are correctly ranked, they can only deliver the first 10 features (a total of 249 story points). To complete all 12 features the team would require a further 3 iterations. This team now has a good set of data to go back and work with the business stakeholders to reach agreement on a scope/schedule trade-off.

Agile Requirements Management

One of the most frequent complaints I hear when I work with agile teams is about the lack of good product requirements. In an overzealous rush to have ‘working software over comprehensive documentation’, many teams have abandoned traditional product requirements documents. However, having not introduced an effective alternative, they have really only succeeded in creating re-work, waste and delay. Requirements continue to be a major bottleneck in getting software projects off to an efficient start. If requirements are not sufficiently well-defined before development begins then potentially everything downstream from that point will be a waste in the lean sense – incomplete work (inventory), re-work, and defects. How do we set up an approach which is non-waterfall – all requirements must be fully defined before the project moves forward –  but which gets us ‘development-ready’ requirements that enable development teams to begin executing as soon as the first feature is properly defined?

First, let’s talk about the characteristics of well-defined requirements, then we will discuss a workflow that gets requirements into this state as quickly as possible so that development teams can become productive. Some of the key qualities attributable to good requirements are:

  • Completeness. The requirement is fully stated with all necessary context information, like use cases, diagrams, and dependencies.
  • Precision. The requirement is stated concisely and objectively. It is subject to a completely unambiguous interpretation.
  • Consistency. The requirement does not contradict any other requirement and is consistent with any source material for the requirement.
  • Testability. The requirement is written in a way that its implementation can be validated through testing.
  • Traceability. The requirement is traceable to business needs as stated by the customer or stakeholders. The requirement should be traceable to other project artifacts.

Too often development teams struggle with one-line product requirements definitions that leave team members to make too many assumptions, or having to guess. Forcing teams to make multiple trips back to product owners to clarify requirements is not agile, it is highly wasteful. Scrum masters should treat this problem as a project impediment,  and push hard with the appropriate people until the information is forthcoming.

The next thing we need is a requirements-level workflow. I can think of several reasons why this is valuable:

  1. We need an approach to requirements creation that is consistent and repeatable
  2. The organization needs to be clear about the basic steps to get a requirement into the ‘Defined’ state – one from which user stories can be developed, and from which sprint-level planning can begin.
  3. The workflow ensures there is no ambiguity about the state of development of any individual feature, and provides a framework for tracking and reporting the status of all features in a release.

A generic workflow might look something like:

FR-Workflow

FR-Workflow

If we wanted to have a complete separation of focus on feature definition vs. feature realization workflows then we might adopt something like:

Definition-vs-Realization Workflows

Definition-vs-Realization Requirements Workflows

To apply this to sprint planning, we sort the release back log and place all of the Defined features on the top of the stack. ‘Defined’ means development-ready requirements, and sprint planning should be able to proceed with any features in the defined state.

Release Backlog

A Release Backlog

As part of release planning, teams will be working to get enough features into the defined state so that development can get underway. One definition of a ‘Defined’ feature might be:

  • Feature has a detailed description
  • An architectural assessment has been made – requirement decomposed and allocated to various subsystems
  • User stories have been created, and story size estimates made

For tracking progress in the effort to get features to the defined state, a simple Kanban-like chart helps:

Feature Definition Kanban

Feature Definition Kanban

This helps to visualize bottlenecks in the process and to ensure the effort is properly resourced. This approach can also be applied as the ‘Portfolio’ level – for all projects that the organization has underway.  This can bring clarity to where resources should be focused to optimize feature development throughput. WIP limits can be imposed to prevent overloading of one part of the organization or another. Consider the following distribution of requirements across various stages of the lifecycle:

Portfolio Kanban

Portfolio Kanban

The numbers in each bucket are requirements. For those requirements that are at the Defined stage or beyond, we should have a total story points estimate. Thus for example if the 20 requirements in development between various releases amount to 400 story points of work, and the 80 Defined requirements require 1000 story points, then there is little point in adding more requirements to the front end of the work flow – where they can potentially actually slow down throughput.  In this case development is the bottleneck and the organization should consider some rebalancing of resources to increase overall throughput.

As the release backlog takes shape it might begin to look something like:

Evolving Release Backlog

Evolving Release Backlog

If the team has a velocity of say, 20 points per sprint, then in this example we have enough work to keep the team going for at least 2 sprints. Getting the remaining features to the Defined state continues as an ongoing process. A snapshot of a properly groomed release backlog shows us which features can be pulled into the sprint planning process, which ones need further analysis to get to the defined state, and which ones need a basic discussion about whether they should be in-scope for the current release.

In the waterfall model, development work would wait until all features were fully defined. With an agile approach, work should commence as soon as the first feature is defined.

The key piece of advice offered in this chapter is to make the up-front investment to get requirements at least mostly right. This approach is completely aligned with the lean concept of eliminating waste. Grinding through a project with partially-completed features, incorrectly specified features, or with teams simply guessing what to build is a colossal example of waste. Agility requires both discipline and focus. Robust requirements management is a fundamental agile practice.

 

The Agile Feature Factory

I’ve never met a customer who cared about user stories. Why would they? They care no more about user stories than they care about test cases, code reviews or any other artifact from the software development process. Yet almost the entire agile software industry, one that outputs volumes of books, videos, papers, seminars, training, tools, seems obsessed with just that. Take the leading tools for example, I won’t mention specific vendors, but these tools are designed around the creation and management of user stories. Put these tools into the hands of development teams and we’ve now got projects whose primary focus is the delivery of user stories. In all my years of working with agile development teams I have never seen so much abuse of a concept. Iteration backlogs have become dumping grounds for every imaginable project task, the vast majority of which provide no conceivable benefit to an actual user of the software. One typical consequence of this is that people in customer-facing roles such as product managers, program managers or executives often struggle to get the answer to a simple question – how many customer committed product features have been completed so far, and what is the status of the remainder? The frequent inability of project teams to come up with a straightforward answer to this simple and entirely reasonable question should be a cause of major concern. Typically the tools do not help. What happens next after the escalations begin usually involves exporting project user stories to spreadsheets and applying elaborate post-processing to get close to what might represent a reasonable answer to the question we started with.

User stories are a great way to break down product features into smaller fragments of functionality that can be implemented and demonstrated in a single iteration. But go ahead and take a look at a backlog created by one of your development teams. You may be left wondering what this list of stuff is actually for, and you may struggle to identify anything that remotely resembles a feature your organization plans to deliver to an actual customer. The team may show you a burn-down chart that suggests that they are completely on track. But on-track to deliver what?

Customers have no interest in ‘user stories’. Customers pay for product features. This is not just semantics. Some of the tool vendors have finally caught on and are now talking about ‘Portfolio Management’. This seems like a way of introducing their users to feature delivery like its a brand new concept, requiring more training, seminars, tool licenses and so on.

My goal in this book is to start with the customer and their requirements stated in terms of product features. We then focus on how to design a software development framework where feature delivery – not user stories – is the primary objective. The question then is: what is the optimum framework for transforming customer requirements into production quality product increments? We continue to embrace the concept of delivering small batches of customer-facing functionality in short, fixed time-boxes, as a way to achieve maximum flexibility in the face on uncertainty, while maximizing quality and minimizing risk.  However, we also need a ‘front-end’ process that takes in raw requirements and transforms them into ‘development-ready’ units of customer-facing functionality (user stories) as an input to a product realization process that can proceed without delay, interruption or re-work. This front-end process must run concurrently – not sequentially – with the realization process, keeping development teams supplied with at least 2 iterations worth of development-ready stories at all times.

Finally, organizations must at all times have a clear, unambiguous and accurate picture of the status of a release, right down to the status of individual features. Tracking a release by focusing on user stories is misleading, and can sidetrack both development and management teams with a false sense of security. We introduce an approach to ‘feature-centric’ project tracking and reporting that eliminates any confusion about project status, and provides early warning signals that can be acted upon to mitigate risk.

Software Business Models

These days we hear a lot about ‘Dev Ops’ and businesses that provide a cloud-based ‘Software-as-a-Service’. A good example is Spotify – the music streaming service. In this example, and many others like Netflix, SalesForce, and LinkedIn, the software is built internally by the company itself and end-users typically pay a monthly subscription to use the service. These companies have total control over what, when and how to release new features to their users, and users have little direct say in the content of any new release.

Meanwhile, the vast majority of software developed today is from businesses that operate the more traditional model, where software is delivered in a series of releases which are then licensed to customers. Obvious examples are Microsoft, Oracle, SAP, IBM and Symantec. These companies are typically ‘roadmap-driven’, where releases are planned by product marketing teams based on market trends, competitive forces and emergence of new technologies.

A third major classification is for businesses who develop software in response to individual customer contracts. In this model there is little debate about which features to build – it’s in the contract. This model typically involves ‘Statements-of-Work’ (SOWs), with payments made against the completion of specific milestones in the contract.

Finally, a large amount of software is constructed by IT departments within companies. Typical examples are banks and financial services companies. These companies do not sell software to end users, but use it to operate their businesses and provide services to their customers.

This book is applicable to software development within any of these operating models. It is particularly relevant for those directly responsible for delivering on business objectives – meeting the sales forecast, achieving earnings targets, and nurturing customer relationships. There is much in today’s agile literature about empowering development teams and this is a critical dimension of any agile enterprise. Development teams should be completely aligned with the organization’s business goals – and understand them sufficiently to be able to make good tradeoffs and priority calls, and must play a prominent role in bringing ideas and solutions to problems to the broader organization. Let’s be clear though, software developers  are not directly responsible for the P&L, and software teams do themselves no favors when they make statements like:

  • “Leave us alone, we will deliver the right product”
  • “We are agile, we don’t gives dates or do reports”
  • “We cannot tell you when feature X will be done”

This kind of talk terrifies executives. Agile Development is a means to an end, and that end is to improve business performance by delivering software faster, with higher quality at lower cost and risk, and with more flexibility to absorb changes in product requirements.

Those in customer-facing business roles need visibility and data from software projects to be able to accurately articulate progress and risk to both internal stakeholders and customers. This reality does not change whether software is developed in an agile or waterfall fashion. This is not an issue of ‘command and control’  – no-one is telling developers how to do their jobs, but development teams exist to serve some business purpose – and solid alignment between the development team and the business team is crucial for a successful business outcome.

This book is about how to achieve successful business outcomes in the field of software development. Whatever the business model, this book offers guidance for anyone involved in the construction of complex software systems, where new feature creation typically involves modification to multiple subsystem components, and validation involves testing end-to-end on a fully integrated system. There are a handful of practices that are required to establish a foundation for agile development.

  1. Requirements. Agile development requires a solid process for converting customer requirements into development-ready work items in the form of User Stories, that permit development teams to deliver incremental product deliveries in a ‘shippable’ state.
  2. Adaptive Planning. Means keeping the backlog that feeds the development team continuously filled with development-ready user stories so that there are no delays between requirements elaboration and realization.  This also means only producing enough ready work required for the team to complete at least one iteration, so that plans can be revised without throwing way a lot of unnecessary detail.
  3. Iterations. Breaking a project down into small batches of work that deliver measurable incremental progress is a way to improve productivity, boost quality and reduce risk. To succeed in today’s tumultuous environment, development teams need to be able to turn on a dime and abruptly change direction whilst ensuring that work already completed is not waste. Iterative development is the best defense against this kind of risk.
  4. Production-quality code with every iteration. Delivering partially completed work in iterations is a wasteful and expensive mistake. Practices like peer code reviews, continuous integration and maximum test automation are an absolute prerequisite to achieving this goal.

The overall goal of agile development is to achieve an uninterrupted flow of value-adding activities from the arrival of feature requests to the delivery of ship-quality product increments.

In this book we will be digging into the details of each of these while keeping our primary focus on feature delivery. Readers are assumed to be familiar with the basic concepts of agile and scrum, however, for completeness, I have included a primer on these topics in an appendix.

 

 

 

Large Systems Integration

You have a large complex system comprising multiple subsystems that interact to provide a set of services to end-users. This could be a large financial system or a piece of communications infrastructure. You have perhaps 10 development teams working on different subsystem components, and a system integration team that takes the output from the development teams and then does 2 sets of things:

  1. Runs end-to-end testing on the fully integrated system to validate the new functionality
  2. Runs a full system regression to ensure that the system still performs as expected.

There are basically 2 approaches to this (plus many variations):

1. The system integration testers are embedded within each of the development teams. Each user story follows a work-flow, for example: Backlog -> Defined -> In Progress -> Completed -> Accepted. ‘Defined’ means that the user story is fully defined, including acceptance criteria, and hence test cases can be created. ‘Completed’ means that coding and unit test are done on the story, and the story is now available in a branch of code for testing. System testers can take the build and test it end-to-end on a fully integrated system. Upon successful completion of this testing, the story is marked ‘Accepted’. In this model there are no separate phases or hand-offs. Testing happens concurrently with development and is triggered by stories reaching the ‘Completed’ state. This model may require a longer iteration length, say 4 weeks, to give the system testers sufficient time to verify all stories end-to-end, and to complete a full regression. If a full regression cannot be completed, then at least a ‘core’ regression should be done, and arrangements made to complete the remaining tests at another time – perhaps a full regression requires a 2-sprint cycle.

Concurrent Testing Framework

Concurrent Testing Framework

2. The system integration testers run their own separate sprints, and lag one full sprint behind the development teams as in the diagram below. This is far from ideal, but we might have a situation where the system integration team has thus far only automated say less than 50% of their test library of 3,000 test cases, leaving a very significant manual test effort to get through on each iteration. In the second approach the system integration team picks up a completed set of user stories at the beginning of their iteration, and completes testing of all new functionality, plus a full regression within their cycle. This process repeats itself at the end of each development sprint. If the sprint length is 3 weeks then it will take 6 weeks to deliver a ‘potentially shippable product increment’ from sprint 1, but a ship-quality release is available every 3 weeks thereafter.

System Integration Test Team

System Integration Test Team lags one full sprint