Charlie Alfred’s Weblog

Agile vs. Waterfall is not the Right Question

Introduction

About 5 years ago, I posted an entry to this blog (here) on the topic of agile method and architecture.  During this time, this topic has received a considerable amount of discussion, including:

What’s most interesting to me is how polarized the positions are – as if there were one right answer to the global question.  As if we could ask the NFL to stage a special Super Bowl to determine a winner, once and for all.  This isn’t going to happen, and it isn’t because the NFL has better things to do, or the Meadowlands stadium is in use, or because all the hotel rooms in Manhattan are booked.

It’s not going to happen because the choice of a development process is context-dependent.  As my friend Gene Hughson is fond of saying, “context is king

Context-Dependent Process

Whether you are using agile or waterfall, the fundamental goal is to develop a solution that satisfies the important needs or wants of the key stakeholders, especially the development sponsor, since he/they are paying the bills.  In waterfall methods, these needs/wants are expressed as marketing or product requirements. In agile methods, they usually are expressed as user stories.

There certainly are valid reasons for choosing one approach versus the other and we will explore this topic later.  But as an introduction to this exploration, let’s explore the relationship between feasibility, context and process.

Feasibility and Context

Context is a key driver for development process; it also drives feasibility.  In fact, context, process and feasibility form a triangle.  Feasibility is a property of a given project that indicates:

There is at least one viable solution to the problem that the development teams knows (with a very high degree of certainty) and is capable of executing.

Feasibility does not require that the identified solution is optimal, or even among the best.  It simply requires that the team has the ability to deliver a solution that satisifes the stakeholdeer wants, with a high-degree of certainty, limited by the stated constraints. Note that each of the four clauses in this statement has the ability to be a feasibility-killer:

  1. Over-goaled problems are those whose goals are unreachable.  For example, a hard release date of 9 months might be set as a negotiating ploy, while the real goal might vary between 9-15 months, with the most value being 9 and the least 15.
  2. Over-constrained problems are those where constraints preclude satisfying the goals.
  3. Capability-constrained problems are those where the team lacks the necessary knowledge or ability.  A problem is not feasible just because somebody else has accomplished it, as evidenced by the Healthcare.gov example.
  4. Uncertainty-constrained problems are those whose solutions are limited by unknowns or uncertainty.  For example, a framework might be required to support an open-ended set of devices with unknown performance characteristics.

The problem here consists of a collection of stakeholder wants (presumably with known completion criteria) and a set of constraints.  The stakeholder wants might be expressed as marketing requirements, use case scenarios, user stories, or possibly other forms.  They might come from different types of stakeholders (e.g. different types of end  users, support engineers, people paying for development, or developers).  These wants may even have inherent conflicts.  The inexperienced users want “easy to learn” while the experienced users want powerful features that make it “easy to use”.

Constraints are restrictions under which the development process must execute.  Laws, governmental regulations and industry standards are constraints.  Integration with existing systems or workflows are also constraints.  So are schedule and budget.

A collection of stakeholder wants and constraints, by themselves, are not guaranteed to be feasible.  As shown above, wants expressed by different groups of stakeholders might conflict.  Constraints might make wants extremely difficult, if not impossible.  The team might lack the knowledge or experience it needs (i.e. it is not sufficient for a problem to be feasible for another team).  President Obama said of Healthcare.gov:

“This is real simple,” Mr. Obama said, during a speech in Maryland on Sept. 26. “It’s a website where you can compare and purchase affordable health insurance plans side by side the same way you shop for a plane ticket on Kayak, same way you shop for a TV on Amazon. You just go on, and you start looking, and here are all the options.”

The fact that Healthcare.gov exhibited similarities with Amazon.com does not guarantee that it was feasible for a group of 47 contractors, given a budget of $319 million dollars.  Just as it doesn’t mean that it is feasible for your recreational ice hockey team to win a gold medal at the 2018 Winter Olympics.

At the heart of feasibility is a process that contains identify, assess, evaluation and mitigate steps.  However, reality is more complex than the textbook approach.  Each of these steps contains uncertainty, and the more uncertain steps require more investigation.

For example:

  1. A system requiring high performance and scalability should be simulated with a prototype to better understand the real world uncertainties and constraints related to these two goals.
  2. A situation where a development team lacks experience with a technology (e.g. mobile applications or cloud deployment) is significantly helped by applying these technologies to the development of a well-defined prototype.  Not only is the knowledge risk mitigated, but previously-hidden factors also surface.
  3. Stakeholder stories or requirements are often stated in absolute terms:
    • “the system shall do X”
    • “the system must do these steps to satisfy this story.”

In most cases, these are “unconstrained goals” and take the form of Children’s “Letters to Santa”.  They tend to create conflicts (especially with uncontrollable constraints) and preclude tradeoff decisions.   In most cases, stakeholders have an unstated sliding scale characterized by degrees of suitability (which happen to mimic school grades A,B,C,D and F)

    • Ideal – What is the level of outcome above which you hit diminshing returns?
    • Desirable – What is the level of outcome that is better than average but not quite ideal?
    • Acceptable – What level of outcome would you be satisfied with?
    • Adequate – What level of outcome is not quite acceptable, but with which you would be able to get by?
    • Undesirable – What level of outcome eliminates virtually all value and must be exceeded

The Marriage of Context, Feasibility and Process

While three-party marriages are uncommon in most societies, we will attempt to establish this union here.  Let’s examine four example scenarios, where context and feasibility drive which process offers the best fit.

  • Sometimes stakeholder needs are vague and quite volatile, while solution technologies are stable and well-understood.
    • In this situation, an agile approach might be ideal.
    • Frequent iterations with stakeholder review help stakeholders visualize the solution
    • Technology knowledge means behaviors are understoold, which helps streamline refactoring
  • Sometimes the solution requirements are flexible while the cost and schedule are extremely constrained.  For example, a business process may already exist, and there is flexibility regarding how much gets automated versus remains manual.
    • In this case, an agile approach is also likely to work well
    • Stakeholders are able to identify the biggest pain points with the current process
    • Developers are able to identify aspects that the big pain points depend on
    • Iterations can be defined around critical capabilities and their essential dependencies
    • Enough flexibility exists to refactor without seriously jeopardizing existing solutions
    • Work can stop when time or budget is consumed
  • Sometimes the system is safety critical.  Human lives may be jeopardized.  This might be a class 3 medical device, like a defibrillator or a stent navigation system.  Or it might be an altimiter or collision avoidance system onboard an aircraft.
    • In these cases, the safety-critical concerns likely favor a waterfall model
    • Safety, like security, performance or reliability is a system property
    • Mitigation strategies need to integrate multiple components with shared responsibilities
    • Inter-component interactions need to be experimented with and proven
  • A 3rd party is building part of a system under a fixed-price contract.  The specifications for what the component will do, what it’s performance criteria are, and how it will be integrated are clear
    • This may also be a good candidate for a waterfall model
    • Feasibility is high.  The deliverable is defined and has low technical and integration risk
    • The prime contractor needs a known deliverable for a known price and schedule
    • A waterfall process specifies requirements, enabling the 3rd party to bid cost and schedule
    • The 3rd party needs clearly-defined requirments and an agreed to ECO process to handle changes.
    • An iterative process can be “time-boxed” but also has risk of scope creep

Conclusion

While four examples do not constitute a definitive proof, they are sufficient to begin to expose some patterns and inferences:

  1. Choose the process to fit the context and feasibility.  One size definitely does not fit all
    • Waterfall and agile processes have quite different characteristics  – both strengths and limitations
    • There are situations where each excels over the other.
    • There are other situations where each approach has merits, but with clear tradeoffs (Mini Cooper vs. Toyota Tundra)
  2. Context has much to do with which approach fits better (e.g. safety-critical vs. time-boxed with solution flexibility)
  3. Feasibility is just as critical as context.  If feasibility is not known, it is like traveling through a country alone where you can’t read the alphabet and don’t speak the language.  The risk of making a serious error escalates.
  4. Establishing feasibility first is imperative, regardless of which process is chosen.  There are three key pieces of advice here:
    • Make sure you are crystal clear about stakeholder wants.  Establish the sliding scale and degrees of importance
    • Similarly, make sure you understand the constraints, especially which are hard and which have give.
    • Identify critical challenges (how constraints impact goals) and address them early.  Design decisions add new constraints, and feasibility is compromised by deferring decisions on critical challenges.
Advertisements

11 Comments »

  1. Lots of material here to digest, Charlie. We see this as an issue which often has an answer in a third approach. http://www.commonplaces.com/blog/agile-vs-waterfall-revisited

    Comment by Gary Locke — March 19, 2014 @ 4:53 pm | Reply

    • Hi Gary, thanks for the reply and congratulations on your visit to 1600 Pennsylvania Ave! I enjoyed the blog post you referenced and posted a reply (which is in moderation).

      Comment by charliealfred — March 19, 2014 @ 5:23 pm | Reply

  2. Hi Charlie, good points. But concerning your title : please give us just ONE example context, of any kind, where waterfall — which by definition gives minimal feedback about the working system until very late in the project — can give better results than a process (any process!) which does give feedback early and repeatedly during the project?
    Thank you! Chris

    Comment by Chris F Carroll — March 19, 2014 @ 5:26 pm | Reply

    • Hi Chris,

      Thanks for the comment. How’s this for an example:

      Your company makes medical diagnostic devices. One is a small handheld device that inexpensively takes blood pressure, temperature, and pulse oxide (SPO2) measurements on patients.

      This is the second generation of this product. The sensors are provided by 3rd party vendors. They plug in through USB and have defined API’s. Product marketing has defined the key requirements. Because the product is mobile, it must be battery powered and support WiFi. Existing competitors dictate the cost and size constraints for the product.

      Suppose you are working on the embedded control software, which must be designed and written from scratch (the first generation product is too old to be mined for components).

      This strikes me as an example where the requirements are well-understood and the solution technology risk is relatively low. Also, as a medical device, it must be developed under a regulated process and must have requirements to trace to design and test.

      I think, so far, this is classic waterfall.

      Now the $64K question, which is the one I think you are asking (I may be wrong). Would you try to complete the entire design before starting implementation, and complete the entire implementation before test.

      I doubt it. There certainly is value in iterating through the design, implementation and test cycles. Maybe 6-8 week iterations, not 2. And maybe the requirements are adapted slightly or don’t change between iterations. But here, the feedback is about feasibility of design not volatility of requirements.

      Hope this answers your question. What I described is probably closer to the Spiral Method, proposed by Barry Boehm 30+ years ago. Most people I know wouldn’t call it agile, but then it’s probably not fair to call it waterfall either.

      Charlie

      Comment by charliealfred — March 19, 2014 @ 6:24 pm | Reply

      • Hi Charlie, yes you were right, that was my question! Honestly I concur with your conclusion. But it bugs me when people talk about waterfall as good thing. Maybe it’s a bee in my bonnet. Spiral, DSDM & RUP were very good. But I’m convinced that waterfall was never good, except to the extent — and I think you’ve illustrated this — that it got turned into something else. In the very article in which Walker Royce coined the phrase, he pointed out the defect.

        Comment by Chris F Carroll — March 19, 2014 @ 6:47 pm

      • Hi Chris,

        I agree that waterfall, as a purely serial process is dead (unless your project is to write something trivial, like “Hello World.” People, including me, keep using the word to mean something other than what Royce originally meant.

        IMHO, the key driver is requirements. Are they volatile? The flip side is requirements that need to be defined up front and kept fixed. Consider a 3rd party asked to bid on a fixed-price, fixed time contract. Without a full set of requirements it’s pretty hard to make that bid. Even ECO’s are problematic if you can’t prove that the requirement wasn’t already covered by existing weasly words.

        Comment by charliealfred — March 19, 2014 @ 7:20 pm

  3. Thanks for the insight, Charlie — as usual, a nuanced discussion of these issues! The software developers at my company have been using more aspects of Agile over the past year or so — at least as observed from the other side of the partition! From the marketing side, as we put together materials that discuss our IT consulting, it’s useful to have the perspectives laid out in this article regarding the situations where each approach can be used effectively. (Does waterfall “by definition give minimal feedback… until very late in the project?” I can’t think of anywhere I’ve worked where there wasn’t at least some feedback given throughout the project… resulting in me, the tech writer, coming in on Monday morning and finding that everything changed over the weekend, so I had to rewrite half the manual!)

    Comment by John Kafalas — March 19, 2014 @ 6:07 pm | Reply

    • Yes! It is of course normal for waterfall projects to have feedback, with the result you describe. In other words, for waterfall projects to actually be a-bit-waterfall-and-a-bit-iterative. Which amounts to “waterfall is made to succeed by making it into not-waterfall.” So why not go the whole hog and get even more success?

      Comment by Chris F Carroll — March 19, 2014 @ 6:23 pm | Reply

  4. HI John, I think there are three key differences.

    1. How you treat requirements
    a. Define them up front and keep them fixed (often necessary with safety-critical systems, or requirements which are hard to isolate)
    b. Treat requirements as something that can change from one iteration to the next

    2. How you manage the requirements vs. schedule trade-off?
    a. Fix the requirements and drive the schedule
    b. Complete as many iterations as time & schedule permit and then either ship what you have or run over.

    3. How do you decide which things to work on next?
    a. Master schedule and project plan
    b. Let the team select work items each iteration

    I also think #3 is somewhat overstated. Systems tend to have inherent dependencies that force what order things can be done in. Where #3 matters is where conditions dictate a change in course. The “plan per iteration” model can provide more flexibility. So in the end, it gets back to #1 and #2, both of which are centered around stable vs. volatile requirements.

    I think both approaches have converged on test-driven development and shorter design, code, test cycles.

    Best Regards,
    Charlie

    Comment by charliealfred — March 19, 2014 @ 7:32 pm | Reply

  5. Charlie, this is super! I have been writing a bit about creating better options and making better decisions. Evolution of DoD decision making from that driven by process -> data -> information -> intelligence this fits right in.

    Comment by aerogman — July 4, 2014 @ 6:41 am | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: