The truth which makes waterfall advocates go blind

Frans Bouma recently posted a post to his blog pointing out some of his frustration with the various ALT.NET communities (specifically mailing lists).   As I understand it, his primary argument was that there wasn’t much academic thinking and citations behind some of the positions and postulations people have voiced on the list. While I certainly appreciate his (and am guilty of it) and agree that there’s a less citation and academic backup behind some of the ideas than there should be.  Part of this is simply because the CS academic community is behind the commercial world in some areas, and the commercial software development world generally doesn’t produce a lot of academic-quality research papers sufficient for the level of citation Frans appears to be demanding.

But aside from that, I have a particular beef with Frans over his position on Waterfall. He has, on several occasions, asserted that there are specific situations where the waterfall methodology makes sense and/or is even preferred — even necessary. He used a specific contrived example of MRI (Magnetic Resonance Imaging) machines and the software that runs them.  This is essentially an argument that mission-critical, life-critical machines MUST or SHOULD be developed using Waterfall.  I’m not sure I agree with this and the two seem to be separate, if not mutually exclusive.So, in order to elevate the discussion using Frans’ suggestion, I did some research to try to find how and what methodologies are used for mission- or life-critical systems such as medical machines, military combat command and control systems, etc.

Case Example: QinetiQ

I stumbled upon this press release from a company called QinetiQ announcing an MRI software deal with Mayo Clinic in Minnesota (a world-renowned top-notch medical facility).  It turns out that QinetiQ also builds other life-critical systems including Hyperbaric Oxygen Wound Healing chambers and, perhaps more interesting, battlefield tactical management and human integration systems. They have a product named (and I love this name): SMITE – Systems Management In Tactical Environments.

(screen shot from QinetiQ’s SMITE application)

Software Methodologies in C2 Systems

Ok, so we’ve established that QinetiQ is doing some pretty heavy life-and-death critical stuff. So, if Frans is right, they are most likely using some form of Waterfall Methodology, because it’s the only one reliable enough to guarantee critical software reliability, right?

Imagine my surprise when I saw this PDF whitepaper that QinetiQ’s UK office prepared for the 8th ICCRTS Command and Control Research Technology Symposium, and indexed by the U.S. Department of Defense’s Scientific and Technical Information Network (suggesting the world’s largest military apparatus is interested in this information). The PDF walks through various popular software methodologies including Waterfall, V-Model, SSADM, and a pragmatic, quasi-Agile “Rapid Application Development” methodology (they call it). They weigh the plusses and minuses of the various methodologies and essentially eliminate them one by one.  Of particular interest, here is an excerpt where they argue that Waterfall will not work for critical “C2″ (command & control) systems:

The waterfall model was based on a number of assumptions:  

  • that all its stages could be completed in sequence
  • that the costs and benefits of an information system could be calculated in
    advance
  • that users knew what they wanted
  • that the work needed was known and could be measured
  • that programs once written could be altered
  • that the right answer could be produced first time

For the development of C2 systems, none of these have been shown to be true. C2
systems have proved difficult to manage because of at least two major problems
which cannot be dealt with by traditional management methods. These are (Grindley,
1993):

  • the difficulty of stating what is required
  • the difficulty of measuring pioneering work

The traditional life cycle approach effectively ignores these issues and developers
have been forced to consider other methods to overcome these problems. One solution
is to integrate prototyping into the user and system requirement processes.

It’s Scientific Frans, Waterfall Doesn’t Work; All Non-Agile Projects Eventually Devolve into Necessary Sub-Standard Agile

Now, granted, this is only one white paper (which cites the book “Managing IT at Board Level: The Hidden Agenda Exposed“), but I’m quite sure that given a few more days, I could come up with quite a few more examples.

Frans, I respect you greatly and you’re a very smart guy, but I defy you to show me quantitative proof that Waterfall has led to the success of anything after the 1950′s, rather than projects succeeding in spite of the Waterfall process they started using.

It has been my anecdotal experience that all projects that aren’t specifically attempting to do Agile will eventually end up there out of force of necessity, but they will do it too late and without proper methodology, thus jeopardizing the project itself, or the moral of the talent on the team.

Agile, at its most basic, it concerned with managing the main, unavoidable (whatever your Waterfall books may say) constant in all projects: You (anyone) simply cannot know everything that will need to be done in order to call the project ‘Done’. Done is something that is eventually discovered, no matter how good the planning, experience, and knowledge of the problem is up front.

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Chad Myers

Chad Myers is the Director of Development for Dovetail Software, in Austin, TX, where he leads a premiere software team building complex enterprise software products. Chad is a .NET software developer specializing in enterprise software designs and architectures. He has over 12 years of software development experience and a proven track record of Agile, test-driven project leadership using both Microsoft and open source tools. He is a community leader who speaks at the Austin .NET User's Group, the ADNUG Code Camp, and participates in various development communities and open source projects.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://weblogs.asp.net/fbouma Frans Bouma

    As I’ve replied on my blog: Philips Medical Systems uses waterfall and a very formal process and formal hierarchy of proving to write their system software. Their domain is fixed and known up front. The thing is: they have to do a lot of hardware too. You HAVE to know up front what you’re doing, you can’t design a piece of hardware and a piece of software for feature X, build it and move on to the next.

    But I stop now… You too put me in the position where I have to defend waterfall etc. I wont do it anymore, because I don’t see why I have to. I don’t use waterfall myself nor promote it, I used 1 tiny stupid example, and oh boy the hounds broke loose.

    Did you enjoy that? I hope not. I as hell didn’t. What especially stunned me was the total lack of understanding of how another person might think, what the reasoning behind a point of view might be.

    I think it was you who mailed me to be glad I joined the alt.net mailinglist. Today I regret I ever did that.

  • http://chadmyers.lostechies.com chadmyers

    @Frans: (cross-posted)
    Philips Medical is transitioning to iterative, using RUP:
    http://www-01.ibm.com/software/success/cssdb.nsf/CS/WJBN-6LBLRF?OpenDocument

    I don’t expect you to defend Waterfall. I know you’re not ADVOCATING it, you’re merely saying it’s not totally dead/worthless and I’m saying it is.

  • http://netcave.org Alan Stevens

    Chad,
    This is a great discussion. I’ve always said that waterfall can work, but you must be able to control all aspects of development. See this: http://www.fastcompany.com/magazine/06/writestuff.html

    ++Alan

  • Connor Peterson

    What about the thousands of apps written in a corporate environment that don’t follow any methodology but manage to function pretty god damn well and exceed their customers expectations?

    My team has about a dozen small to mid to large scale applications that do exactly what the customer wants and are delivered on time (sometimes even against a gantt chart. Shock. Horror. Etc). I don’t disagree that following an agile methodology would result in better written and easier to maintain software, but to suggest that it’s not possible without a stack of index cards, a whiteboard, a ton of acronyms and pithy sayings is ridiculous. (Red! Green! Refactor!…I’d like to smash someone in the mouth with a stapler for that, but I digress). Software can be and is successfully written without adhering to any ivory tower “manifesto”.

  • http://chadmyers.lostechies.com chadmyers

    @Conner: I work in an environment like that. It’s called the U.S. Federal Government. Ever heard of it? :)

    I see the pain that having no methodology, or an assumed (but fatally flawed) Waterfall-esque project management style caused and all the money it wastes and how the customers are never, really, satisfied (it’s always just “good enough”).

    For the record, Conner, I never actually advocated Agile(TM) (the cards and stuff you described) for all circumstances (although I could make a strong argument that quasi-agile is better than quasi-Waterfall in all circumstances). I’m merely referring to the idea that you CAN’T know everything up front as Waterfall requires.

    Waterfall is based on a fatally flawed premise that defies logic, defies all subjective experience, and, more importantly, defies human nature. It will *NEVER* be right and, as far as I can tell, *NEVER* has. Waterfall was the best we had for a long time, but it wasn’t even 75% good. It would get you started and moving and quickly became an iron ball chained around your ankle.

    Everyone who shows me Waterfall “success” stories always end up showing me case studies where people started out with big requirements documents and tons of testing which resulted in tons of bugs and new features slipped in as “bugs” during “bug fixing” phase at the end of the so-called Waterfall project.

    This always-present, amorphous “bug fixing” phase is the NECESSARY agile project that eventually ships the real product.

    Why endure the Waterfall pain at all, and just use the agile “bug fixing” methodology throughout the whole project?

    More real and interesting work gets done on the project in that “bug fixing” phase at the end than got down during most of the rest of the project because it’s real people, forced into a real situation where teamwork and serious effort are required to make the project succeed. No one can hide behind 500 page Specification documents any more.

    In the end, like the Agile Manifesto says, it’s about the people. Whether you want to admit it or not, it starts with that and ends with it.

    All 500 page spec docs do is delay projects and distract people from the real task at hand.

  • http://chadmyers.lostechies.com chadmyers

    It’s also important to know that Agile(TM) (as opposed to ‘agile’) is not an ivory tower, it’s a collection of practices that were actually used and found to be effective.

    3×5 cards necessarily keep requirements short and sweet. I’m not sure why anyone would argue that a 500 page spec doc is more useful for anything other than killing trees and wasting ink.

  • Peter Lilly

    Look, first of all, QinetiQ is a large organization and that was just a research paper and it is very likely that the left hand does not know what the right hand is doing and also within it there is disagreement about this topic. Also the people who push for it may not necessarily be the best people in that organization, etc.
    Another thing. Each development system has its merits and that depends on the context of the problem. I remember many years ago going to two presentations on UML, the first industrial where everyone accepted UML over SSADM but the second was an academic one, in that one many of the guys said that URL could not replace SSADM at everything, that there were some nice properties of SSADM that would be thrown away by adopting UML. You see that with every revolution you gain but also you loose. You throw the baby with the water, so please do not be so absolutist in your opinnions.

  • http://davesquared.blogspot.com Dave^2

    Enjoyed the post Chad (and Frans’ too for that matter). All this talk reminded me of something I heard ages ago about there being no such thing as “waterfall development”. I decided to look into it and posted a couple of sentences here:
    http://davesquared.blogspot.com/2008/01/contentious-myth-of-waterfall.html
    Quick summary, the original waterfall model appears to have been used to illustrate a process that doesn’t work.
    Bit more here:
    http://www.idinews.com/waterfall.html
    Regards,
    David

  • David_Ncl

    “There’s no such thing as the Waterfall Approach!
    (and there never was)”

    http://www.idinews.com/waterfall.html

  • David_Ncl

    Gah ! (Ive just read Dave^2 ‘s post. sorry.

  • http://chadmyers.lostechies.com chadmyers

    @Peter: Do you have inside information? lol I see what you’re saying and you’re right that just because QinetiQ talks anti-waterfall doesn’t mean they’ve figured out a way to actually get off of waterfall. But that conference is pretty big and I’m sure it means a lot to their corporation, so it’s hard to believe that they would’ve put together that presentation to sell a staunchy, resistant-to-change military audience on iterative/anti-waterfall just for the sake of doing it.

    It’s possible that their medical/MRI group doesn’t use iterative, but I think it’s reasonable to assume that their military C2 systems are.

  • http://chadmyers.lostechies.com chadmyers

    @Dave, @David_Ncl:

    Thank you for the comments and links. I had seen that waterfall.html link. It only makes my point even more important: An even worse problem with Waterfall is that there never was Waterfall and everyone is doing a bad practice, wrongly.

  • http://blog.sidu.in Sidu

    “Red! Green! Refactor!…I’d like to smash someone in the mouth with a stapler for that, but I digress”
    I say the first bit to myself a couple of times a day at least, lol. I guess it’s up to the gods to defend my poor mouth from the vengeful stapler…

  • http://www.techneeds.co.uk Nish

    Till we find a person who will satisfy what he has, we will not be able to stick to the one requirement.

    I am not an experience in large scale projects. But from my experience I know customers are changing their requirement during the project. So I think Agile is the best method most of the commercial application developments.

    As Lord Buddha said everything is changing in this world.

  • John

    Oh look, more agile advocates, desperately trying to prove that their method is the only one that works.

    Newsflash: agile doesn’t always work. Stop trying to pound the entire world into your viewpoint just because you’re too lazy to think about a design upfront.

  • http://chadmyers.lostechies.com chadmyers

    John,

    Please show me where it doesn’t work. Also please show me, in the case where it didn’t work, what did work instead.

    I’m pretty confident what what you show me will be some form of agile (NOT necessarily Agile(TM)) — that is, some form of iterative requirements discovery, coding, and testing cycles in short timespans vs. long ones.

    I’m still of the belief (and have yet to been proven otherwise), that it is humanly impossible, on any sufficiently large project, to know all the requirements up front with no changes between reqs/specs and testing/release (true bugs/defects notwithstanding).

    I defy someone to show me a large project where this is true.

    Even in one of the more life-critical situations like the Space Shuttle, they’re making changes to the software and the hardware up to and including the days before launch.

    > “just because you’re too lazy to think about a design upfront.”

    lol, is that what you think this is about? I’m not saying you shouldn’t give thought to design and lay down some pre-requisite design and basic foundation work. What I’m saying is that a long, drawn-out 3 months requirements and planning/design marathons are guaranteed to fail (where ‘fail’ means that more design and planning will be required somewhere in the project before it gets released).

    And if more planning and design is required somewhere before it gets released, then all you have done is shoe-horned bad agile into the middle of waterfall.

    so why not plan for change up front, do some design up front (as much as is necessary, but not more) and then start working out the solution.

  • http://chadmyers.lostechies.com chadmyers

    And one other thing…

    I and others have repeatedly asked for proof where agile has failed and waterfall or some-other-non-iterative-agile-by-another-name methodology has succeeded.

    So far, no one has been able to produce proof.

    Even the master-builder Egyptians had to changes some of their plans/design along the way as they built the pyramids.

  • http://www.peterRitchie.com/blog/ Peter Ritchie

    Come on, citing one vendor that doesn’t do waterfall is no more scientific proof that waterfall can’t work than the following proves it’s OK to use memory after it’s freed in C++ in release mode:

    int * const p = new int[10];
    p[1] = 10;
    delete[] p;
    if(p[1] == 10)
    {
    puts(“ten”);
    }
    else
    {
    puts(“not ten”);
    }

    …just because you observe the output to be “ten”…