The truth which makes waterfall advocates go blind

Frans Bouma recently posted a post to his blog pointing out some of his frustration with the various ALT.NET communities (specifically mailing lists).   As I understand it, his primary argument was that there wasn’t much academic thinking and citations behind some of the positions and postulations people have voiced on the list. While I certainly appreciate his (and am guilty of it) and agree that there’s a less citation and academic backup behind some of the ideas than there should be.  Part of this is simply because the CS academic community is behind the commercial world in some areas, and the commercial software development world generally doesn’t produce a lot of academic-quality research papers sufficient for the level of citation Frans appears to be demanding.

But aside from that, I have a particular beef with Frans over his position on Waterfall. He has, on several occasions, asserted that there are specific situations where the waterfall methodology makes sense and/or is even preferred — even necessary. He used a specific contrived example of MRI (Magnetic Resonance Imaging) machines and the software that runs them.  This is essentially an argument that mission-critical, life-critical machines MUST or SHOULD be developed using Waterfall.  I’m not sure I agree with this and the two seem to be separate, if not mutually exclusive.So, in order to elevate the discussion using Frans’ suggestion, I did some research to try to find how and what methodologies are used for mission- or life-critical systems such as medical machines, military combat command and control systems, etc.

Case Example: QinetiQ

I stumbled upon this press release from a company called QinetiQ announcing an MRI software deal with Mayo Clinic in Minnesota (a world-renowned top-notch medical facility).  It turns out that QinetiQ also builds other life-critical systems including Hyperbaric Oxygen Wound Healing chambers and, perhaps more interesting, battlefield tactical management and human integration systems. They have a product named (and I love this name): SMITE – Systems Management In Tactical Environments.

(screen shot from QinetiQ’s SMITE application)

Software Methodologies in C2 Systems

Ok, so we’ve established that QinetiQ is doing some pretty heavy life-and-death critical stuff. So, if Frans is right, they are most likely using some form of Waterfall Methodology, because it’s the only one reliable enough to guarantee critical software reliability, right?

Imagine my surprise when I saw this PDF whitepaper that QinetiQ’s UK office prepared for the 8th ICCRTS Command and Control Research Technology Symposium, and indexed by the U.S. Department of Defense’s Scientific and Technical Information Network (suggesting the world’s largest military apparatus is interested in this information). The PDF walks through various popular software methodologies including Waterfall, V-Model, SSADM, and a pragmatic, quasi-Agile “Rapid Application Development” methodology (they call it). They weigh the plusses and minuses of the various methodologies and essentially eliminate them one by one.  Of particular interest, here is an excerpt where they argue that Waterfall will not work for critical “C2” (command & control) systems:

The waterfall model was based on a number of assumptions:  

• that all its stages could be completed in sequence
  • that the costs and benefits of an information system could be calculated in
    advance
    • that users knew what they wanted
      • that the work needed was known and could be measured
        • that programs once written could be altered

          • that the right answer could be produced first time </ul> For the development of C2 systems, none of these have been shown to be true. C2
            systems have proved difficult to manage because of at least two major problems
            which cannot be dealt with by traditional management methods. These are (Grindley,
            1993):

          • the difficulty of stating what is required

            • the difficulty of measuring pioneering work </ul> The traditional life cycle approach effectively ignores these issues and developers
              have been forced to consider other methods to overcome these problems. One solution
              is to integrate prototyping into the user and system requirement processes.</blockquote>

            It’s Scientific Frans, Waterfall Doesn’t Work; All Non-Agile Projects Eventually Devolve into Necessary Sub-Standard Agile

            Now, granted, this is only one white paper (which cites the book “Managing IT at Board Level: The Hidden Agenda Exposed“), but I’m quite sure that given a few more days, I could come up with quite a few more examples.

            Frans, I respect you greatly and you’re a very smart guy, but I defy you to show me quantitative proof that Waterfall has led to the success of anything after the 1950’s, rather than projects succeeding in spite of the Waterfall process they started using.

            It has been my anecdotal experience that all projects that aren’t specifically attempting to do Agile will eventually end up there out of force of necessity, but they will do it too late and without proper methodology, thus jeopardizing the project itself, or the moral of the talent on the team.

            Agile, at its most basic, it concerned with managing the main, unavoidable (whatever your Waterfall books may say) constant in all projects: You (anyone) simply cannot know everything that will need to be done in order to call the project ‘Done’. Done is something that is eventually discovered, no matter how good the planning, experience, and knowledge of the problem is up front.