In this second part to my ongoing Agile Arguments series, I want to go into a little background and cover a little bit about (what I think) Agile is about and it’s purpose and values. I’d like to also talk a little bit about why I think this stuff is important enough to spend time arguing about.

A little background

First, it’s important to understand the environment under which the collection of practices and processes commonly grouped under the ‘Agile’ moniker were conceived.  After trying, unsuccessfully, many times to get a process working that involved lots of planning up front, lots of design documents, lots of model and entity diagrams, lots of up-front requirements, etc, the progenitors of agile methodologies came to a few conclusions about why it wasn’t working.  From my own experience, I’m going to try to explain some of them and put them in my own words. If you want them in their words, there are several books by them that will explain where they’re coming from. For right now, though, I invite you to read my particular spin on things:

1. Most of the planning turned out to be useless as it involved a lot of false assumptions made from a lack of deep understanding of what needed to be done.
   • The deep understanding necessary to plan was usually not possessed by the people who knew how to plan. Acquiring the deep understanding was, if not impossible, extremely lengthy and expensive and required real experience with the subject matter.
     • Most of the requirements/specifications were wrong, were not specific enough, or involved information that wasn’t necessary/relevant for developing the software.
       • The process of planning and developing the requirements was painful, long, and involved lots of large documents and not enough face-to-face interaction between people involved in the project which lead to misunderstanding, hard feelings, etc
         • Because of #4 above, the various groups of people tended to revert into their own camps and lob large incendiary documents at each other, rather than collaborating as a team
           • Rarely, if ever, was the end user or customer involved at all during this process. If the project ever did manage produce something, the customers were frequently dismayed at what they were given and rarely did it come close to what they expected.
             • Various knee-jerk reactions by managers and executives were used to try to stop the downward slide of the project. Many of these reactions involved placing more controls and checks in place which furthered the process of tribalization among the groups.
               • Various tools and more software was used to help manage the project (MS Project + Gantt  charts, time trackers, document management systems, etc). But all these did were make the process more complicated, further encouraged separation of teams and communication-via-document-only, and other cancers to project success and team moral. Failure was virtually guaranteed at this point.</ol> And, on top of all these problems, whatever semblance of a plan was still remaining, needed to be changed frequently due to changing market conditions or new understanding.  It turned out that the planning phase was not sufficient and, indeed, may never be sufficient.  A total rethinking of how software needed to be done was required. A paradigm shift of epic proportions.

               Why the questions need to be asked

               The problem is, the managers and executives were used to planning, design, execution modes of management because these are required with things like construction and engineering projects, personnel hiring/firing, etc.  A discipline is required with respect to changes on the fly.  Once the foundation is poured on a building, for example, the cost to change the floor plan is prohibitive. So everyone understands the importance of trying to think through everything necessary in the floor plan. Of course you wont’ get it 100% right, and you’ll have to deal with that later. Tough decisions will have to be made about how to cram a new feature into an already full floor plan.  Discipline, wisdom, and experience rule the day here.

               But software — software is different. It’s relatively easy to change. It’s relatively cheap to scrap whole modules and start over.  A discipline was never fully developed, yet the process was still managed like a project that had a discipline (i.e. a physical construction project).  But why does it need to be managed that way? If change is relatively cheap, and the discipline is therefore unnecessary, why do treat it like it’s something that it’s not?

               The answer is, usually, “That’s what we’re familiar with.”  Fear and doubt play a big part in how we humans make decisions. Managers and executives are used to managing the costs and time on a physical construction project and know how to adapt to problems as they arise — that’s what makes them good managers. So the fundamental shift in thinking rightfully concerns them.  Prudence requires they be skeptical. But, as evidence is clearly mounting that software is fundamentally a different type of project and therefore requires a different style of management, managers and executives must learn to manage in this environment and professional software architects must work with them and they must trust each other and try to reach a solution.

               To the questions…

               So, now that we’ve established that things might be different with respect to software development project management, questions arise about the most important aspects of business management: Time, Money, and Scope.  How long will it take, how much will it cost, and how can I know what the end will look like?

               I plan on covering these specific questions in the next part.