Agile Arguments, Part 2 – Background, and Arguments from Fear

In this second part to my ongoing Agile Arguments series, I want to go into a little background and cover a little bit about (what I think) Agile is about and it’s purpose and values. I’d like to also talk a little bit about why I think this stuff is important enough to spend time arguing about.

A little background

First, it’s important to understand the environment under which the collection of practices and processes commonly grouped under the ‘Agile’ moniker were conceived.  After trying, unsuccessfully, many times to get a process working that involved lots of planning up front, lots of design documents, lots of model and entity diagrams, lots of up-front requirements, etc, the progenitors of agile methodologies came to a few conclusions about why it wasn’t working.  From my own experience, I’m going to try to explain some of them and put them in my own words. If you want them in their words, there are several books by them that will explain where they’re coming from. For right now, though, I invite you to read my particular spin on things:

  1. Most of the planning turned out to be useless as it involved a lot of false assumptions made from a lack of deep understanding of what needed to be done.
  2. The deep understanding necessary to plan was usually not possessed by the people who knew how to plan. Acquiring the deep understanding was, if not impossible, extremely lengthy and expensive and required real experience with the subject matter.
  3. Most of the requirements/specifications were wrong, were not specific enough, or involved information that wasn’t necessary/relevant for developing the software.
  4. The process of planning and developing the requirements was painful, long, and involved lots of large documents and not enough face-to-face interaction between people involved in the project which lead to misunderstanding, hard feelings, etc
  5. Because of #4 above, the various groups of people tended to revert into their own camps and lob large incendiary documents at each other, rather than collaborating as a team
  6. Rarely, if ever, was the end user or customer involved at all during this process. If the project ever did manage produce something, the customers were frequently dismayed at what they were given and rarely did it come close to what they expected.
  7. Various knee-jerk reactions by managers and executives were used to try to stop the downward slide of the project. Many of these reactions involved placing more controls and checks in place which furthered the process of tribalization among the groups.
  8. Various tools and more software was used to help manage the project (MS Project + Gantt  charts, time trackers, document management systems, etc). But all these did were make the process more complicated, further encouraged separation of teams and communication-via-document-only, and other cancers to project success and team moral. Failure was virtually guaranteed at this point.

And, on top of all these problems, whatever semblance of a plan was still remaining, needed to be changed frequently due to changing market conditions or new understanding.  It turned out that the planning phase was not sufficient and, indeed, may never be sufficient.  A total rethinking of how software needed to be done was required. A paradigm shift of epic proportions.

Why the questions need to be asked

The problem is, the managers and executives were used to planning, design, execution modes of management because these are required with things like construction and engineering projects, personnel hiring/firing, etc.  A discipline is required with respect to changes on the fly.  Once the foundation is poured on a building, for example, the cost to change the floor plan is prohibitive. So everyone understands the importance of trying to think through everything necessary in the floor plan. Of course you wont’ get it 100% right, and you’ll have to deal with that later. Tough decisions will have to be made about how to cram a new feature into an already full floor plan.  Discipline, wisdom, and experience rule the day here.

But software — software is different. It’s relatively easy to change. It’s relatively cheap to scrap whole modules and start over.  A discipline was never fully developed, yet the process was still managed like a project that had a discipline (i.e. a physical construction project).  But why does it need to be managed that way? If change is relatively cheap, and the discipline is therefore unnecessary, why do treat it like it’s something that it’s not?

The answer is, usually, “That’s what we’re familiar with.”  Fear and doubt play a big part in how we humans make decisions. Managers and executives are used to managing the costs and time on a physical construction project and know how to adapt to problems as they arise — that’s what makes them good managers. So the fundamental shift in thinking rightfully concerns them.  Prudence requires they be skeptical. But, as evidence is clearly mounting that software is fundamentally a different type of project and therefore requires a different style of management, managers and executives must learn to manage in this environment and professional software architects must work with them and they must trust each other and try to reach a solution.

To the questions…

So, now that we’ve established that things might be different with respect to software development project management, questions arise about the most important aspects of business management: Time, Money, and Scope.  How long will it take, how much will it cost, and how can I know what the end will look like?

I plan on covering these specific questions in the next part.

About Chad Myers

Chad Myers is the Director of Development for Dovetail Software, in Austin, TX, where he leads a premiere software team building complex enterprise software products. Chad is a .NET software developer specializing in enterprise software designs and architectures. He has over 12 years of software development experience and a proven track record of Agile, test-driven project leadership using both Microsoft and open source tools. He is a community leader who speaks at the Austin .NET User's Group, the ADNUG Code Camp, and participates in various development communities and open source projects.
This entry was posted in Agile, Agile Arguments. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Remember my 4-d function of business value:

    Value = f (Cost, Time, Scope, Quality)

    Quality is always taking a backseat to everything else.

  • @D.P. -

    Great post! Unless you have an objection, I’m going to link it from the Part 1 kickoff page.

    Something struck me as wrong about your function, though. I think it may be the fact that Quality = Value and so SHOULDN’T require special mention. But as we all know, and as you pointed out, sadly it DOES require special mention because no one believes that Quality = Value.

    Just as ‘Shipping’ is a feature, so is ‘Quality’.

  • Semi-Random-Yet-Related thoughts:

    * Given an environment exhibiting the above 8 points of behavior (or even most of those points), no methodology would produce a quality product. Sadly, these types of behavior are _rampant_ in corporate (and non-corporate) America.

    * The most comprehensive way to find out what is flawed in the operations of any business is to attempt to model that business’ processes in software form.

    * In business software development, successful planning and requirements gathering often requires buy-in from the vertical and buy-in from the horizontal. Vertical buy-in means that the buy-in reaches to the top (or practical top) of the company/department, and horizontal buy-in means the buy-in comprises a sufficient breadth of employees in the problem space. The currency of vertical buy-in is more often just that, currency, and the currency of horizontal buy-in is more often domain knowledge. Both buy-ins require the currency of time as well, granted, the horizontal pays more.

    * No matter what the problem is, it’s always a people problem. -Gerald M. Weinberg