Evolutionary Architecture

A popular cause the Agile folks like to rally against is the idea of a Big Design Up Front (BDUF).  But much like Waterfall, the people doing BDUF will hardly admit that it’s BDUF that they’re doing.  Instead, you’re much more likely to get a bevy of really convincing cautionary tales of what might happen if you don’t do a certain design.  With enough experience under their belt, the architect slips into a “Just-In-Case” design mode, where the myriad of pain points encountered in the past leave a design that’s hardened against every possible difficulty that might come up.

Instead of a well architected design, you’ll have one that’s over-abstracted, over-engineered, over-complicated and very difficult for the next developer to come on board and understand.  But there’s an alternative approach.  The better choice instead of BDUF is a collection of design and architecture techniques that both delay decisions until they’re absolutely necessary, and refactoring practices to evolve our design when assumptions are proved incorrect.

Technique #1: YAGNI

YAGNI stands for “You Aren’t Gonna Need It”.  It’s a generally philosophy of waiting until code actually needs to be written before writing it.  Instead of guessing or projecting on future needs, abstractions or features, the need must present itself in a concrete manner.  This idea can be abused as a way of dismissing true needs for abstractions, so it can take some trust and negotiation in a team when disagreements arise on whether a need has arisen.

The benefit of YAGNI is that you don’t develop features that you might not need.  If you only design based on needs currently evident in the codebase, you’ll never fall into the trap of over-design.  After a design has changed passed the YAGNI tipping point, it can be tempting to castigate yourself, asking “why didn’t we do that sooner”?  Because you didn’t need it sooner.  The trick is to find the right tipping point to guide the next step in design.

Technique #2: Pain-Driven Development

Another common over-engineering technique is refactoring areas that may be ugly, but don’t actually present any problems.  The antidote for refactorbation and speculative design is pain-driven development.  The concept is simple: if it doesn’t hurt, it doesn’t need fixing.  If it hurts, fix it!  The bottom line is that not all areas of your application will have the same sophistication and attention to detail in its design.  But that’s normal, every project works under constraints and you have to focus your time on the most critical areas.

The most critical areas tend to be the ones that change the most.  In Feathers’ Legacy Code book, he recommends refactoring code only when it requires change.  If we only refactor when code needs to change, it naturally leads to the most-changed areas having the best design.  If the most-changed areas are where the most features are added, it tends to follow that the design needs to be more robust in those areas.  By paying attention to pain, we can gauge how much time and effort we need to address in changing a design.  A design may not be ideal, but if it’s not causing much pain, there’s not much ROI on “fixing” the problem.

If you view pain as a cost or a debt, then a design improvement must be measured on how much pain the fix solves, rather than how much better it makes us feel or lets us flex our intellectual muscles.  For long-lasting success, it’s important that we spend our time improving areas that will have the most long-term impact.  Paying attention to pain is a great technique for doing so.

Corollary to Pain-Driven Development

Sometimes a pain can be so slight, but so endemic, that you become numb to pain.  An alternative solution doesn’t seem like it will be that much better, since you’ve become adjusted to the more difficult design or tool.  This tends to close your mind to opportunities that the new design or tool provides.  For example, we used the HBM NHibernate mapping files for a long time because we were all quite familiar with using them, and didn’t cause that much pain.  However, Fluent NHibernate opened new doors that our numbness to the HBM pain was actually an opportunity cost because we couldn’t take advantage of things like conventions to drastically reduce the amount of manual mapping code we needed.

So while the current design may not incur too much pain, we still have to be aware of the opportunities alternative designs might afford us.

Technique #3: Iterative Design

Another big issue I’ve seen with long-lived codebases is that although good design concepts are introduced, they are sparsely, almost randomly applied to the outside observer.  Improving the architecture of a system is good, but leaving a trail of decaying, older designs still in the codebase can cripple further innovation.  When we encounter a system-wide pain, we don’t just fix it in one or two spots.  Instead, we estimate the cost to apply the fix system-wide.  Only once we’ve applied the design system-wide will we try to apply the next iteration in our design.

The benefits to this approach are several.  With several older designs in our system in varying degrees in the evolution of our system, it becomes harder and harder to determine what the “right” design is.  If we have a system-wide concept that we only improve one spot at a time, we wind up with a looooong trail of various designs, and it becomes that much harder to find what the next step is.  When it comes to system-wide architecture and design, the more examples we have of the current design, the better our next choice will be.

Other Techniques

I could go on forever on other techniques, such as DRY, Responsibility-Driven Design, Convention over Configuration, Last Responsible Moment, Simplest Thing that Could Possibly Work and more.  Since some decisions are not easily reversible, we would like to wait until we have as much information as possible before making a long-lasting, costly to reverse decision.  When it comes to choosing between something like Rails or ASP.NET MVC, it would be only responsible to ask as many questions as possible before we make our final decision.  Or choosing between Active Record or the Domain Model pattern.  Are we in that 5% slice that actually needs DDD?  Or is it just 5% of our current application that might require DDD?  These are tough questions to answer, and there’s not always all the information you need to know where to go.

In my experience, I’ve found that choosing the simplest thing that could possibly work is a great starting point, and applying iterative design as we go tends to produce the best final result.

About Jimmy Bogard

I'm a technical architect with Headspring in Austin, TX. I focus on DDD, distributed systems, and any other acronym-centric design/architecture/methodology. I created AutoMapper and am a co-author of the ASP.NET MVC in Action books.
This entry was posted in Design. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Daniel Fernandes

    We all understand the risks of BDUF, although I’m still interested by what D means to people.
    Is it:
    - High level only architecture (like, we’ve got those subsystems with specific roles)
    - Architecture detailed to the nth degree
    - Documents with reams of class diagrams
    - Simply the fact that it’s something produced by an architect or an analyst

    I’ve only seen only twice true BDUF in a shop doing waterfall. Yes it was indeed painful because it was far too detailed. In the first case it didn’t work because it was produced by analysts who stopped coding years ago, whilst on the other case it worked because it was produced by developers who were forced in producing such document but were not held by it.

    Regarding YAGNI, I’m glad you’re warning against it. I’ve seen too often compromised designs because “you don’t need that abstraction”. YAGNI should indeed first be applied to features, and then maybe design. Design quality should be directed by design principles the team is aiming at.


  • jdn

    The trick is finding the right balance between the seeming contradiction of #2 and #3:

    “if it doesn’t hurt, it doesn’t need fixing. If it hurts, fix it! The bottom line is that not all areas of your application will have the same sophistication and attention to detail in its design”


    “Improving the architecture of a system is good, but leaving a trail of decaying, older designs still in the codebase can cripple further innovation”

  • Mehdi Khalili

    Jimmy, Thanks for the post.

    It kinda addresses my question in your previous post; however, what I was and am concerned about is not reversible decisions. I am concerned about things like:
    - client-server vs distributed system
    - data access patterns
    - messaging/communication patterns

    These are kind of questions that you almost always have to answer in the first few months (if not weeks) of project initiation, and unfortunately these are rather the most irreversible of decisions you have to make.

    Of course everything could change to a better design in an iterative approach, but the cost associated with a wrong (or I should say not very well-informed) decision could be high. I wish stakeholders could make up their minds first ;-) that is not alway possible though.


  • Arnis L.

    Jimmy, change the world and come up with detailed instructions about this. I’ve thought about exactly the same thing quite a lot. Main problem is – there aren’t well defined methodology that helps to go through (movement itself) from simple architecture to complex one and would let to identify and unite this idea as a unit. What are exact problems when identifying things that should be more complex, how and when to identify them and what are exact problems when switching to more complex solution and how to avoid them.

    Despite that this would get highly abstract, i think there’s a great potential. At least my intuition says so.

  • Tim

    On point 3: Iterative Design

    What about also recording your decisions for making design/architecture changes somewhere for later reference?

  • RichB

    An average developer can take a lot of pain, and consequently stick with an ugly system for a long long time.

  • Pingback: Software Development Word of the Day « A-Cuppa-Code()