The (missing) value of software architecture

One of the most critical questions about software architecture is what is its actual value? As software development processes focus more and more on value delivered to end-users and time-to-market, the difficulty in assigning an actual value on the effort spent working on the architecture makes it much harder for software architects to convince project managers or product owners (or whomever represent the customer) to spend much effort on architecture-related activities, leading often rapidly to a large amount of technical debt [1].

At any point in time, a software development team is faced with a choice: what do we focus on in the next release cycle, or simply the next iteration or sprint. In the “backlog” of things not done yet, there are 4 kinds of elements (Figure from [1]):

Items that have visible value:

• The green stuff: the most obvious items are new features (services, functionalities, capabilities) to be added to the system; sometimes visible improvements in some quality attribute (capacity, response time, interoperability).

• The red stuff: if the software product is already released, then it is likely to have defects, hurting some customers directly, or indirectly by giving negative press to you

Both have a cost to implement, and some tradeoff has already to be negotiated between parties here: how much defect fixing relative to new features can we afford to do?

But there are also items that are completely invisible to the outside world:

• The yellow stuff: architectural elements, infrastructure, frameworks, deployment tools, etc. Known to the internal development team, and architects, they often are deferred in favour of more green or red stuff. Their cost is often very lumpy: they are hard to break down to small increments. We do know that they add value, in the long term, by increasing future productivity and often key quality attributes. But this value is hard to define.

• Finally, the black stuff: there are elements that have both a negative value, and are invisible: big lumps of technical debt. They are the result of earlier architectural decisions that were wise and optimal at the time, but which in the current context are clearly suboptimal and hurt the project in several ways: usually by reduced productivity, or impact on the evolution of the system. The black stuff is known by the development team, but rarely expressed visibly at the level of key decisions makers who decide the future release roadmap. Short cuts or omission to develop the yellow stuff increases the amount of black stuff, further preventing progress.

A compounding factor is that the various elements have many dependencies between them, especially dependencies of the green and red stuff on the yellow or black stuff.  The tradeoffs between the various colours is now much more complicated and require diverse expertise, not just market value. Time plays a crucial role, too: the value of delivering a new feature is immediate; whereas the value of developing a good architecture may be reaped only over a long period of time.

The key issue is how much value is there in the yellow and black stuff? What is really the value of software architecture?

Some economic concept such as Net Present Value, combined with dependency analysis should be able to reconcile all four colours in making development choices for the future, short- or long-term. The Incremental Funding Method by Mark Denne and Jane Cleland-Huang is a step in this direction [2]. Real Options could be another one [3].

To read more about the 4 colours idea, look in my Talks page for a 2009 presentation (file here), or a more recent one 2013 here [4]. And a video of the talk here

References:

1. P. Kruchten, R. Nord, and I. Ozkaya, “Technical debt: from metaphor to theory and practice,” IEEE  Software, vol. 29(6),  pp. 18-21, 2012.
2. M. Denne and J. Cleland-Huang, “The Incremental Funding Method: Data-Driven Software Development,” IEEE Software, vol. 21(3),  pp. 39-47, 2004.
3. K. J. Sullivan, P. Chalasani, S. Jha, and V. Sazawal, “Software Design as an Investment Activity: A Real Options Perspective,” in Real Options and Business Strategy: Applications to Decision Making, L. Trigeorgis (ed.), Risk Books, 1999.
4. P. Kruchten, “What colours is your backlog? (slides),” 2009  (updated 2013). Video here.