Skip to content

The (missing) value of software architecture

December 11, 2013

One of the most critical questions about software architecture is what is its actual value? As software development processes focus more and more on value delivered to end-users and time-to-market, the difficulty in assigning an actual value on the effort spent working on the architecture makes it much harder for software architects to convince project managers or product owners (or whomever represent the customer) to spend much effort on architecture-related activities, leading often rapidly to a large amount of technical debt [1].

At any point in time, a software development team is faced with a choice: what do we focus on in the next release cycle, or simply the next iteration or sprint. In the “backlog” of things not done yet, there are 4 kinds of elements (Figure from [1]): Image

Items that have visible value:

  • The green stuff: the most obvious items are new features (services, functionalities, capabilities) to be added to the system; sometimes visible improvements in some quality attribute (capacity, response time, interoperability).
  • The red stuff: if the software product is already released, then it is likely to have defects, hurting some customers directly, or indirectly by giving negative press to you

Both have a cost to implement, and some tradeoff has already to be negotiated between parties here: how much defect fixing relative to new features can we afford to do?

But there are also items that are completely invisible to the outside world:

  • The yellow stuff: architectural elements, infrastructure, frameworks, deployment tools, etc. Known to the internal development team, and architects, they often are deferred in favour of more green or red stuff. Their cost is often very lumpy: they are hard to break down to small increments. We do know that they add value, in the long term, by increasing future productivity and often key quality attributes. But this value is hard to define.
  • Finally, the black stuff: there are elements that have both a negative value, and are invisible: big lumps of technical debt. They are the result of earlier architectural decisions that were wise and optimal at the time, but which in the current context are clearly suboptimal and hurt the project in several ways: usually by reduced productivity, or impact on the evolution of the system. The black stuff is known by the development team, but rarely expressed visibly at the level of key decisions makers who decide the future release roadmap. Short cuts or omission to develop the yellow stuff increases the amount of black stuff, further preventing progress.

A compounding factor is that the various elements have many dependencies between them, especially dependencies of the green and red stuff on the yellow or black stuff.  The tradeoffs between the various colours is now much more complicated and require diverse expertise, not just market value. Time plays a crucial role, too: the value of delivering a new feature is immediate; whereas the value of developing a good architecture may be reaped only over a long period of time.

The key issue is how much value is there in the yellow and black stuff? What is really the value of software architecture?

Some economic concept such as Net Present Value, combined with dependency analysis should be able to reconcile all four colours in making development choices for the future, short- or long-term. The Incremental Funding Method by Mark Denne and Jane Cleland-Huang is a step in this direction [2]. Real Options could be another one [3].

To read more about the 4 colours idea, look in my Talks page for a 2009 presentation (file here), or a more recent one 2013 here [4]. And a video of the talk here

References:

  1. P. Kruchten, R. Nord, and I. Ozkaya, “Technical debt: from metaphor to theory and practice,IEEE  Software, vol. 29(6),  pp. 18-21, 2012.
  2. M. Denne and J. Cleland-Huang, “The Incremental Funding Method: Data-Driven Software Development,IEEE Software, vol. 21(3),  pp. 39-47, 2004.
  3. K. J. Sullivan, P. Chalasani, S. Jha, and V. Sazawal, “Software Design as an Investment Activity: A Real Options Perspective,” in Real Options and Business Strategy: Applications to Decision Making, L. Trigeorgis (ed.), Risk Books, 1999.
  4. P. Kruchten, “What colours is your backlog? (slides),” 2009  (updated 2013). Video here.

Friction

November 24, 2013

In his 2000 ICSE Keynote in Limerick, Ireland, my colleague Grady Booch said: “There is still much friction in the process of crafting complex software; the goal of creating quality software in a repeatable and sustainable manner remains elusive to many organizations, especially those who are driven to develop in Internet time.” Friction?

“Friction: the resistance that one surface or object encounters when moving over another.” [Merriam-Webster dict.]

By analogy, in software development, friction is the set of phenomena that limits or constraints our progress, therefore reduces our velocity (or productivity). An element of friction that we have been looking at more closely in the last few years is the result of technical debt: the accumulation of design or coding decisions that looked expedient at the time we made them, but are in retrospect suboptimal, and a hindrance now.

But there is another aspect of friction that is not related to the state of the code, but resides at the organizational and social level. Damian Tamburri, from VU in Amsterdam, has introduced the notion of social debt, as a counter part of technical debt [ICSE2013 workshop]. Social debt is a state of a development project which is the result of the accumulation over time of decisions about the way the development team (or community) communicates, collaborates and coordinates; in other words, decisions about the organizational structure, the process, the governance, the social interactions, or some elements inherited through the people: their knowledge, personality, working style, etc.

Friction in SW

Social debt + Technical debt => Friction => delays, unpredictable schedule, and/or poor quality.

To reduce friction, we have to work in parallel on both aspects, technical and social. An ideal, frictionless project, would have zero technical friction (i.e., a perfect design, and perfect code), and zero social friction: a team that collaborate, communicate and co-ordinate at zero cost, without error. Like in physics, it is impossible to reduce friction to nothing, to eliminate it, to have a completely frictionless development. But at least it gives us something to aspire to.

Friction in everyday language also means “conflict or animosity caused by a clash of wills, temperaments, or opinions” [Merriam-Webster], and for sure we can witness these often in he social relationships of software development teams, but much of the social debt is more subtle in nature.

As we have defects and code smells on the technical side, we can observe on the social side defects and “smells”: not problems but potential source of a series of concrete problems if left not addressed. Examples of social and organizational and social smells Tamburri identified and studied are: Organizational silos, or Prima donnas [paper submitted to ICSE 2014], maybe not issues in themselves in some circumstances, but certainly a potential for many ills.

Friction resulting for social debt is highly dependent on the context, therefore will vary greatly in form and intensity based on [see Octopus]:

  • size of the project (in whatever unit of measure: SLOC, function-Points, person-months, or staff);
  • geographic distribution of the development team (compounded by cultural differences)
  • governance rules imposed externally (Sarbanes-Oxley, Basel III)
  • age of the system (dragging old habits or process form last century)
  • stability of the environment  (commercial/contractual environment, human resources, etc.)
  • business model (internal development, software product, open-source community…)

There may be other aspects of the physics of friction, both static friction and dynamic friction, that could be exploited, making size of the project an analog of the mass, and linear speed to project velocity, and defining a concept of coefficient of friction in presence of various kinds of lubricants.

Your thought?

“Friktion ist der einzige Begriff, welcher dem ziemlich allgemein entspricht, was den wirklichen Krieg von dem auf dem Papier unterscheidet.” Carl von Clausewitz, Vom Krieg, 1832.
(Friction is the one concept that separates real war from a mere paper exercise. My translation)

Interview on Technical debt

September 24, 2012

The nice folks at TechDebt.org have “interviewed” me over the web about technical debt. Transcript available on their web site here.

Podcast on Devnology

September 23, 2012

During my last trip through the Netherlands, I spent an hour with Freek Lemhuis  and  Arne Timmerman from Devnology (a dutch community software development website/blog), and they recorded this podcast, asking me about the agile elephants in the room, RUP, and other topics, such as cognitive biases and reasoning fallacies.

Podcast here: http://devnology.nl/nl/podcast/10-content/224-devnology-podcast-032-philippe-kruchten

Via iTunes: http://itunes.apple.com/podcast/devnology-podcast/id335170002?mt=2&subMediaType=Audio

Agility and architecture koan

November 1, 2011

“Self-organizing teams can decide everything by themselves. So they don’t need an architect.” writes Samudra Kanankearachchi on the Software Architecture group of LinkedIn.

This feels to me like one of these strange agile koans. If you repeat it often and long enough, it will gradually become the truth.
Self-organizing teams have very little to do with the architecture of your system. Self-organizing teams are about: task allocation, collaboration, communication, accountability, … it may have to do with time-boxing and therefore what gets accomplished in a certain time-frame. Architecture is about making decisions (choices) about the structure, composition, organization of the software system. It also feels like “architect” is necessarily not a member of the team. Not my personal experience or recommendation. Architect is a role, not necessarily a person (whose only role is to be an architect, though these exist in large organizations.)

Most systems tackled by small agile teams have a pre-defined stable architecture. So yes, they do not need to have anyone playing the role of architect. For novel complex systems, where architectural decisions need to be made, if they are made “by the team”, it means that the team plays the role of architect, and hopefully they have the knowledge and experience to do so. Like some teams have a scrummaster and a product owner, such teams should have an architecture owner, who drives the discovery of architectural issues and their resolution. Because an architecture is not going to gradually emerge out of weekly refactorings (another agile koan), unless this emergence is guided somehow.

There was a whole issue of IEEE Software magazine last year dedicated to the interplay agility-architecture. Start here: http://www.computer.org/portal/web/computingnow/archive/april2010 . Some of my own prose to be found there.

We do not need richer software process models

March 11, 2011

A group of researchers is looking to issue a Manifesto for Rich Software Process Models. Here’s my position on this topic.

“… perfection is achieved not when there is nothing left to add, but when there is nothing left to take away.”  Antoine de St. Exupéry, Terre des Hommes, 1939, chap.3

Over the last 30 years we have tried very hard  the rich process models, and we have not been extremely successful at it. Maybe we should try lean and mean software process models, rather than making them “richer.” At minimum, we should try to analyze why the rich approaches have not worked; where they failed. Could it be that we were trying to solve the wrong problem? or that the real problems by far overshadow the process model issue? Or maybe the whole construction paradigm we use for software development is not adapted anymore? My position is that we should try the route of very simple software process models, to ensure a wider applicability, greater versatility, and acceptance. Possibly these new process models would be based on other paradigms of software or system development than the “technical-rational” construction idea. I would be wary of richer process models. Read more…

The Daily Stand-up Meeting from Hell

March 4, 2011

Last year, while I was the “academic-in-residence” at Software Education in New Zealand, we wrote and shot this little video, to illustrate all that can go wrong during a daily stand-up meeting. I use it in class to start a discussion on the practice. Ask me if you want the original file (197Mb).

Click here to see the video on Youtube

Daily Stand Up video

Video on YouTube: The daily Stand Up Meeting

.

Creative Commons License
The Daily Stand up by Philippe Kruchten is licensed under a Creative Commons Attribution-NoDerivs 2.5 Canada License.