Skip to content

Appendix to June 2, 2020 talk on Software architecture

June 2, 2020

On June 2, 2020, I was invited by the SEI to make a presentation. Slides are posted here.

A recording of the talk is here, but be aware of a possible gap of 8-9 minutes around minute 34; just go “fast forward”.

You will find here pointers to various papers and books I referred to in the talk.

For  my own papers on Software architecture, see this other tab here.

References I made in the talk:

  • Butler W. Lampson. 1983. Hints for computer system design. SIGOPS Oper. Syst. Rev. 17, 5 (October 1983), 33–48. DOI: 10.1145/773379.806614
  • John A. Mills. 1985. A pragmatic view of the system architect. Commun. ACM 28, 7 (July 1985), 708–717. DOI: 10.1145/3894.3897
  • Dewayne E. Perry and Alexander L. Wolf. 1992. Foundations for the study of software architecture. SIGSOFT Softw. Eng. Notes 17, 4 (Oct. 1992), 40–52. DOI: 10.1145/141874.141884
  • Mary Shaw and David Garlan. Software architecture: perspectives on an emerging discipline. Prentice-Hall, Inc., USA. (1996.)
  • Mary Shaw and Paul C. Clements. 1997. A Field Guide to Boxology: Preliminary Classification of Architectural Styles for Software Systems. In Proc. of COMPSAC ’97. IEEE Computer Society, USA, 6–13.
  • M. Fowler, “Design – Who needs an architect?,” in IEEE Software, vol. 20, no. 5, pp. 11-13, Sept.-Oct. 2003, doi: 10.1109/MS.2003.1231144.
  • Simon Brown: Are you a software architect, InfoQ Feb. 2010 https://www.infoq.com/articles/brown-are-you-a-software-architect/
  • Simon Brown, The C4 model for visualising software architecture, https://c4model.com
  • S. Redwine and W. Riddle, “Software Technology Maturation,” Proc. 8th Int’l Conf. Software Eng., IEEE CS Press, 1985, pp. 189–200.
  • David Garlan. 2000. Software architecture: a roadmap. In Proceedings of the Conference on The Future of Software Engineering (ICSE ’00). ACM, New York, NY, USA, 91–101. DOI: 10.1145/336512.336537
  • M. Shaw, “The coming-of-age of software architecture research,” in Proc. IEEE/ACM ICSE ’01, Toronto, Canada, 2001 pp. 657-664a. doi: 10.1109/ICSE.2001.919147
  • M. Shaw and P. Clements, “The golden age of software architecture” in IEEE Software, vol. 23, no. 02, pp. 31-39, 2006. doi: 10.1109/MS.2006.58
  • P. Kruchten, H. Obbink, J. Stafford, “The Past, Present, and Future for Software Architecture” in IEEE Software, vol. 23, no. 2, pp. 22-30, 2006. DOI : 10.1109/MS.2006.59
  • P. Clements and M. Shaw, “The Golden Age of Software Architecture” Revisited. in IEEE Software, vol. 26, no. 04, pp. 70-72, 2009. doi: 10.1109/MS.2009.83
  • David Garlan and Mary Shaw. Software architecture: reflections on an evolving discipline. In Proc. Of ESEC/FSE ’11. ACM, New York, 2011. DOI: 10.1145/2025113.2025116
  • SEI (2017) What is your definition of software architecture, https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=513807
  • SAFE & agile architecture: https://www.scaledagileframework.com/agile-architecture/
  • LESS & architectural design: https://less.works/less/technical-excellence/architecture-design.html

Useful books on Software architecture:

  • L. Bass, P. Clements, R. Kazman, Software Architecture in Practice (3rd Ed.), Addison-Wesley (2012)
  • Ian Gorton, Essential software architecture (2nd ed), Springer (2011)
  • Simon Brown, Software architecture for developers (vol 1 & 2), LeanPub (2018) https://leanpub.com/software-architecture-for-developers
  • N. Rozanski and E. Woods, Software Systems Architecture: Working With Stakeholders Using Viewpoints and Perspectives (2nd ed) Addison Wesley (2011)
  • George Fairbanks, Just Enough Software Architecture: A Risk-Driven Approach , Marshall & Brainerd (2010)
  • F. Buschmann, M. Stal & al., Pattern-Oriented Software Architecture (vol 1 & 2), Wiley (1996, 2000)
  • M. Maier & E. Rechtin, The Art of Systems Architecting, 3rd ed, CRC Press (2009)
  • G. Hohpe, 37 Things One Architect Knows About IT Transformation: A Chief Architect’s Journey, LeanPub (2016).
  • M. Richards & N. Ford, Fundamentals of Software Architecture, OReilly (2020). (Just arrived on my shelf!)

 

Small update to the Mission to Mars game

May 24, 2018

Following the Agile Vancouver Meetup on May 22nd…  see  /https://philippe.kruchten.com/articles/mtm/

If you’re in Vancouver and want to borrow a few games, just email me.

Concrete things you can do about your technical debt

February 14, 2017

“Technical debt, yes we have some of this, but what can we do…?” Here’s a few ideas to get you started:

  1. Organize a lunch-and-learn with your team to introduce the concept of technical debt. Illustrate it with examples from your own projects, if possible.
  2. Create a category “TechDebt” in your issue tracking system, distinct from defects, or new features. Point at the specific artifacts involved.
  3. Standardize on one single form of “Fix me” or “Fix me later” comment in the source code to mark places that should be revised and improved later. They will be easier to spot with a tool.
  4. Acquire and deploy in your development environment a static code analyser to detect code-level “code smells”. (Do not panic in front of the large number of positive warnings).
  5. Prioritize technical debt items to fix or refactor, by doing them first in the parts of your code that are the most actively modified, leaving aside or for later the parts that are never touched.
  6. Organize small 1-hour brainstorming sessions around the question: “What design decision did we make in the past that we regret now because it is costing us much?” or “If we had to do it again, what should have we done?” This is not a blame game, or a whining session; just identify high level structural issues, the key design decisions from the past that have turned to technical debt today.
  7. For identified tech debt items, give not only estimates of the cost to “reimburse” them or refactor them (in staff effort), but also estimate of the cost to not reimburse them: how much it drags the progress now. At least describe qualitatively the impact on productivity or quality. This can be assisted by tools from your development environment, to look at code churn, and effort spent.
  8. At each development cycle, try to constantly reduce some of the technical debt by explicitly bringing some tech debt items into your iteration or sprint backlog.
  9. Refine in your issue tracker the TechDebt category into at least 2 subcategories: simple, localized, code-level debt, and wide ranging, structural or architectural debt.
  10. For your major kinds of technical debt, identify the root cause –schedule pressure, process or lack of process, people availability or turn over, knowledge or lack of knowledge, tool or lack of tool, change of strategy or objectives–  and plan specific actions to address these root causes, or mitigate their effect.
  11. Acquire and deploy a tool that will give you hints about structural issues in your code: dependency analysis.
  12. Develop an approach for systematic regression testing, so that fixing technical debt items does not run you in the risk of breaking the code. (Counter the “It is not really broken, so I won’t fix it.”)
  13. If you are actively managing risks, consider bringing some major tech debt items in your list of risks.

More advice here:
Ambler, S. (2017). 11 Strategies for Dealing With Technical Debt, Blog entry at http://www.disciplinedagiledelivery.com/technical-debt/

Refining the definition of technical debt

April 22, 2016

At a Dagstuhl seminar, sponsored by the Leibniz society, a group of experts tried to refine and agree on a common definition of the elusive concept of “technical debt”:

In software-intensive systems, technical debt consists of design or implementation constructs that are expedient in the short term, but set up a technical context that can make a future change more costly or impossible. Technical debt is a contingent liability whose impact is limited to internal system qualities, primarily maintainability and evolvability.

(starting from Steve McConnell’s definition)

Three “-tures”: architecture, infrastructure, and team structure

October 8, 2014

At a recent workshop, at XP 2014, we looked into practices that support scaling up agile, and in particular the role of architecture.

One conjecture we arrived at is that architects  typically work on three distinct but interdependent structures:

  • The Architecture (A) of the system under design, development, or refinement, what we have called the traditional system or software architecture.
  • The Structure (S) of the organization: teams, partners, subcontractors, and others.
  • The Production infrastructure (P) used to develop and deploy the system, especially important in contexts where the development and operations are combined and the system is deployed more or less continuously.
3-tures

System architecture (A), Organizational structure (S), and Production infrastructure (P)

These three structures must be kept aligned over time, especially to support an agile development style. We can examine the alignment of these structures from the perspective of A and the role of the architect in an agile software-development organization.

The relationship of A to S is also known as (socio-technical) congruence and has been extensively studied, especially in the context of global, distributed software development. It is akin to the good old Conway’s law. It is very pertinent at the level of the static architectural structure (development view), where a development team wants to avoid conflicts of access to the code between teams and between individuals, while having clear ownership or responsibility over large chunks of code. When A is lagging, we face a situation of technical debt; when S is lagging, we have a phenomenon called “social debt” akin to technical debt, which slows down development. See what Ruth Malan wrote on the topic; in particular: “”if the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins”.

The alignment of A with P is seeing renewed interest with increased focus on continuous integration and deployment and the concept of “DevOps” combining the development organization with the operations organization, and having the tools in place to ensure continuous delivery or deployment, even in the case of very large on-line, mission-critical systems (e.g., Netflix, Facebook, Amazon). When P is lagging, we witness a case of “infrastructure debt” as described by Shafer in the Cutter IT journal, which is another source of friction. It is also explored by M. Erder and P. Pureur’s Continuous Architecture.

A, S, and P must be “refactored” regularly to be kept in sync so that they can keep supporting each other. Too much early design in any of the three will potentially result in excessive delays, which will increase friction (by increased debt), reduce quality, and lead to overall product delivery delays.

3-tures

Congruence, DevOps

With my colleagues I. Ozakaya and R. Nord from the SEI, we’ve examined some of this conjecture in a paper to be published by Springer-Verlag later in the fall. I’ll add a DOI when it happens.

More:

Herbsleb & Grinter (1999)  Architectures, Coordination and Distance: Conway’s Law and Beyond, IEEE Software

Managing Technical Debt event in Zürich (Technische Schuld)

December 18, 2013

The Software Engineering  division of the Swiss Informatics Society is organizing a 2-day event on Technical Debt on January 23-24, 2014, at the University of Zürich.

Tutorials on Thursday, presentations on Friday. Pick one, or both.
I will participate on both days. Note that most of the presentations will be in English, even though the site is in German.

Find details and registration, go here.

https://i2.wp.com/si-se.ch/2014/img/sise_logo.png

Agile architecture

December 11, 2013

The phrase “agile architecture” evokes two things:

  1. a system or software architecture that is versatile, easy to evolve, to modify, flexible in a way, while still resilient to changes
  2. an agile way to define an architecture, using an iterative lifecycle, allowing the architectural design to tactically evolve gradually, as the problem and the constraints are better understood

The two are not the same: you can have a non-agile development process leading to a flexible, adaptable architecture, and vice versa, an agile process may lead to a rather rigid and inflexible architecture. One does not imply the other. But for obvious reasons, in the best of worlds, we’d like to have an agile process, leading to a flexible architecture.

There is a naïve thinking that just by being agile, an architecture will gradually emerge, out of bi-weekly refactorings. This belief was amplified by a rather poorly worded principle #11 in the agile manifesto [1], which states that:

“The best architectures, requirements, and designs emerge from self-organizing teams.”

and cemented by profuse amount of repeated mantras like: YAGNI (You Ain’t Gonna Need It) or No BUFD (No Big Up-Front Design), or “Defer decision to the last responsible moment”. (This principle is neither prescriptive, not can it be tested, as Séguin et al. showed in [2], so it is probably not a principle, but merely an observation or a wish.)

This naïve thinking about the spontaneous emergence of architecture is reinforced by the fact that most software endeavors nowadays do not require a significant amount of bold new architectural design: the most important design decisions have been made months earlier, or are fixed by current pre-existing conditions, or are a de facto architectural standard set-up in this industry. Choices of operating system, servers, programming language, database, middle ware, and so on are pre-determined in the vast majority of software development projects or have a very narrow range of possible choices. There is in fact little architectural work left to be done.

Architectural design, when it is really needed because of the project novelty, has an uneasy relationship with the traditional agile practices. Unlike functionality of the system, it cannot easily be decomposed in small little chunks of work, user stories or “technical stories”. Most of the difficult aspects of architectural design are driven by non-functional requirements, or quality attributes: security, high availability, fault tolerance, interoperability, scalability, etc, or development related (testability, certification, and maintainability) which cannot be parceled down, and for which tests are difficult to produce up-front. Key architectural choices cannot be easily retrofitted on an existing system, by means of simple refactorings. Some of the late decisions may gut out large chunks of the code, and therefore whether the agilists like it or not, much of the architectural decisions have to be taken early, although not all at once up front.

Many have grappled with the issue of marrying an agile approach and a need for having a solid architecture. Alistair Cockburn and his “walking skeleton” or the Scaled Agile Framework (SAFe) by Dean Leffingwell & co.

The most common thinking nowadays is that architectural design and the gradual building of the system (i.e., its user visible functionality) must go hand-in-hand, in subsequent iterations, and the delicate issue is actually:  how do we pace ourselves, how we address architectural issues, and make decisions over time in a way that will lead to a flexible architecture, and enable developers to proceed. In which order do we pick the quality attribute aspects and address them?

As for an agile architecture, the concept is not new: evolvability, software evolution, re-engineering of existing systems have been studied and understood for a long time. Manny Lehman started this circa 1980 [5]. The word agile here is just new paint on an old concept.

The book by Simon Brown, Software architecture for the developers [6], is a nice example of agile architecting, while the book by Jason Bloomberg, The agile architecture revolution [7], is a good example of agile architecture.

Note: the full story, in collaboration with my colleagues of the Software Engineering Institute, will appear in the Cutter IT Journal in February 2014 [8], under the tongue in cheek title “How to agilely architect an agile architecture”. Beyond YAGNI…

References

  1. Agile Alliance, Manifesto for Agile Software Development, June 2001 http://agilemanifesto.org/.
  2. N. Séguin, G. Tremblay, and H. Bagane, “Agile Principles as Software Engineering Principles: An Analysis,” vol. 111, Lecture Notes in Business Information Processing, C. Wohlin, Ed. Berlin Heidelberg: Springer, 2012, pp. 1-15.
  3. Scaled Agile Framework, http://scaledagileframework.com/
  4. A. Cockburn, “Walking Skeleton,” http://alistair.cockburn.us/Walking+skeleton
  5. M. M. Lehman, “Programs, lifecycles, and laws of software evolution,” Proceedings of the IEEE – Special issue on software engineering, vol. 68(9),  pp. 1060-1076, 1980.
  6. S. Brown, Software Architecture for the developers, LeanPub, 2013
  7. J. Bloomberg, The agile architecture revolution, Wiley CIO, 2013.
  8. S. Bellomo, P. Kruchten, R. L. Nord, and I. Ozkaya, “How to agilely architect an agile architecture?,” Cutter IT Journal, vol. 27(2),  pp. 12-17, Feb. 2014.