Agile architecture
Image from http://libarynth.org/project_txoom_design
The phrase “agile architecture” evokes two things:
- a system or software architecture that is versatile, easy to evolve, to modify, flexible in a way, while still resilient to changes
- an agile way to define an architecture, using an iterative lifecycle, allowing the architectural design to tactically evolve gradually, as the problem and the constraints are better understood
The two are not the same: you can have a non-agile development process leading to a flexible, adaptable architecture, and vice versa, an agile process may lead to a rather rigid and inflexible architecture. One does not imply the other. But for obvious reasons, in the best of worlds, we’d like to have an agile process, leading to a flexible architecture.
There is a naïve thinking that just by being agile, an architecture will gradually emerge, out of bi-weekly refactorings. This belief was amplified by a rather poorly worded principle #11 in the agile manifesto [1], which states that:
“The best architectures, requirements, and designs emerge from self-organizing teams.”
and cemented by profuse amount of repeated mantras like: YAGNI (You Ain’t Gonna Need It) or No BUFD (No Big Up-Front Design), or “Defer decision to the last responsible moment”. (This principle is neither prescriptive, not can it be tested, as Séguin et al. showed in [2], so it is probably not a principle, but merely an observation or a wish.)
This naïve thinking about the spontaneous emergence of architecture is reinforced by the fact that most software endeavors nowadays do not require a significant amount of bold new architectural design: the most important design decisions have been made months earlier, or are fixed by current pre-existing conditions, or are a de facto architectural standard set-up in this industry. Choices of operating system, servers, programming language, database, middle ware, and so on are pre-determined in the vast majority of software development projects or have a very narrow range of possible choices. There is in fact little architectural work left to be done.
Architectural design, when it is really needed because of the project novelty, has an uneasy relationship with the traditional agile practices. Unlike functionality of the system, it cannot easily be decomposed in small little chunks of work, user stories or “technical stories”. Most of the difficult aspects of architectural design are driven by non-functional requirements, or quality attributes: security, high availability, fault tolerance, interoperability, scalability, etc, or development related (testability, certification, and maintainability) which cannot be parceled down, and for which tests are difficult to produce up-front. Key architectural choices cannot be easily retrofitted on an existing system, by means of simple refactorings. Some of the late decisions may gut out large chunks of the code, and therefore whether the agilists like it or not, much of the architectural decisions have to be taken early, although not all at once up front.
Many have grappled with the issue of marrying an agile approach and a need for having a solid architecture. Alistair Cockburn and his “walking skeleton” or the Scaled Agile Framework (SAFe) by Dean Leffingwell & co.
The most common thinking nowadays is that architectural design and the gradual building of the system (i.e., its user visible functionality) must go hand-in-hand, in subsequent iterations, and the delicate issue is actually: how do we pace ourselves, how we address architectural issues, and make decisions over time in a way that will lead to a flexible architecture, and enable developers to proceed. In which order do we pick the quality attribute aspects and address them?
As for an agile architecture, the concept is not new: evolvability, software evolution, re-engineering of existing systems have been studied and understood for a long time. Manny Lehman started this circa 1980 [5]. The word agile here is just new paint on an old concept.
The book by Simon Brown, Software architecture for the developers [6], is a nice example of agile architecting, while the book by Jason Bloomberg, The agile architecture revolution [7], is a good example of agile architecture.
Note: the full story, in collaboration with my colleagues of the Software Engineering Institute, will appear in the Cutter IT Journal in February 2014 [8], under the tongue in cheek title “How to agilely architect an agile architecture”. Beyond YAGNI…
References
- Agile Alliance, Manifesto for Agile Software Development, June 2001 http://agilemanifesto.org/.
- N. Séguin, G. Tremblay, and H. Bagane, “Agile Principles as Software Engineering Principles: An Analysis,” vol. 111, Lecture Notes in Business Information Processing, C. Wohlin, Ed. Berlin Heidelberg: Springer, 2012, pp. 1-15.
- Scaled Agile Framework, http://scaledagileframework.com/
- A. Cockburn, “Walking Skeleton,” http://alistair.cockburn.us/Walking+skeleton
- M. M. Lehman, “Programs, lifecycles, and laws of software evolution,” Proceedings of the IEEE – Special issue on software engineering, vol. 68(9), pp. 1060-1076, 1980.
- S. Brown, Software Architecture for the developers, LeanPub, 2013
- J. Bloomberg, The agile architecture revolution, Wiley CIO, 2013.
- S. Bellomo, P. Kruchten, R. L. Nord, and I. Ozkaya, “How to agilely architect an agile architecture?,” Cutter IT Journal, vol. 27(2), pp. 12-17, Feb. 2014.
Trackbacks
- Agile Architecture – perspectives from Philippe Kruchten | systems perestroika – éminence grise
- Agile stories, architecture, and religious processes | Selrahc's Platinum
- “Think big, act small” – what does it mean in architecture? | LEANARCH LIMITED
- Philippe Kruchten on Agile Architecture - ZapThinkTank
- Agile Architecture by Philippe Kruchten | Agile Architecture
- Agile & Architecture | Chess and the Art of Enterprise Architecture
- Agile & Architecture | Enterprise Architecture Professional Journal
- Agile & Architecture – R&A Enterprise Architecture
Philippe Kruchten 
Excellent point… I recently also learned of the existence of yet another buzzword from the resilience community, i.e. Antifragility. If you mix an Agile Organization with an Agile Architecture, do you get an Antifragile Architecture??
Philippe, I’ve really been waiting for someone of your caliber chiming in on this topic, thank you! In our new brave world where notions like ‘just-in-time-architecture’ have started appearing, and where agile development now is mainstream in many places, your perspective is much needed. I’ll post a link on my own blog to your post, this is very important albeit virtually unheard of stuff…
Cheers,
Tommy/Ratscan
Damian, I actually think that the “Antifragile” thinking, coined by Nassim Nicholas Taleb, is very much in line with ‘Agility’ and other modern approaches to complex systems. ‘A robust system stays the same after a chock, an antifragile system gets better from the chock’.
http://en.wikipedia.org/wiki/Antifragile:_Things_That_Gain_from_Disorder
Nice summary, Philippe. For us architects, I think the key to progress on this topic is to stop blaming the agilists for the uneasy relationship, but to look at how we can change from “big up-front designers” to participants in the delivery of our solutions. Looking at architecting as a decision making discipline helps. I drew a picture of how to create an agile-style backlog of architectural concerns in another blogpost on agility and architecture: http://eltjopoort.blogspot.nl/2013/07/architecture-can-be-agile-too.html
Philippe, I would appreciate reference of anyone in particular stating that architecture will gradually emerge just by being Agile. Craftsmanship is since we’re trying to be Agile, never been so important. Architecture has finally become important to everyone involved. At least in my experience.
Gradually letting Architecture emerge is certainly possible, but it is a quite difficult task, demanding discipline and knowledge from teams and constant attention for architecture. Many of us have already delivered complex 24/7 critical systems successfully in this manner. It is also never a black/white thing. As you already mention, no matter how gradually we take architectural decisions, some are already pre-existing. And some (not many) crucial decisions are made up-front. I do find your definition of out-of-box architectures rather simplistic. Architecture today is in imo much more about things closer to users, than choosing underlaying technologies. I would rather call your examples commodities, which definitely have limiting effect on architecture. Still, most of them have become quite cheap to replace later in the project. There are many examples of systems using multiple programming languages in same runtime environment.
Yes, many of quality attributes are difficult to retrofit on existing system. I do find more interesting to know why such system is not allowing me to cheaply improve certain quality. In other words, we should choose our commodity technologies by how easy they are to change or replace.
Principle #11 does not say anything about the way architecture emerges. What it does state is a problem that still very much exists in today’s organisations. Existence of a title architect as someone with more power than the team, while team can and does make better decisions.
It might also be a good idea to never completely separate people aspect on this subject. For me, term Agile Architecture is mainly important because everyone is involved instead of only architects with some special powers.
I would very much appreciate your reply in order to learn, despite my seemingly strong opinion on this.
I do find your definition of out-of-box architectures rather simplistic.
pbk>>> this is a blog post, not a treaty on Software Architecture. My only point was that in many projects there is actually no need for much bold, novel architectural design. Most difficult architectural choices have been made. Reinforcing the impression that architectural design is useless. For more of my views on software architecture see the wikipedia entry: http://en.wikipedia.org/wiki/Software_architecture
Principle #11 does not say anything about the way architecture emerges. What it does state is a problem that still very much exists in today’s organisations. Existence of a title architect as someone with more power than the team, while team can and does make better decisions.
pbk>>>> Then you would agree with me that principle 11 is badly formulated. It mixes two things that are not connected, or not strongly implying each other.
…also, would not be more relevant to compare “Defer decision to the last responsible moment” to BUFD principle? The last one is only mentioned by Agilists, but very much taken for granted by non-agile people. It seems to me more correct to give scientific proof of BUFD, instead of dismissing this agile principle based on testability.
I am only saying that the Principle #11 is not formulated as a principle. (I am not dismissing BUFD nor Defer decision…). And I am actually not the only one saying so; see the study by Séguin et al. Or we have mantras and wishes, and not principles. It’s unfortunate, but the manifesto was gathered hastily in a couple of days.
Agree. Although it is called a principle, I’ve never looked at it in such way. More like a mantra asking for explanation.
Philippe, I agree with you that it’s extremely important to address the “issue of marrying an agile approach and a need for having a solid architecture.” I also believe that the focus should be “evolvability”. I adopt the approach of Adaptable Design Up Front (ADUF):
http://effectivesoftwaredesign.com/adaptable-design-up-front/