1129 lines
148 KiB
TeX
1129 lines
148 KiB
TeX
% ============================================
|
|
% PART II: CAPABILITY PROFILES:
|
|
% THE SEVEN ARCHITECTURAL PATTERNS
|
|
% ============================================
|
|
\part{Capability Profiles: The Seven Architectural Patterns}
|
|
\label{part:patterns}
|
|
|
|
Part~\ref{part:foundations} did not leave the seven patterns as mere names: you met each candidate in one sentence, with its signature strength and its signature cost (Table~\ref{tab:patternpreview}); you saw, dimension by dimension, which structural fact earns a $++$ and which earns a $--$ (Table~\ref{tab:capexamples}); and you watched one complete match run end to end (Table~\ref{tab:minimatch}). This part walks through the seven candidates one at a time -- the layered architecture (L), the modular monolith (MM), the hexagonal architecture or ports-and-adapters pattern (HX), microservices (MS), event-driven architecture (EDA), pipes-and-filters (PF), and serverless computing (SL) -- and every section follows the same rhythm: what the pattern \emph{is} (definition and topology, with the variants you will meet in practice); what real-world problem it was invented to solve, with named production systems; how it behaves on the twelve dimensions of Table~\ref{tab:dimensions} and why -- a compact table walks D1--D12 in order, rating and structural reason side by side, before the prose takes up the cells that need a story; what the pattern does to your engineering day -- build, test, CI/CD, deployment, operations, maintenance, team structure; where you can build it and inspect it in running open-source code (a ``Build it and study it'' box per pattern; Table~\ref{tab:osscatalog} at the end of the part collects all seven into one reference card); and, closing each section, the anti-patterns with measurable alarms and the signals for choosing or avoiding the pattern.
|
|
|
|
The machinery for comparing all seven side by side -- the consolidated capability table, the reading rules that keep it honest, the two structural variables that compress most of it, and the evidence base with its caveats -- comes deliberately at the \emph{end} of the part (Section~\ref{sec:pat-method}), once you have seen all seven cases. The order is the point. Software engineering is not mathematics: its patterns were not derived from axioms but abstracted from systems that worked and systems that failed expensively, and this part follows that order of discovery -- examples first, generalisation after.
|
|
|
|
One piece of Part~\ref{part:foundations} is worth restating before the first pattern, because every section speaks its vocabulary -- the capability profile, recalled here:
|
|
\label{subsec:pat-profile-def}
|
|
|
|
\begin{definitionbox}[Capability profile]
|
|
The \emph{capability profile} of an architectural pattern $p$ is the vector
|
|
\[
|
|
C(p) = \bigl(c_1(p), \dots, c_{12}(p);\; S(p)\bigr),
|
|
\]
|
|
where each $c_i(p) \in \{++,\, +,\, \circ,\, -,\, --\}$ is an \emph{ordinal} rating of how well the pattern structurally supports profile dimension $D_i$ of Table~\ref{tab:dimensions}, justified by the architectural tactics the pattern bundles or impedes, and $S(p)$ is the pattern's \emph{native workload shape} -- the interactive request/response, continuous stream, scheduled batch, or event-triggered mode of operation for which its topology was conceived.
|
|
\end{definitionbox}
|
|
|
|
One sentence on where the ratings come from must suffice for now: every rating in this part is \emph{anchored} -- in the published star ratings of \citet{richards2025fundamentals} where those exist, and in primary pattern literature and documented production systems where they do not -- and every deviation from the source ratings is footnoted. What that evidence base can and cannot support, and where its honest gaps lie, is discussed in full in the closing section of this part (Section~\ref{sec:pat-method}), at the point where you know what is being rated.
|
|
|
|
% ============================================
|
|
% SECTION: L -- LAYERED ARCHITECTURE / 3-TIER
|
|
% ============================================
|
|
\section{L -- Layered Architecture / 3-Tier}
|
|
\label{sec:pat-layered}
|
|
|
|
\emph{Two of you must ship a working product by December: what is the simplest structure that does the whole job?} Every architect's first candidate answers exactly that question -- and this section prices the answer.
|
|
|
|
\subsection{Definition and topology}
|
|
|
|
The layered architecture is the modal enterprise structure of the last three decades and the natural baseline of the catalogue. It decomposes a system \emph{horizontally}, by technical role, into layers with a directed dependency rule; the classical codification is the \emph{Layers} pattern of \citet{buschmann1996posa}, and the enterprise application patterns of \citet{fowler2002poeaa} presuppose it. ``3-tier'' names its physical distribution: presentation tier, application tier, database tier. In the taxonomy of \citet{richards2025fundamentals}, the pattern is \emph{technically partitioned} -- its units are cut by technical role, not by business capability -- and forms exactly one \emph{architecture quantum}: one unit that is deployed, scaled, and fails as a whole. Keep both terms in view; they recur in every section of this part, and the closing generalisation (Section~\ref{sec:pat-method}) shows how much of the seven profiles these two structural facts alone explain.
|
|
|
|
\begin{definitionbox}[Layered architecture (L)]
|
|
A macro-structure that partitions a system into horizontal technical layers -- typically presentation, business logic, persistence, and database -- with a directed dependency rule: each layer may use only the layer(s) beneath it. The whole system is deployed, scaled, and fails as one unit: exactly one architecture quantum \citep{buschmann1996posa,richards2025fundamentals}.
|
|
\end{definitionbox}
|
|
|
|
Figure~\ref{fig:pat-layered} shows the topology, the single quantum boundary, and the cross-cutting path of a typical feature change.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
sysbox/.style={rectangle, draw, rounded corners=4pt, minimum width=3.2cm, minimum height=1.2cm, align=center, font=\small\sffamily, line width=0.8pt},
|
|
layer/.style={sysbox, fill=bankgreen!15, draw=bankgreen, minimum width=7.2cm, minimum height=0.95cm},
|
|
extern/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=7.2cm, minimum height=0.95cm},
|
|
arr/.style={-{Stealth[length=2.5mm]}, thick, gray!60!black}
|
|
]
|
|
\node[layer] (pres) at (0,0) {Presentation layer};
|
|
\node[layer, below=0.45cm of pres] (biz) {Business layer};
|
|
\node[layer, below=0.45cm of biz] (pers) {Persistence layer};
|
|
\node[extern, below=0.45cm of pers] (db) {Database};
|
|
\draw[arr] (pres) -- (biz);
|
|
\draw[arr] (biz) -- (pers);
|
|
\draw[arr] (pers) -- (db);
|
|
% quantum boundary
|
|
\draw[dashed, gray!60!black, rounded corners=6pt]
|
|
([xshift=-0.5cm,yshift=0.35cm]pres.north west) rectangle ([xshift=0.5cm,yshift=-0.35cm]db.south east);
|
|
\node[font=\footnotesize\sffamily, text=gray!60!black, below=0.15cm of db, yshift=-0.35cm, align=center]
|
|
{one architecture quantum: one unit of deployment, scaling, and failure};
|
|
% feature change arrow
|
|
\draw[-{Stealth[length=2.5mm]}, thick, bankred, dashed]
|
|
([xshift=-1.3cm]pres.west) -- ([xshift=-1.3cm]db.west)
|
|
node[midway, left, font=\footnotesize\sffamily, text=bankred, align=center] {typical\\feature\\change};
|
|
\end{tikzpicture}%
|
|
\caption{Topology of the layered architecture: horizontal technical layers with a top-down dependency rule inside a single architecture quantum. The dashed red arrow marks the pattern's structural weakness: a typical business-facing feature cuts across every layer, which is the root cause of the $-$ rating on evolvability (D7).}
|
|
\label{fig:pat-layered}
|
|
\end{figure}
|
|
|
|
\subsection{Variants}
|
|
|
|
Three variation axes recur in practice. (a)~\emph{Closed versus open layers}: whether a request may bypass a layer (layer bridging) or must pass through each one; open layers trade isolation for performance. (b)~\emph{Strict versus relaxed layering}: whether a layer may see only its direct neighbour or all layers beneath it. (c)~\emph{Technical versus domain-oriented layering} -- the core criticism in \citet{richards2025fundamentals}: technical partitioning cuts orthogonally to business change, so a typical feature touches all layers. Above the macro-structure, the classical enterprise patterns organise the business layer itself -- Transaction Script, Domain Model, Service Layer \citep{fowler2002poeaa}. Physically, the same logical layering deploys as two, three, or $n$ tiers.
|
|
|
|
\subsection{The problem it solves}
|
|
|
|
The problem is as old as commercial computing. A business needs a tool -- capture orders, look up customers, post invoices -- and it needs it soon: a small team, one relational database, a deadline measured in months. Nothing about the requirement is exotic: forms over data, a modest set of business rules, one consistent store of record. What such a project needs first is not elasticity or independent deployability; it is a structure that a handful of developers can hold in their heads while shipping.
|
|
|
|
Layering was the natural first answer because it mirrors both the technology and the team. The stack already separates concerns -- something renders screens, something executes rules, something persists rows -- and so do the skills in the room: interface developers, application developers, database specialists. Cutting the system along those seams costs almost no design effort, and adding the one rule that keeps the cut from collapsing -- dependencies point downwards only -- turns the habit into a pattern \citep{buschmann1996posa}. The reward is exactly what a small team under deadline wants: one build, one artefact, one ACID database, one thing to operate.
|
|
|
|
The pattern is the de-facto standard of enterprise applications since the 1990s: Spring MVC, .NET, Rails and Django in their default configurations all scaffold it. Metabase deliberately ships its entire BI serving tier as a single JAR for exactly the deployability-of-one-artefact reason \citep{metabase2024docs}. Practically every CRUD tool built since the 1990s -- inventory screens, admin consoles, booking and reporting systems -- is this pattern, whether or not its builders ever named it. What the pattern was never designed to give, and does not give, is independent scaling and change isolation: the problem it solves is shipping the whole tool cheaply, not evolving its parts independently -- and the profile below prices that trade cell by cell.
|
|
|
|
\subsection{Capability profile}
|
|
|
|
With the topology and the problem it solves in mind, the twelve-dimension mapping follows almost mechanically. Table~\ref{tab:cap-l} walks column L of the consolidated capability table (Table~\ref{tab:capability}, assembled at the end of this part once all seven patterns have been seen) through all twelve dimensions, naming for each rating the structural fact that earns it; the note numbers refer to that table's notes, where every deviation from the published source ratings is documented.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{The layered architecture across the twelve dimensions -- column L of Table~\ref{tab:capability}; note numbers refer to that table's notes.}
|
|
\label{tab:cap-l}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{3.2cm}cp{8.5cm}@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{Rating} & \textbf{Structural reason} \\
|
|
\midrule
|
|
D1 Read scalability & $\circ$ & stateless replication of the application tier plus caching scales reads; the single write path remains the limit (note~1) \\
|
|
D2 Write scalability \& elasticity & $--$ & every write funnels through one database inside one quantum -- no partitioned ingest, no elastic consumers \\
|
|
D3 Latency \& predictability & $+$ & in-process calls: no network hop, no serialisation, no tail amplification (note~7) \\
|
|
D4 Consistency \& integrity & $++$ & one ACID transaction boundary under one process \\
|
|
D5 Availability \& fault isolation & $-$ & one process: blast radius $100\,\%$ by construction \\
|
|
D6 Security \& auditability & $+$ & one audit log and one small compliance scope \\
|
|
D7 Evolvability \& maintainability & $-$ & technical partitioning: a typical feature cuts across every layer \\
|
|
D8 Simplicity \& time-to-market & $++$ & one repository, one pipeline -- the lowest cognitive and operational entry threshold in the catalogue \\
|
|
D9 Testability \& deployability & $-$ & any change, however local, redeploys the whole artefact with full regression scope \\
|
|
D10 Operating cost efficiency & $++$ & one cheap deployable, classical monitoring, near-zero platform staff \\
|
|
D11 Team scaling (Conway) & $-$ & one quantum, one release train: effectively a single-team pattern (note~19) \\
|
|
D12 AI integrability & $\circ$ & an AI call is easy to host but receives no structural boundary, no queue, and no measurement point \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & \multicolumn{2}{l}{interactive} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
The table gives the full profile; three cells deserve the story behind the number -- the read/write split, the latency deviation, and the negative change axis:
|
|
|
|
\begin{itemize}
|
|
\item \textbf{D1 read scalability $= \circ$, D2 write scalability $= --$.} \citet{richards2025fundamentals} rate scalability at one star, but that single star conflates two dimensions this theory separates. Stateless replication of the application tier plus aggressive caching demonstrably scales \emph{reads} in this pattern class -- Stack Overflow's nine web servers \citep{craver2016stackoverflow} and Instagram's replicated Django monolith \citep{woodruff2019static} are the documented existence proofs -- while the single write path through one database remains the structural bottleneck. Hence the split: $\circ$ for reads, $--$ for writes (table notes~1).
|
|
\item \textbf{D3 latency $= +$.} A deviation from the two-star performance rating: in-process calls give low and, more importantly, \emph{predictable} latency -- no network hops, no serialisation, no tail amplification. The two stars reflect throughput under load, which D1/D2 capture separately (note~7).
|
|
\item \textbf{D7 evolvability $= -$, D9 testability \& deployability $= -$.} Both follow from technical partitioning and the single quantum: a feature disperses across all layers, and any change -- however local -- redeploys the whole artefact with full regression scope. D11 team scaling $= -$ (note~19): one quantum means one release train, effectively a single-team pattern of up to roughly eight to ten developers \citep{skelton2019team}; the rating is kept at $-$ rather than $--$ to preserve ordinal distance to the modular monolith's $\circ$.
|
|
\end{itemize}
|
|
|
|
\subsection{Software engineering implications}
|
|
|
|
\textbf{Build and CI/CD.} One build, one pipeline, one artefact: continuous integration is trivially simple to set up, but release cadence is bounded by full-regression scope -- the pipeline is cheap, its gate is expensive. \textbf{Test.} In-process integration tests are fast and need no test doubles across network boundaries; the risk is test suites coupled to technical layers, so that refactoring a layer breaks hundreds of tests that verify structure rather than behaviour. Regression effort grows with the system, not with the change. \textbf{Deployment and operations.} Whole-system releases and whole-system rollbacks; one process to operate; classical application performance monitoring suffices, and no distributed tracing is needed -- an underrated operational virtue. \textbf{Maintenance.} Without enforced conformance the pattern decays predictably (see anti-patterns below); Lehman's second law -- complexity increases unless work is invested to reduce it -- has no structural counterweight here \citep{lehman1980laws}. \textbf{Team size (Conway).} One team; organising sub-teams by layer (a UI team, a database team) reproduces the layer boundaries as hand-off friction, the textbook Conway effect \citep{conway1968committees,skelton2019team}.
|
|
|
|
\subsection{Exemplars, anti-patterns, and selection signals}
|
|
|
|
\begin{examplebox}[Build it and study it -- layered]
|
|
\textbf{Build.} Django (Python, BSD-3, very active) scaffolds the pattern out of the box: its model--view--template split \emph{is} an enforced persistence/logic/presentation layering \citep{django2026web}. The canonical Java stack is Spring Boot MVC (Apache-2.0), with its Controller $\rightarrow$ Service $\rightarrow$ Repository convention \citep{springboot2026}.
|
|
|
|
\textbf{Study.} \texttt{healthchecks/healthchecks} ($\sim$10k stars, BSD-3, active in 2026) is a production Django monolith worth an afternoon: under \texttt{hc/} sit four Django apps (\texttt{accounts}, \texttt{api}, \texttt{front}, \texttt{payments}), each with clean \texttt{models.py}/\texttt{views.py}/template layering \citep{healthchecks2026github}. It runs locally with \texttt{pip install -r requirements.txt \&\& ./manage.py runserver} on default SQLite -- the easiest start in this catalogue.
|
|
|
|
\textbf{A habit to build now:} before adopting \emph{any} system in these boxes -- or anywhere -- check its licence and its maintenance status (last commit, open issues). The boxes below deliberately leave the findings visible: a no-derivatives licence here, an archived flagship there, a framework that quietly left plain open source. Verifying this is part of the engineering, and it takes two minutes.
|
|
\end{examplebox}
|
|
|
|
\begin{examplebox}[Stack Overflow -- a layered monolith at planetary scale]
|
|
Stack Overflow serves on the order of 1.3~billion page views per month, with peaks above 6{,}000 requests per second, from roughly nine on-premise web servers in front of a SQL~Server cluster whose working set is RAM-resident, plus a Redis cache -- with render times around 12\,ms \citep{craver2016stackoverflow,infoq2015stackoverflow,milanovic2023stackoverflow}. The strategy is scale-up plus cache-friendliness instead of scale-out. Read the case through the profile, not through the headline: a read-heavy, cache-friendly workload is exactly what note~1 describes -- stateless replication and caching carry D1 -- so the case is a documented counter-example to ``scale requires microservices'' \emph{and} simultaneously evidence for the single-write-path limit that keeps D2 at $--$.
|
|
\end{examplebox}
|
|
|
|
\paragraph{Anti-patterns and measurable alarms.} The \emph{architecture sinkhole} arises when requests pass through layers that add no logic -- pure delegation ceremony; the working rule of thumb in \citet{richards2025fundamentals} is that more than about 80\,\% pass-through requests signal that the layering is structure without function. The alarm is measurable: instrument the share of requests traversing all layers untransformed, and track \emph{change dispersion} (layers touched per average feature -- the D7 response measure of Table~\ref{tab:dimensions}). Unguarded, the pattern decays into the \emph{big ball of mud} \citep{foote1997bigball}; the countermeasure is the same dependency-rule-as-CI-gate machinery introduced for the modular monolith in Section~\ref{sec:pat-mm}, and the alarm is a rising count of cross-layer dependency violations.
|
|
|
|
\paragraph{Selection signals.} \textbf{Choose when:} the team is small ($\leq$ roughly ten), budget and schedule are tight, the domain is not yet understood, the workload is CRUD-dominant at modest scale, and one ACID database satisfies the consistency requirement. \textbf{Avoid when:} subdomains need independent scaling, the change rate per subdomain is high, fault isolation is required, or several teams must release independently -- each of these is a one-star axis of this pattern.
|
|
|
|
\begin{ailinse}[The layered pattern and AI components]
|
|
D12 $= \circ$, and the rating is instructive because the problem is not difficulty but \emph{ease}: nothing stops a developer from calling an LLM synchronously from a business-layer service -- and that is precisely the risk. The pattern offers the non-deterministic component no structural boundary, no queue, and no measurement point, which are the three things a slow, fallible, per-call-priced component needs most (Part~\ref{part:ai}). Token costs are invisible to the classical APM tooling this pattern is operated with, and non-determinism leaks freely across layer boundaries, which encode technology, not trust. Hosting an AI call is easy; containing one is unsupported.
|
|
\end{ailinse}
|
|
|
|
\begin{keypoint}
|
|
The layered architecture buys the lowest entry cost in the catalogue ($++$ on D8 and D10, an honest $++$ on D4) and pays for it on every axis of change and scale (D2, D5, D7, D9, D11 all negative). That is not a defect; it is a specific trade -- maximal day-one simplicity against minimal structural options later. The pattern is mis-chosen only when the requirements profile weights the options it sold off.
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: MM -- MODULAR MONOLITH
|
|
% ============================================
|
|
\section{MM -- Modular Monolith}
|
|
\label{sec:pat-mm}
|
|
|
|
\emph{You want monolith economics but fear the big ball of mud: can hard boundaries live inside one deployable?} This pattern's answer is yes -- provided the boundaries are checked by a machine, not by good intentions.
|
|
|
|
\subsection{Definition and topology}
|
|
|
|
The modular monolith keeps the deployment topology of the monolith -- one artefact, one process, one quantum -- and replaces its \emph{internal} organisation: instead of technical layers, domain-oriented modules with enforced boundaries. Each module encapsulates a \emph{bounded context} -- domain-driven design's name for a business subdomain with its own self-consistent vocabulary and model \citep{evans2003ddd} -- owns its persistence, and communicates with other modules only through explicit public APIs or in-process events. The pattern has been an unnamed practice for decades; \citet{richards2025fundamentals} promote it to a style with its own chapter and rating profile in the second edition, and \citet{su2024modular} document it as a measurable industry trend.
|
|
|
|
\begin{definitionbox}[Modular monolith (MM)]
|
|
A single deployment artefact (one process, one architecture quantum) whose interior is partitioned by \emph{domain} into modules with enforced boundaries: each module encapsulates one bounded context including sovereignty over its own persistent data, and inter-module communication passes exclusively through published module APIs or in-process events. Boundary conformance is verified automatically \citep{richards2025fundamentals,evans2003ddd,drotbohm2023modulith}.
|
|
\end{definitionbox}
|
|
|
|
Figure~\ref{fig:pat-mm} shows the topology: domain-cut modules with owned persistence inside one quantum boundary.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
sysbox/.style={rectangle, draw, rounded corners=4pt, align=center, font=\small\sffamily, line width=0.8pt},
|
|
module/.style={sysbox, fill=bankblue!20, draw=bankblue, minimum width=3.0cm, minimum height=1.1cm, font=\small\sffamily\bfseries},
|
|
data/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=3.0cm, minimum height=0.8cm, font=\footnotesize\sffamily},
|
|
arr/.style={-{Stealth[length=2.5mm]}, thick, gray!60!black},
|
|
darr/.style={{Stealth[length=2.5mm]}-{Stealth[length=2.5mm]}, thick, bankgreen}
|
|
]
|
|
\node[module] (m1) at (-4.2,0) {Portfolio\\module};
|
|
\node[module] (m2) at (0,0) {Instrument\\module};
|
|
\node[module] (m3) at (4.2,0) {Reporting\\module};
|
|
\node[data, below=0.5cm of m1] (d1) {owned tables};
|
|
\node[data, below=0.5cm of m2] (d2) {owned tables};
|
|
\node[data, below=0.5cm of m3] (d3) {owned tables};
|
|
\draw[arr] (m1) -- (d1);
|
|
\draw[arr] (m2) -- (d2);
|
|
\draw[arr] (m3) -- (d3);
|
|
\draw[darr] (m1) -- node[above, font=\footnotesize\sffamily, text=bankgreen] {public API /} node[below, font=\footnotesize\sffamily, text=bankgreen] {in-process events} (m2);
|
|
\draw[darr] (m2) -- node[above, font=\footnotesize\sffamily, text=bankgreen] {public API /} node[below, font=\footnotesize\sffamily, text=bankgreen] {in-process events} (m3);
|
|
% single deployable boundary
|
|
\draw[rounded corners=6pt, line width=1pt, gray!60!black]
|
|
(-6.1,1.1) rectangle (6.1,-2.6);
|
|
\node[font=\footnotesize\sffamily, text=gray!60!black, align=center] at (0,-3.05)
|
|
{single deployable: one process, one architecture quantum};
|
|
\node[font=\footnotesize\sffamily, text=bankred, align=center] at (0,1.55)
|
|
{boundary conformance verified in CI (ArchUnit, Spring Modulith, Packwerk)};
|
|
\end{tikzpicture}%
|
|
\caption{Topology of the modular monolith: domain-oriented modules with owned persistence and explicit communication paths inside a single deployable. The boundaries are not a diagram convention but a CI subject -- violations fail the build.}
|
|
\label{fig:pat-mm}
|
|
\end{figure}
|
|
|
|
\subsection{Variants}
|
|
|
|
(a)~\emph{Convention plus verification}: package or namespace conventions whose boundary rules are enforced by architecture tests (ArchUnit and its analogues) \citep{archunit2024userguide}. (b)~\emph{Framework-supported}: Spring Modulith -- generally available since August 2023 -- verifies module boundaries in tests, records event publication, and generates module documentation from the code itself \citep{drotbohm2023modulith}. (c)~\emph{Rails Engines plus Packwerk}, the Shopify lineage \citep{westeinde2019deconstructing,shopify2020under}. (d)~The \emph{distributed-ready monolith}: modules cut so that any one of them can later be extracted into a service along a strangler seam \citep{fowler2004strangler} -- the cheapest option purchase on a future microservices decision (Maxim~5, Part~\ref{part:fit}).
|
|
|
|
\subsection{The problem it solves}
|
|
|
|
The modular monolith answers a failure mode that every successful layered system eventually demonstrates: growth without boundaries. As features accumulate, a technically layered codebase gives change no natural home -- each feature smears across presentation, logic, and persistence -- and without a structural counterweight the system decays into the big ball of mud \citep{foote1997bigball}. Through the 2010s the fashionable escape was to distribute: cut the system into services and let the network enforce the boundaries. But distribution charges rent -- platform staffing, eventual consistency, operational complexity -- whether or not the organisation needed anything beyond the boundaries themselves. Hence the precise form of the problem: how does a team keep monolith economics -- one build, one deployable, one ACID database, one thing to operate -- and still get hard, change-absorbing domain boundaries?
|
|
|
|
The best-documented answer comes from a company that met the question at extreme scale and answered it deliberately.
|
|
|
|
\begin{examplebox}[Shopify -- the canonical modular monolith]
|
|
Shopify runs one of the largest Rails codebases in existence -- more than 2.8~million lines of Ruby -- and decided deliberately \emph{against} microservices: since 2017 the codebase has been reorganised into components with enforced boundaries, checked by the purpose-built tool Packwerk, later restructured around Rails Engines \citep{westeinde2019deconstructing,shopify2020under}. Scaling is handled by sharding whole monolith instances into pods, which carried Black Friday/Cyber Monday peaks of roughly 280~million requests per minute \citep{shopify2024bfcm}. The case demonstrates both halves of the profile: microservices-grade modularity (D7) at monolith-grade cost (D10), and the documented mitigation that lifts D2 from $--$ to $-$ (note~4).
|
|
\end{examplebox}
|
|
|
|
Further documented exemplars: Apache Fineract, a core-banking platform built as a modular monolith \citep{fineract2024core}; the ERP systems Odoo and ERPNext, whose module-over-shared-ACID-model structure is the pattern in its purest commercial form \citep{odoo2024docs,frappe2024erpnext}; and Mattermost, which ships its entire collaboration server as a single Go binary \emph{by design}, because one artefact is a feature for self-hosting and compliance customers \citep{mattermost2024docs}. Mastodon demonstrates that even social-media fan-out mechanics fit in a Rails monolith at non-trivial scale \citep{mastodon2024docs}.
|
|
|
|
\subsection{Capability profile}
|
|
|
|
With the topology and Shopify's problem in mind, the mapping follows. Table~\ref{tab:cap-mm} walks column MM of Table~\ref{tab:capability} (assembled at the end of this part) through all twelve dimensions, naming for each rating the structural fact that earns it.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{The modular monolith across the twelve dimensions -- column MM of Table~\ref{tab:capability}; note numbers refer to that table's notes.}
|
|
\label{tab:cap-mm}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{3.2cm}cp{8.5cm}@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{Rating} & \textbf{Structural reason} \\
|
|
\midrule
|
|
D1 Read scalability & $\circ$ & the same single quantum as L: replication and caching scale reads, the single write path remains (note~1) \\
|
|
D2 Write scalability \& elasticity & $-$ & the single quantum still funnels writes through one database; tenant or pod sharding of whole-monolith instances is the documented mitigation that lifts it from $--$ (note~4) \\
|
|
D3 Latency \& predictability & $+$ & in-process calls between modules: no network hop between bounded contexts \\
|
|
D4 Consistency \& integrity & $++$ & one process, one ACID transaction scope across all modules \\
|
|
D5 Availability \& fault isolation & $-$ & a single process remains a single blast radius; replication mitigates \\
|
|
D6 Security \& auditability & $+$ & one audit log and one compliance scope, with module boundaries as policy seams \\
|
|
D7 Evolvability \& maintainability & $+$ & the domain cut absorbs a typical feature inside one module (Maxim~4) \\
|
|
D8 Simplicity \& time-to-market & $+$ & monolith-simple minus one step: boundary governance is a permanent line item \\
|
|
D9 Testability \& deployability & $+$ & module APIs are natural test seams and enable module-scoped test selection \\
|
|
D10 Operating cost efficiency & $++$ & monolith-cheap operations: one artefact, one process, classical monitoring \\
|
|
D11 Team scaling (Conway) & $\circ$ & roughly three to five teams can share one release train if they accept coordination \\
|
|
D12 AI integrability & $+$ & a hard, CI-verifiable module boundary contains the AI subsystem at monolith cost \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & \multicolumn{2}{l}{interactive} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
The table gives the full profile; its most instructive reading is \emph{against Table~\ref{tab:cap-l}}, because the comparison isolates the effect of the partitioning axis (stated as Maxim~4 in the closing section of this part) at a constant quantum count: D4 and D10 are unchanged, while D7 and D9 each rise from $-$ to $+$ -- domain partitioning, not distribution, buys the evolvability -- at the price of one step on D8. Beyond that comparison, one cell deserves the story behind the number: D2 $= -$ rather than $--$ (note~4), because tenant or pod sharding of whole-monolith instances is a documented write-scaling path -- Shopify's Black Friday/Cyber Monday peaks of roughly 280~million requests per minute run on sharded monolith pods \citep{westeinde2019deconstructing,shopify2024bfcm}. D5's mitigation by replicated instances returns in the fit matrix cells of Part~\ref{part:fit}.
|
|
|
|
The decisive software engineering property of the pattern deserves its own statement: \textbf{architecture conformance becomes a CI subject}. The declared structure is executable -- a failing test, not a slide.
|
|
|
|
\begin{lstlisting}[language={}, caption={A module-boundary fitness function in ArchUnit-style pseudocode \citep{archunit2024userguide,drotbohm2023modulith,ford2022evolutionary}. Boundary erosion breaks the build instead of accumulating silently.}, label={lst:boundary}]
|
|
rule "module boundaries hold" {
|
|
classes in module("portfolio")
|
|
may only be accessed through its published API;
|
|
no cycles between modules();
|
|
domain packages must not depend on framework packages;
|
|
no module accesses tables owned by another module;
|
|
}
|
|
\end{lstlisting}
|
|
|
|
\subsection{Software engineering implications}
|
|
|
|
\textbf{Build and CI/CD.} Still one build and one pipeline, but the domain cut enables module-scoped test selection: a change in the reporting module runs the reporting tests, not the world. Architecture verification (Listing~\ref{lst:boundary}) runs on every build -- the first fitness function most teams ever write \citep{ford2022evolutionary}. \textbf{Test.} Module APIs are natural test seams; hermetic module tests replace whole-system fixtures. \textbf{Deployment and operations.} Identical to the monolith, and identically cheap: one artefact, one process, classical monitoring. \textbf{Maintenance.} The failure mode is boundary erosion; the countermeasure is automated verification, because Lehman's second law never sleeps and code review alone demonstrably does not hold the line \citep{lehman1980laws,drotbohm2023modulith}. \textbf{Team size (Conway).} Roughly three to five teams, each owning modules, with accepted release coordination; beyond that, the single release train becomes the binding constraint and D11 is exhausted \citep{skelton2019team}.
|
|
|
|
\subsection{Exemplars, anti-patterns, and selection signals}
|
|
|
|
\begin{examplebox}[Build it and study it -- modular monolith]
|
|
\textbf{Build.} Spring Modulith (Java, Apache-2.0, active) turns module boundaries into first-class artefacts: verification tests, recorded event publication, and generated module documentation inside one Spring Boot deployable \citep{drotbohm2023modulith}. In Python the same discipline is a linter: \texttt{import-linter} (BSD-2, active) declares layer and independence contracts between packages and fails CI on any violation \citep{importlinter2026} -- remember it, it returns unchanged for the hexagonal pattern.
|
|
|
|
\textbf{Study.} Apache Fineract ($\sim$2.3k stars, Apache-2.0, active), the core-banking platform met above among the documented exemplars, is the pattern at industrial scale: 34 \texttt{fineract-*} Gradle modules (\texttt{fineract-loan}, \texttt{fineract-accounting}, \texttt{fineract-savings}, \dots) composed into \emph{one} deployable, \texttt{fineract-provider}, plus a \texttt{custom/} directory for extensions -- Figure~\ref{fig:pat-mm} in a real build file \citep{fineract2024core}. It runs via the provided \texttt{docker-compose-postgresql.yml}: moderate effort (JVM, heavy). For a first contact, the bookstore sample \texttt{sivaprasadreddy/spring-modular-monolith} (Apache-2.0, active) is far smaller \citep{reddy2026modularmonolith}.
|
|
\end{examplebox}
|
|
|
|
\paragraph{Anti-patterns and measurable alarms.} \emph{Boundary erosion without automated verification} -- the slow relapse into the big ball of mud \citep{foote1997bigball}; alarms: declared-boundary violations $>0$ in CI, a rising trend of cross-module dependencies, and a rising share of features touching more than two modules (change dispersion, D7). \emph{Shared database tables across module boundaries} -- the erosion's data-layer twin; alarms: foreign keys crossing module schemas, and migrations that only pass when several modules deploy together.
|
|
|
|
\paragraph{Selection signals.} \textbf{Choose when:} the domain is known or explorable, the organisation is below roughly fifty developers, time-to-market matters \emph{and} long-term evolvability is required -- the MonolithFirst strategy \citep{fowler2015monolith}. \textbf{Avoid when:} modules have fundamentally different scaling or compliance profiles, or independent deployment is business-critical -- the signals that D2/D5/D11 are about to become High-weight dimensions.
|
|
|
|
\begin{ailinse}[The modular monolith as an AI host]
|
|
D12 $= +$: the pattern hosts the deterministic majority of an AI-bearing system -- data access, computation, validation, audit -- at monolith cost, and gives the AI subsystem what the layered pattern cannot: a \emph{hard, CI-verifiable module boundary}. The determinism boundary of the course project -- no domain module talks to the LLM gateway except through its declared port -- is exactly one more rule in Listing~\ref{lst:boundary}: statically checkable, enforced on every build \citep{drotbohm2023modulith}. What the single quantum does not provide is independent scaling or failure isolation for the AI workload itself; that is what the event-driven edge is for (Section~\ref{sec:pat-eda}).
|
|
\end{ailinse}
|
|
|
|
\begin{keypoint}
|
|
The modular monolith dominates the layered architecture on almost every dimension except day-one simplicity, at the price of continuous boundary governance. It is the default starting point of modern systems -- a synthesis this theory shares with \citet{richards2025fundamentals} and \citet{fowler2015monolith}: domain partitioning now, distribution only when a measured requirement demands it.
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: HX -- HEXAGONAL / PORTS AND ADAPTERS
|
|
% ============================================
|
|
\section{HX -- Hexagonal Architecture / Ports and Adapters}
|
|
\label{sec:pat-hx}
|
|
|
|
\emph{How do you make the domain logic testable in milliseconds when the database, the broker, and an LLM provider all sit at the edges?} By making ``at the edges'' a property the compiler can check -- which is this pattern's entire content.
|
|
|
|
\subsection{Definition and topology}
|
|
|
|
The hexagonal architecture, published by Alistair Cockburn in 2005 and given its canonical book form two decades later, answers a different question than the other six patterns \citep{cockburn2005hexagonal,cockburn2024hexagonal}. It does not distribute a system; it organises \emph{dependency direction}. The application core -- the domain logic -- knows nothing of the outside world. Every interaction crosses a technology-neutral \emph{port}: an interface defined by the application itself, in the application's vocabulary. Technology-specific \emph{adapters} plug into the ports: \emph{driving} (primary) adapters call the application -- a REST controller, a CLI, a test harness; \emph{driven} (secondary) adapters are called by it -- a database gateway, a message-broker client, an LLM provider. Cockburn names two purposes: testability without infrastructure, and protection against business-logic leakage into technology code.
|
|
|
|
\begin{definitionbox}[Hexagonal architecture / ports and adapters (HX)]
|
|
A pattern of dependency organisation in which the application core interacts with everything outside itself exclusively through technology-neutral interfaces (\emph{ports}) that the core itself defines; technology-specific \emph{adapters} call the core through driving ports or implement driven ports. All source-code dependencies point inwards, towards the core -- adapters depend on ports, never the reverse \citep{cockburn2005hexagonal,cockburn2024hexagonal}.
|
|
\end{definitionbox}
|
|
|
|
The demarcation from Part~\ref{part:foundations} bears repeating because it governs how the HX column of Table~\ref{tab:capability} must be read: \textbf{hexagonal architecture is not a distribution style}. It is orthogonal to the monolith/microservices axis and \emph{composes} with the other six patterns -- a hexagonal modular monolith, hexagonal microservices. \citet{richards2025fundamentals} accordingly assign it no star profile; its capability column is a \emph{delta}: the $\diamond$ cells inherit the host pattern's ratings, and the pattern's own cells describe what the ports-and-adapters cut adds to any host. Figure~\ref{fig:pat-hx} shows the topology: the core defines the ports; all source-code dependencies point inwards.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
sysbox/.style={rectangle, draw, rounded corners=4pt, align=center, font=\small\sffamily, line width=0.8pt},
|
|
drivead/.style={sysbox, fill=bankgreen!15, draw=bankgreen, minimum width=2.6cm, minimum height=0.9cm},
|
|
drivenad/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=2.6cm, minimum height=0.9cm},
|
|
aiad/.style={sysbox, fill=violet!15, draw=aiviolet, minimum width=2.6cm, minimum height=0.9cm},
|
|
port/.style={rectangle, draw=bankblue, fill=white, minimum size=0.32cm, inner sep=0pt, line width=0.8pt},
|
|
arr/.style={-{Stealth[length=2.5mm]}, thick, gray!60!black}
|
|
]
|
|
% hexagon (flat-top), radius 2.3
|
|
\draw[line width=1pt, draw=bankblue, fill=bankblue!15]
|
|
(30:2.3) -- (90:2.3) -- (150:2.3) -- (210:2.3) -- (270:2.3) -- (330:2.3) -- cycle;
|
|
\node[font=\small\sffamily\bfseries, text=bankblue, align=center] at (0,0)
|
|
{Domain core\\ \footnotesize (knows nothing of\\ \footnotesize the outside world)};
|
|
% ports on the boundary
|
|
\node[port] (p1) at (150:2.3) {};
|
|
\node[port] (p2) at (210:2.3) {};
|
|
\node[port] (p3) at (30:2.3) {};
|
|
\node[port] (p4) at (330:2.3) {};
|
|
\node[font=\scriptsize\sffamily, text=bankblue, above left=0.05cm of p1] {driving ports};
|
|
\node[font=\scriptsize\sffamily, text=bankblue, above right=0.05cm of p3] {driven ports};
|
|
% driving adapters (left)
|
|
\node[drivead] (rest) at (-5.4,1.4) {REST API\\adapter};
|
|
\node[drivead] (test) at (-5.4,-1.4) {Test harness\\(fake driver)};
|
|
\draw[arr] (rest) -- (p1);
|
|
\draw[arr] (test) -- (p2);
|
|
% driven adapters (right)
|
|
\node[drivenad] (db) at (5.4,1.8) {Database\\adapter};
|
|
\node[drivenad] (mq) at (5.4,0) {Message-broker\\adapter};
|
|
\node[aiad] (llm) at (5.4,-1.8) {LLM-provider\\adapter};
|
|
\draw[arr] (p3) -- (db);
|
|
\draw[arr] (p3) -- (mq);
|
|
\draw[arr] (p4) -- (llm);
|
|
\node[font=\footnotesize\sffamily, text=gray!60!black, align=center] at (0,-3.3)
|
|
{runtime calls flow outwards; source-code dependencies point inwards:\\adapters depend on ports, the core depends on nothing outside itself};
|
|
\end{tikzpicture}%
|
|
\caption{Topology of the hexagonal architecture: a domain core behind technology-neutral ports, with swappable driving adapters (left) and driven adapters (right). The LLM-provider adapter (violet) previews the pattern's role as the anti-corruption layer for AI components (Part~\ref{part:ai}).}
|
|
\label{fig:pat-hx}
|
|
\end{figure}
|
|
|
|
\subsection{Variants}
|
|
|
|
The Onion Architecture (Palermo, 2008) and the Clean Architecture (Martin, 2017) express the same core idea -- invert all dependencies towards the domain -- with different vocabulary and ring counts; for the purposes of this module the three are one family, and ``hexagonal'' stands for all of them. The load-bearing shared mechanism is the dependency inversion principle: the domain defines the interface, the technology implements it \citep{martin2003agile}.
|
|
|
|
\subsection{The problem it solves}
|
|
|
|
The pain that produced this pattern is testability. Cockburn's diagnosis, published in 2005, was that business logic in the typical enterprise application ends up \emph{welded} to its surroundings: rules leak into UI event handlers on one side and into data-access code and stored procedures on the other, until the only way to check whether a discount is computed correctly is to start the whole application, click through screens, and inspect database rows \citep{cockburn2005hexagonal}. Tests of that kind are slow, brittle, and so expensive that teams write few of them -- and the logic most worth testing is precisely the logic hardest to reach.
|
|
|
|
The same weld causes a second, slower problem: the system cannot follow its technology. A new UI generation, a database migration, an additional delivery channel -- each ought to be a peripheral change, but while business rules are smeared through UI and persistence code, every peripheral change is open-heart surgery. Cockburn's response inverts the dependency: let the application core define technology-neutral interfaces in its own vocabulary, and make every technology -- the test harness included -- an interchangeable plug. A test harness then drives the core exactly as the production UI does, and a fake replaces the database in milliseconds. The two purposes named in the definition above -- testability without infrastructure, protection against business-logic leakage -- are therefore not features bolted onto the pattern but the problem statement it was written against \citep{cockburn2024hexagonal}.
|
|
|
|
The pattern is the standard core-isolation discipline of banking backends -- long-lived booking logic behind ports, volatile channel and provider adapters outside -- and the house style of domain-driven design reference implementations \citep{cockburn2024hexagonal,evans2003ddd}.
|
|
|
|
\subsection{Capability profile}
|
|
|
|
With Cockburn's problem in mind, Table~\ref{tab:cap-hx} walks column HX of Table~\ref{tab:capability} (assembled at the end of this part) through all twelve dimensions; because HX is a delta pattern -- a discipline inside a host, not a distribution style -- the symbol $\diamond$ marks the dimensions on which it simply inherits its host pattern's rating, and only its own five cells carry reasons of their own.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{The hexagonal architecture across the twelve dimensions -- column HX of Table~\ref{tab:capability}; note numbers refer to that table's notes.}
|
|
\label{tab:cap-hx}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{3.2cm}cp{8.5cm}@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{Rating} & \textbf{Structural reason} \\
|
|
\midrule
|
|
D1 Read scalability & $\diamond$ & inherited from the host pattern -- the ports-and-adapters cut does not change this dimension \\
|
|
D2 Write scalability \& elasticity & $\diamond$ & inherited from the host pattern -- the ports-and-adapters cut does not change this dimension \\
|
|
D3 Latency \& predictability & $\diamond$ & inherited from the host pattern -- the ports-and-adapters cut does not change this dimension \\
|
|
D4 Consistency \& integrity & $\diamond$ & inherited from the host pattern -- the ports-and-adapters cut does not change this dimension \\
|
|
D5 Availability \& fault isolation & $\diamond$ & inherited from the host pattern -- the ports-and-adapters cut does not change this dimension \\
|
|
D6 Security \& auditability & $+$ & ports are natural audit and policy chokepoints \\
|
|
D7 Evolvability \& maintainability & $++$ & technology migrations become localised adapter tasks at exactly the seams where change is anticipated \\
|
|
D8 Simplicity \& time-to-market & $-$ & indirection that amortises only under change -- ceremony for pure CRUD pass-through \\
|
|
D9 Testability \& deployability & $++$ & hermetic domain tests behind ports: fakes replace infrastructure, feedback in milliseconds \\
|
|
D10 Operating cost efficiency & $\diamond$ & inherited from the host pattern -- the ports-and-adapters cut does not change this dimension \\
|
|
D11 Team scaling (Conway) & $\diamond$ & inherited from the host pattern -- the ports-and-adapters cut does not change this dimension \\
|
|
D12 AI integrability & $++$ & the port is an imposed anti-corruption boundary that an ML component cannot erode \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & \multicolumn{2}{l}{(host's)} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
The table gives the full profile; the pattern's own cells deserve the story behind the numbers. \textbf{D7 $= ++$ and D9 $= ++$} are the pattern's reason for existing, and both are directly measurable. The share of \emph{hermetic} tests -- tests requiring no database, no broker, no network -- rises sharply once domain logic sits behind ports, and test feedback time for that logic falls to milliseconds, because fakes replace infrastructure \citep{cockburn2024hexagonal}. Technology migrations -- a database swap, a message-broker swap, an LLM-provider swap -- become \emph{localised adapter tasks}: this is Booch's cost-of-change criterion (assumption A1, Part~\ref{part:foundations}) applied surgically at the exact seams where change is anticipated. \textbf{D12 $= ++$} is the strongest AI-integrability cell in the catalogue (AI lens below). The honest cost is \textbf{D8 $= -$}: for pure CRUD pass-through with no domain logic, ports and adapters are ceremony -- waste, in lean terms.
|
|
|
|
\subsection{Software engineering implications}
|
|
|
|
\textbf{Build and CI/CD.} Unchanged from the host pattern -- the hexagon is invisible to the pipeline topology, but adds one gate: the dependency rule itself is a fitness function (``the domain imports no framework, no vendor SDK''), enforceable with the same ArchUnit-class tooling as module boundaries \citep{archunit2024userguide,ford2022evolutionary}. \textbf{Test.} The test pyramid gains a wide, fast base of hermetic domain tests; each port acquires contract tests that every adapter -- including every fake -- must pass, so that fakes cannot drift from reality. \textbf{Deployment and operations.} Inherited from the host. \textbf{Maintenance.} The payoff channel: volatile peripheries (UIs, third-party providers, AI services) churn in adapters while the core stays still. \textbf{Team size (Conway).} Neutral -- inherited from the host -- but the pattern demands design skill: the ports must be cut in domain vocabulary, which is a modelling task, not a refactoring.
|
|
|
|
\subsection{Exemplars, anti-patterns, and selection signals}
|
|
|
|
\begin{examplebox}[Build it and study it -- hexagonal]
|
|
\textbf{Build.} The hexagon is a discipline, not a runtime, so its ``framework'' is a rule checker. ArchUnit (Java, Apache-2.0, active) unit-tests the dependency rules -- its \texttt{onionArchitecture()} API encodes the inward-pointing arrows of Figure~\ref{fig:pat-hx} directly \citep{archunit2024userguide}; in Python, \texttt{import-linter} (BSD-2, active) enforces the same contracts between packages \citep{importlinter2026}.
|
|
|
|
\textbf{Study.} \texttt{cosmicpython/code} ($\sim$2.7k stars) is the example application of \emph{Architecture Patterns with Python} (Percival \& Gregory), whose full text is freely readable online -- the ideal companion for this module's Python-first audience \citep{percival2020cosmicbook,cosmicpython2025code}. Look at \texttt{src/allocation/}: the split into \texttt{domain/}, \texttt{service\_layer/}, \texttt{adapters/}, and \texttt{entrypoints/}, and in particular the repository and unit-of-work \emph{ports} with their swappable adapters. Runs via docker-compose plus pytest: easy to moderate. Licence caveat, as promised in the first box: the repository is CC-BY-ND -- free to study, \emph{not} free to reuse in derived work.
|
|
\end{examplebox}
|
|
|
|
\begin{examplebox}[Storage migrations behind a seam -- Discord]
|
|
Discord's message store migrated twice at trillion-row scale -- MongoDB to Cassandra to ScyllaDB, with hot paths re-implemented as Rust data services -- cutting p99 read latencies from 40--125\,ms to roughly 15\,ms \citep{ingram2023discord}. The migrations were survivable \emph{because} data access sat behind stable service-internal interfaces: the storage technology churned, the callers did not. This is the D7/D9 delta of the hexagonal cut observed in production -- and a preview of why the same discipline turns an LLM-provider deprecation from a crisis into an adapter task.
|
|
\end{examplebox}
|
|
|
|
\paragraph{Anti-patterns and measurable alarms.} \emph{Hexagonal in name only}: the folder structure says ports and adapters, the dependencies say otherwise. Three measurable symptoms. First, \emph{ports leaking vendor types} -- a port signature mentioning a provider SDK class; alarm: a static rule ``no vendor or framework types in port signatures'' with target zero violations. Second, \emph{adapters containing business logic} -- decisions made where they cannot be hermetically tested; alarm: the share of hermetic tests stagnates after the supposed hexagonal refactoring (if the cut were real, it would rise). Third, the \emph{anaemic domain} behind perfect ports -- all logic in application services, the core reduced to data bags \citep{fowler2002poeaa}; alarm: core modules with high afferent coupling (many modules depend on them) but near-zero cyclomatic complexity (almost no decision logic inside).
|
|
|
|
\paragraph{Selection signals.} \textbf{Choose when:} long-lived domain logic meets volatile peripheries -- many UIs, exchangeable providers, AI services -- and the testability requirement is high. \textbf{Avoid when:} the component is pure CRUD pass-through with no domain logic; then D8 $= -$ buys nothing, and the indirection never amortises.
|
|
|
|
\begin{ailinse}[The anti-corruption layer for AI components]
|
|
D12 $= ++$ -- the strongest cell in the D12 row, and the reasoning previews Part~\ref{part:ai}. ML components resist modularisation: ``changing anything changes everything'' (CACE), and they erode abstraction boundaries unless a boundary is \emph{imposed} on them \citep{sculley2015hidden,amershi2019software}. The port is that imposed boundary -- an anti-corruption layer in the domain-driven design sense \citep{evans2003ddd}. An LLM behind a port is swappable when the provider deprecates the model \citep{openai2026deprecations}, mockable in every test, and replaceable by a deterministic fake so that the deterministic 95\,\% of the system can be tested deterministically. Assumption A1's cost-of-change criterion bites hardest at model replacement -- which is why, of all cells in the table, this one gains the most from the AI era (Part~\ref{part:ai}).
|
|
\end{ailinse}
|
|
|
|
\begin{keypoint}
|
|
The hexagonal architecture is not a competitor of the other six patterns but a discipline inside them. Its profile is a delta: $++$ on evolvability, testability, and AI integrability, bought with $-$ on day-one simplicity. It purchases options on change -- and, like all options, it is worth exactly nothing where change never comes.
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: MS -- MICROSERVICES
|
|
% ============================================
|
|
\section{MS -- Microservices}
|
|
\label{sec:pat-ms}
|
|
|
|
\emph{Thirty teams share one release train and step on each other with every deployment: what buys their independence -- and what does it cost?} This section derives both halves of the answer, and the cost half is the longer one.
|
|
|
|
\subsection{Definition and topology}
|
|
|
|
Microservices structure a system as a suite of small, independently deployable services, each built around a business capability, each owning its data (\emph{database per service}), communicating over lightweight protocols. The canonical definition is \citet{lewis2014microservices}; the engineering standard works are \citet{newman2021building} -- whose one-sentence definition, ``independently deployable services modelled around a business domain'', names the two load-bearing properties -- and the pattern language of \citet{richardson2018microservices}. In the taxonomy of \citet{richards2025fundamentals}: domain partitioning, \emph{many} architecture quanta, the highest aggregate star score of all styles -- and one star on overall cost and simplicity, two stars on performance.
|
|
|
|
\begin{definitionbox}[Microservices (MS)]
|
|
A macro-structure of independently deployable services, each modelled around one business capability, each owning its persistent data exclusively, communicating via lightweight synchronous or asynchronous protocols. Every service is its own architecture quantum: its own unit of deployment, scaling, and failure \citep{lewis2014microservices,newman2021building}.
|
|
\end{definitionbox}
|
|
|
|
Figure~\ref{fig:pat-ms} shows the topology: many quanta, one database per service, lightweight protocols between them.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
sysbox/.style={rectangle, draw, rounded corners=4pt, align=center, font=\small\sffamily, line width=0.8pt},
|
|
gateway/.style={sysbox, fill=bankblue!20, draw=bankblue, minimum width=4.6cm, minimum height=0.95cm, font=\small\sffamily\bfseries},
|
|
svc/.style={sysbox, fill=bankgreen!15, draw=bankgreen, minimum width=2.9cm, minimum height=1.0cm},
|
|
data/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=2.9cm, minimum height=0.75cm, font=\footnotesize\sffamily},
|
|
arr/.style={-{Stealth[length=2.5mm]}, thick, gray!60!black},
|
|
saga/.style={-{Stealth[length=2.5mm]}, thick, bankred, dashed}
|
|
]
|
|
\node[gateway] (gw) at (0,2.6) {API gateway};
|
|
\node[svc] (s1) at (-4.2,0.6) {Catalogue\\service};
|
|
\node[svc] (s2) at (0,0.6) {Order\\service};
|
|
\node[svc] (s3) at (4.2,0.6) {Payment\\service};
|
|
\node[data, below=0.45cm of s1] (db1) {own database};
|
|
\node[data, below=0.45cm of s2] (db2) {own database};
|
|
\node[data, below=0.45cm of s3] (db3) {own database};
|
|
\draw[arr] (gw) -- (s1);
|
|
\draw[arr] (gw) -- (s2);
|
|
\draw[arr] (gw) -- (s3);
|
|
\draw[arr] (s1) -- (db1);
|
|
\draw[arr] (s2) -- (db2);
|
|
\draw[arr] (s3) -- (db3);
|
|
\draw[saga] (s2) -- node[above, font=\footnotesize\sffamily, text=bankred] {saga, no ACID} (s3);
|
|
% quantum boundaries
|
|
\draw[dashed, gray!60!black, rounded corners=5pt] (-5.85,1.25) rectangle (-2.55,-1.75);
|
|
\draw[dashed, gray!60!black, rounded corners=5pt] (-1.65,1.25) rectangle (1.65,-1.75);
|
|
\draw[dashed, gray!60!black, rounded corners=5pt] (2.55,1.25) rectangle (5.85,-1.75);
|
|
\node[font=\footnotesize\sffamily, text=gray!60!black, align=center] at (0,-2.3)
|
|
{one quantum per service: own pipeline, own release, own blast radius, own database};
|
|
\end{tikzpicture}%
|
|
\caption{Topology of a microservices system: independently deployable services behind an API gateway, database per service. Cross-service consistency is bought with sagas instead of ACID transactions (dashed red) -- the structural reason for D4 $= --$.}
|
|
\label{fig:pat-ms}
|
|
\end{figure}
|
|
|
|
\subsection{Variants}
|
|
|
|
(a)~\emph{Orchestration versus choreography} for cross-service workflows: a central saga orchestrator versus event-driven choreography \citep{richardson2018microservices}. (b)~\emph{Service mesh and sidecars} (Istio, Linkerd) to factor operational cross-cutting concerns out of service code. (c)~\emph{Self-contained systems}: coarser-grained services, each with its own UI. (d)~\emph{Service-based architecture} as the pragmatic precursor: a handful of coarse domain services, often over a shared database -- rated by \citet{richards2025fundamentals} at around four stars on deployability and fault tolerance at markedly lower cost, and characterised by them as the most pragmatic of the distributed styles. Teams that need some independent deployability but cannot pay the full premium should evaluate this variant before the full pattern.
|
|
|
|
\subsection{The problem it solves}
|
|
|
|
The problem microservices solve is organisational before it is technical. By the mid-2000s, Amazon -- and, a few years later, Netflix -- had grown to hundreds of teams contributing to shared deployables, and a shared deployable means a shared release train: every team's change waits on every other team's change, integration and regression scope grow with the organisation rather than with the change, and one team's defect can roll back everyone's release. Past a certain organisational size, the queue for the release train -- not compute, not traffic -- becomes the binding constraint on how fast the company ships.
|
|
|
|
The pattern's answer is to make the team's unit of ownership the system's unit of deployment: cut the system along business capabilities, give each service exclusive ownership of its data and its own pipeline, and let each team release independently -- ``you build it, you run it''. That is why the canonical definition leads with independent deployability and domain modelling rather than with service size \citep{lewis2014microservices,newman2021building}, and why the strongest empirical support for the pattern is organisational: in high-performing organisations, deployment frequency scales linearly with team count instead of collapsing under coordination \citep{forsgren2018accelerate}. What it takes to run the model at a licensed retail bank -- Monzo, on roughly 2{,}800 services -- is documented in the Monzo and Segment box below, together with the conditions that make it possible.
|
|
|
|
Netflix and Amazon are the founding exemplars; the open-source reference implementations of the ``Build it and study it'' box below (Microsoft's eShop, Google's Online Boutique) reproduce the topology at teaching scale \citep{eshop2026,google2026onlineboutique}. Two documented cases bracket the pattern's viability conditions (the Monzo and Segment box below), and a third documents its upper bound: Uber, at roughly 2{,}200 critical services, found that service count itself had become the problem and re-introduced a second structuring level -- roughly 70 domains with gateways as sole entry points (Domain-Oriented Microservice Architecture) -- a documented convergence back towards macro-modules, with 25--50\,\% faster feature onboarding reported \citep{gluck2020doma}.
|
|
|
|
\subsection{Capability profile}
|
|
|
|
With the release-train problem in mind, Table~\ref{tab:cap-ms} walks column MS of Table~\ref{tab:capability} (assembled at the end of this part) through all twelve dimensions, naming for each rating the structural fact that earns it -- the catalogue's steepest trade, with the top of the table bought at the bottom's expense.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{Microservices across the twelve dimensions -- column MS of Table~\ref{tab:capability}; note numbers refer to that table's notes.}
|
|
\label{tab:cap-ms}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{3.2cm}cp{8.5cm}@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{Rating} & \textbf{Structural reason} \\
|
|
\midrule
|
|
D1 Read scalability & $++$ & independently replicated, cacheable read paths per service \\
|
|
D2 Write scalability \& elasticity & $++$ & per-service horizontal scaling; fine-grained provisioning rather than scale-to-zero (note~5) \\
|
|
D3 Latency \& predictability & $-$ & network hops, serialisation, and tail-latency amplification along synchronous call chains \\
|
|
D4 Consistency \& integrity & $--$ & database-per-service: no ACID transaction spans a boundary, and compensation is not rollback \\
|
|
D5 Availability \& fault isolation & $++$ & bulkheads: one failing service is one failing service, not a failing product \\
|
|
D6 Security \& auditability & $\circ$ & expanded network attack surface and scattered audit trails against per-service isolation and compliance-scope carving (note~12) \\
|
|
D7 Evolvability \& maintainability & $++$ & domain partitioning: a typical change lands inside one independently deployable service \\
|
|
D8 Simplicity \& time-to-market & $--$ & platform engineering before the first feature -- the microservice premium \\
|
|
D9 Testability \& deployability & $+$ & per-service tests and releases are excellent; system-level verification shifts towards production (note~16) \\
|
|
D10 Operating cost efficiency & $--$ & the premium is paid mostly as platform staffing \\
|
|
D11 Team scaling (Conway) & $++$ & independent deployability: teams release without queueing for a shared release train \\
|
|
D12 AI integrability & $\circ$ & per-service isolation helps, but synchronous chains multiply seconds-scale LLM latency and failure probability \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & \multicolumn{2}{l}{interactive} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
The table gives the full profile; four cells deserve the story behind the number:
|
|
|
|
\begin{itemize}
|
|
\item \textbf{D1/D2 $= ++$} (note~5): per-service horizontal scaling is the pattern's mechanical strength; the source ratings split five-star scalability from three-to-four-star elasticity -- fine-grained provisioning, not scale-to-zero -- and the $++$ reads the pair jointly while naming the weaker half.
|
|
\item \textbf{D11 $= ++$ is the real payoff.} Independent deployability lets deployments per developer per day scale \emph{linearly} with team count in high-performing organisations -- the DORA scaling result \citep{forsgren2018accelerate}. Teams stop queueing for a shared release train. This, not raw traffic, is the requirement that legitimately forces the pattern.
|
|
\item \textbf{D4 $= --$ is structural, not accidental} -- this cell served as the method's negative example in Part~\ref{part:foundations} (Section~\ref{subsec:derivation}), so only the deepening is new here. Database-per-service means no ACID transaction spans a boundary; a cross-service invariant (order $\to$ stock $\to$ ledger) becomes a saga, and the sting sits in what the micro-derivation compressed into one clause: \emph{compensation is not rollback}. A rollback erases an intermediate state as if it never existed; a compensating action cannot -- other services have already seen and acted on that state -- so every ``undo'' is a new business operation (cancel the order, restock, refund) with its own logic, its own tests, and its own failure modes \citep{richardson2018microservices,newman2021building}. No implementation skill removes this cell; it can only be relocated (by cutting boundaries so that invariants live inside one service).
|
|
\item \textbf{D8/D10 $= --$}: the ``microservice premium'' \citep{fowler2015premium} is paid mostly as platform staffing; self-managed Kubernetes total cost of ownership runs roughly three times managed offerings, dominated by personnel \citep{portainer2024k8scost,gcore2024k8stco}. And \textbf{D9 $= +$} (note~16, a flagged deviation) names where the remaining verification bill lands: per-service testability and deployability rate four to five stars, but \emph{system-level} integration verification shifts towards production -- consumer-driven contracts (each consumer's expectations of an interface recorded as tests the provider must keep passing), canary releases (routing a small share of live traffic to the new version before the rest), progressive delivery (widening such rollouts stepwise under live metrics) \citep{newman2021building,humble2010continuous}. The $+$ averages a genuine split.
|
|
\end{itemize}
|
|
|
|
\subsection{Software engineering implications}
|
|
|
|
The SDLC consequences are the steepest in the catalogue, and they are \emph{constitutive} -- omitting them does not produce a leaner variant of the pattern but a broken one. \textbf{Build and CI/CD.} One pipeline, one versioning scheme, one contract-test suite \emph{per service}; consumer-driven contracts guard every interface. \textbf{Test.} Hermetic per-service tests are excellent; whole-system verification moves into production-adjacent techniques -- canaries, feature flags (switches that activate code paths at runtime, decoupling release from deployment), progressive delivery \citep{humble2010continuous}. \textbf{Deployment.} Independent, on demand; elite performers deploy per service without cross-team coordination \citep{forsgren2018accelerate}. \textbf{Operations.} Distributed tracing, centralised logging, per-service SLOs, circuit breakers (callers stop calling a dependency that keeps failing), and bulkheads (resources partitioned so one failing component cannot drain the rest) are not optional hardening but part of the pattern's definition-in-practice \citep{nygard2018release}. \textbf{Maintenance.} Local change is cheap; cross-cutting change (a shared-library upgrade, a protocol migration) multiplies by service count -- the exact mechanism of the Segment failure below. \textbf{Team size (Conway).} Stream-aligned teams with full ownership (``you build it, you run it'') are a \emph{precondition}, not an outcome; a platform team is usually required \citep{skelton2019team}.
|
|
|
|
\begin{hinweisbox}
|
|
The \emph{distributed monolith} -- services that can only be built, tested, and released together -- combines the costs of both worlds: distribution's latency, operational burden, and eventual consistency without independent deployability, the one payoff that justifies them \citep{newman2021building}. It is the most common real-world outcome of premature microservice adoption, and it is measurable: track the \emph{lockstep release ratio} (share of releases involving more than one service), cross-service change dispersion (services touched per feature), and synchronised version bumps across repositories. If those numbers look like a monolith's, the organisation is paying the premium without the product.
|
|
\end{hinweisbox}
|
|
|
|
\subsection{Exemplars, anti-patterns, and selection signals}
|
|
|
|
\begin{examplebox}[Build it and study it -- microservices]
|
|
\textbf{Build.} FastAPI (Python, MIT, very active) is the per-service building block: one small HTTP service with a generated OpenAPI contract per business capability \citep{fastapi2026}; Kubernetes is the usual substrate underneath. The canonical Java stack is Spring Cloud (Apache-2.0, active): service discovery, configuration, gateway \citep{springcloud2026}.
|
|
|
|
\textbf{Study.} Google's Online Boutique (\texttt{GoogleCloudPlatform/microservices-demo}, $\sim$20.6k stars, Apache-2.0, active): twelve polyglot services under \texttt{src/} (Go, Python, C\#, Java, Node), their gRPC contracts centralised in \texttt{protos/}, one deployment per service under \texttt{kubernetes-manifests/} \citep{google2026onlineboutique}. It runs on a local kind or minikube cluster via \texttt{skaffold run}: moderate to hard -- and instructively so, because the platform effort you will feel \emph{is} D8~$=$~$--$. Alternative: \texttt{dotnet/eShop} (MIT, active) \citep{eshop2026}. The Sock Shop demo that older literature cites was archived in December 2023 -- the maintenance-status check of the first box, in action.
|
|
\end{examplebox}
|
|
|
|
\begin{examplebox}[Monzo and Segment -- the homogeneity condition]
|
|
Monzo operates a licensed retail bank on roughly 2{,}800 microservices -- and it works because the company enforces extreme technological homogeneity: one language (Go), one monorepo, shared infrastructure libraries, and central migration automation that upgrades hundreds of services mechanically \citep{sewell2024monzo}. Segment cut its customer-data pipeline into more than 140 services -- one per analytics destination, i.e.\ along \emph{instances of configuration} rather than domain seams -- and publicly reversed course in 2018: a single shared-library change required over a hundred deployments, and test and operations load crushed a small team \citep{noonan2018goodbye,infoq2020segment}. Read jointly: the pattern's viability at scale depends on \emph{where the boundaries run} (Maxim~4) and on paying the platform premium centrally -- not on service count.
|
|
\end{examplebox}
|
|
|
|
\paragraph{Anti-patterns and measurable alarms.} The \emph{distributed monolith} (see box above). \emph{Entity services} -- services cut around nouns instead of capabilities, so every workflow spans many of them; alarm: fan-out per business transaction. \emph{Grains of sand} -- services too small to own an invariant; alarm: service count growing faster than team count, and rising saga complexity for elementary operations.
|
|
|
|
\paragraph{Selection signals.} \textbf{Choose when:} many teams must deliver in parallel without coordination (D11 measurably binding); subdomains have independent scaling, failure, or compliance profiles; the organisation is DevOps-mature enough to staff a platform. \textbf{Avoid when:} the premium exceeds the system's complexity \citep{fowler2015premium}; the domain is not yet understood well enough to cut stable boundaries -- refactoring \emph{between} services is far costlier than \emph{within} a monolith \citep{fowler2015monolith}; or the motivation is fashion rather than a measured requirement.
|
|
|
|
\begin{ailinse}[Microservices and AI: the synchronous-chain trap]
|
|
D12 $= \circ$, and the ambivalence is precise. Per-service isolation and blast-radius control are welcome for a fallible component, and an AI capability can be its own service with its own SLO. But an LLM call placed inside a \emph{synchronous} service chain multiplies seconds-scale latency and per-hop failure probability along the chain -- without constitutive stability patterns (timeouts, circuit breakers per provider, bulkheads, fallbacks) this is a cascade design \citep{nygard2018release}. And the AI-era cost dimension -- token cost per request -- is a \emph{gateway} concern orthogonal to distribution: splitting a system into services does nothing to measure or cap it. Asynchronous integration and a single LLM gateway do (Part~\ref{part:ai}).
|
|
\end{ailinse}
|
|
|
|
\begin{keypoint}
|
|
Microservices are the only pattern rated $++$ on team scaling (D11) -- and everything else in the column is the bill: $--$ on consistency, simplicity, and cost are structural properties of many quanta, not implementation accidents. Adopt the pattern for measured organisational scale, never for traffic alone -- traffic has cheaper answers (notes~1 and~4).
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: EDA -- EVENT-DRIVEN ARCHITECTURE
|
|
% ============================================
|
|
\section{EDA -- Event-Driven Architecture}
|
|
\label{sec:pat-eda}
|
|
|
|
\emph{Tomorrow a new consumer needs every order event: can you attach it without touching the producer?} If the answer must be yes, an intermediary has to stand between the two -- and everything in this section follows from what that intermediary gives and takes.
|
|
|
|
\subsection{Definition and topology}
|
|
|
|
Event-driven architecture decouples the parts of a system in \emph{time}: producers emit immutable event records describing things that have happened; consumers react asynchronously, without the producer knowing who they are or whether they are currently running. The messaging vocabulary is canonically catalogued in \citet{hohpe2003enterprise}; the broker pattern goes back to \citet{buschmann1996posa}. \citet{richards2025fundamentals} distinguish two topologies, and the distinction is load-bearing for the profile: the \emph{broker topology} -- decentralised chains of events across topics, maximal scaling and responsiveness, no central workflow control and hard error recovery -- and the \emph{mediator topology} -- a central orchestrator dispatching commands to processors via queues, gaining workflow visibility and recoverability at the price of coupling and a potential bottleneck.
|
|
|
|
\begin{definitionbox}[Event-driven architecture (EDA)]
|
|
A macro-structure of asynchronously decoupled event producers and consumers connected through a messaging substrate (broker or mediator). Processing is a reaction to events -- immutable records of facts -- rather than a response to synchronous calls; producers and consumers share knowledge of event schemas, never of each other \citep{richards2025fundamentals,hohpe2003enterprise}.
|
|
\end{definitionbox}
|
|
|
|
Figure~\ref{fig:pat-eda} contrasts the two topologies -- broker and mediator -- side by side.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
sysbox/.style={rectangle, draw, rounded corners=4pt, align=center, font=\footnotesize\sffamily, line width=0.8pt},
|
|
proc/.style={sysbox, fill=bankgreen!15, draw=bankgreen, minimum width=2.0cm, minimum height=0.8cm},
|
|
med/.style={sysbox, fill=bankblue!20, draw=bankblue, minimum width=2.2cm, minimum height=0.9cm, font=\footnotesize\sffamily\bfseries},
|
|
broker/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=5.6cm, minimum height=0.7cm},
|
|
queue/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=1.7cm, minimum height=0.55cm, font=\scriptsize\sffamily},
|
|
arr/.style={-{Stealth[length=2.5mm]}, thick, gray!60!black}
|
|
]
|
|
% --- (a) broker topology, left half ---
|
|
\node[proc] (prod) at (-4.2,2.2) {Producer};
|
|
\node[broker] (bk) at (-4.2,0.7) {event broker (topics)};
|
|
\node[proc] (c1) at (-5.9,-0.9) {Processor A};
|
|
\node[proc] (c2) at (-2.5,-0.9) {Processor B};
|
|
\node[proc] (c3) at (-4.2,-2.4) {Processor C};
|
|
\draw[arr] (prod) -- node[right, font=\scriptsize\sffamily] {event} (bk);
|
|
\draw[arr] (bk.south west)+(0.6,0) -- (c1.north);
|
|
\draw[arr] (bk.south east)+(-0.6,0) -- (c2.north);
|
|
\draw[arr] (c1.east) -- node[above, font=\scriptsize\sffamily] {derived event} (c3.west |- c1.east) -- (c3.north west);
|
|
\draw[arr] (c2.south) -- (c3.north east);
|
|
\node[font=\footnotesize\sffamily\bfseries, text=gray!60!black] at (-4.2,-3.3) {(a) broker topology};
|
|
% --- (b) mediator topology, right half ---
|
|
\node[proc] (src) at (4.2,2.2) {Initiating event};
|
|
\node[med] (md) at (4.2,0.7) {Mediator};
|
|
\node[queue] (q1) at (2.6,-0.7) {queue};
|
|
\node[queue] (q2) at (5.8,-0.7) {queue};
|
|
\node[proc] (w1) at (2.6,-2.0) {Processor A};
|
|
\node[proc] (w2) at (5.8,-2.0) {Processor B};
|
|
\draw[arr] (src) -- (md);
|
|
\draw[arr] (md) -- (q1);
|
|
\draw[arr] (md) -- (q2);
|
|
\draw[arr] (q1) -- (w1);
|
|
\draw[arr] (q2) -- (w2);
|
|
\draw[arr, dashed] (w1.east) to[bend right=25] node[right, font=\scriptsize\sffamily] {status} (md.south);
|
|
\node[font=\footnotesize\sffamily\bfseries, text=gray!60!black] at (4.2,-3.3) {(b) mediator topology};
|
|
\end{tikzpicture}%
|
|
\caption{The two event-driven topologies of \citet{richards2025fundamentals}: (a) the broker topology -- decentralised event chains, maximal scaling, no central workflow control; (b) the mediator topology -- orchestrated command dispatch with workflow visibility and recoverability, at the price of a potential bottleneck.}
|
|
\label{fig:pat-eda}
|
|
\end{figure}
|
|
|
|
\subsection{Variants}
|
|
|
|
Fowler's taxonomy separates four things commonly conflated under ``event-driven'': \emph{event notification} (a thin fact, consumers fetch what they need), \emph{event-carried state transfer} (the event carries the data, consumers keep local copies), \emph{event sourcing} (the event log \emph{is} the system of record; state is a projection), and \emph{CQRS} (separate write and read models). Substrate-wise, log-centric streaming platforms (Kafka -- durable, partitioned, replayable) differ materially from classical message brokers (RabbitMQ -- transient routing), and stream processors (Flink, Kafka Streams) build the bridge to the pipeline pattern of Section~\ref{sec:pat-pf}. Event sourcing has a spectacular documented exemplar: LMAX's deterministic, single-threaded, event-sourced trading core processed six million orders per second on one JVM thread -- consistency \emph{and} speed, bought with a radically unconventional topology \citep{fowler2011lmax}.
|
|
|
|
\subsection{The problem it solves}
|
|
|
|
The problem that produced the pattern's modern substrate was integration at scale. Around 2010, LinkedIn's activity data -- page views, profile edits, connection events -- was wanted by an ever-growing set of consumers: search indexes, recommendation engines, metrics warehouses, security monitoring. Wiring each producer to each consumer point-to-point yields a web of bespoke integrations that grows roughly quadratically with the number of systems and breaks with every schema change. LinkedIn's engineering answer was to put one durable, partitioned, replayable log in the middle and let every consumer read from it at its own pace -- the system that became Apache Kafka \citep{kreps2011kafka}.
|
|
|
|
The general form of the problem recurs wherever the set of consumers is open-ended: the producer of a fact -- an order placed, a sensor fired, a light switched on -- cannot know today who will need that fact tomorrow. Home Assistant faces the domestic version of the same explosion: thousands of device integrations, none of which may depend on any other, coordinated by firing state-change events through one bus \citep{homeassistant2026core}. Decoupling in topology (producers never know their audience) and in time (a consumer may be down, slow, or not yet written) is exactly what an intermediary provides -- and what no synchronous call graph can.
|
|
|
|
How far the answer carries in production is documented:
|
|
|
|
\begin{examplebox}[Kafka at LinkedIn and the Uber real-time stack]
|
|
Kafka originated at LinkedIn as a durable, partitioned, replayable commit log for activity data \citep{kreps2011kafka}; by 2019 LinkedIn's deployment carried more than seven \emph{trillion} messages per day \citep{lee2019kafka} -- the existence proof for D1/D2 $= ++$ at the substrate level. Uber's real-time platform composes Kafka (transport), Flink (stream processing), and Pinot (serving) into an end-to-end event-driven stack handling trillions of messages and petabytes daily \citep{fu2021realtime} -- and previews the canonical EDA-plus-pipeline hybrid that Part~\ref{part:fit} recommends for the IoT/streaming class (C8).
|
|
\end{examplebox}
|
|
|
|
\subsection{Capability profile}
|
|
|
|
With LinkedIn's integration problem in mind, Table~\ref{tab:cap-eda} walks column EDA of Table~\ref{tab:capability} (assembled at the end of this part) through all twelve dimensions, naming for each rating the structural fact that earns it.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{Event-driven architecture across the twelve dimensions -- column EDA of Table~\ref{tab:capability}; note numbers refer to that table's notes.}
|
|
\label{tab:cap-eda}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{3.2cm}cp{8.5cm}@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{Rating} & \textbf{Structural reason} \\
|
|
\midrule
|
|
D1 Read scalability & $++$ & consumers replicate and cache independently behind the broker \\
|
|
D2 Write scalability \& elasticity & $++$ & partitioned ingest with elastic, decoupled consumers \\
|
|
D3 Latency \& predictability & $+$ & asynchronous throughput is excellent; request/response round trips through event flows are not the pattern's mode (note~8) \\
|
|
D4 Consistency \& integrity & $--$ & eventual consistency moves correctness out of the database and into the design \\
|
|
D5 Availability \& fault isolation & $++$ & a slow or dead consumer does not stall producers \\
|
|
D6 Security \& auditability & $\circ$ & a durable event log supports audit, but causal trails across asynchronous hops need disciplined correlation identifiers (note~13) \\
|
|
D7 Evolvability \& maintainability & $++$ & the intermediary made structural: new consumers attach to existing streams without touching producers \\
|
|
D8 Simplicity \& time-to-market & $--$ & eventual consistency and broker operations must be designed and staffed before the first feature ships \\
|
|
D9 Testability \& deployability & $-$ & non-deterministic event flows resist exhaustive end-to-end testing \\
|
|
D10 Operating cost efficiency & $\circ$ & few quanta and no per-service premium, but the broker substrate is a standing operational cost \\
|
|
D11 Team scaling (Conway) & $+$ & teams per processor or domain topic decouple well; the broker itself needs an owner \\
|
|
D12 AI integrability & $++$ & queues absorb LLM latency, rate limits, and outages; the durable log doubles as the audit journal \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & \multicolumn{2}{l}{stream / async} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
The table gives the full profile -- the $++$ cells transcribe the source's five stars on performance, scalability, fault tolerance, and evolvability -- and three cells deserve the story behind the number. \textbf{D7 $= ++$} you have already derived: it was the worked method example of Part~\ref{part:foundations} (Section~\ref{subsec:derivation}) -- the ``use an intermediary'' tactic made structural \citep{bass2021software}. What that micro-derivation compressed can now be unpacked: the decoupling the broker buys is threefold -- in \emph{topology} (producers know the event schema, never the audience), in \emph{time} (a consumer may be down, slow, or not yet written when the event is published), and in \emph{organisation} (the team attaching a fraud detector needs no meeting with the team owning the order flow). All three reappear below as concrete SE-lifecycle effects. \textbf{D3 $= +$} (note~8) refines the five-star performance rating: throughput and responsiveness are excellent, but synchronous request/response round trips \emph{through} asynchronous flows are not the pattern's mode. \textbf{D4 $= --$} is the same coin as D7, seen from the other side: eventual consistency moves correctness out of the database and into the design -- read-your-writes anomalies, ordering across partitions, duplicate delivery are all \emph{design obligations} \citep{vogels2009eventually,kleppmann2017designing} -- and its testing bill is what keeps D8 at $--$ and D9 at $-$.
|
|
|
|
\subsection{Software engineering implications}
|
|
|
|
\textbf{Build and CI/CD.} Deploying an individual processor is easy; the hard artefact is the \emph{event schema}: schema evolution becomes a governance discipline, with a schema registry and compatibility rules (backward, forward, full) as pipeline gates, and AsyncAPI-style contracts taking the role that OpenAPI plays for synchronous interfaces. \textbf{Test.} Schema and contract tests per event type, consumer replays against recorded streams, idempotency tests -- processing the same event twice must have exactly the effect of processing it once -- under deliberate duplicate delivery; end-to-end tests are possible only sample-wise. \textbf{Operations.} Consumer lag (how far behind the newest event a consumer is running), dead-letter queues (parking spaces for messages that repeatedly fail processing), and duplicate rates are the RED metrics of this world; debugging requires correlation-ID tracing across hops -- a core competency, not an advanced topic. \textbf{Maintenance.} Additive evolution is the pattern's glory (attach a consumer); \emph{changing} an established event schema is its most expensive operation. \textbf{Team size (Conway).} D11 $= +$: teams per processor or per domain topic decouple well; the broker itself needs an owner.
|
|
|
|
\subsection{Exemplars, anti-patterns, and selection signals}
|
|
|
|
\begin{examplebox}[Build it and study it -- event-driven]
|
|
\textbf{Build.} The substrate is the broker: Apache Kafka (Apache-2.0) for the durable, partitioned, replayable log \citep{kreps2011kafka}, RabbitMQ (MPL-2.0) for classic AMQP routing \citep{rabbitmq2026}. The Python-first layer on top is FastStream (Apache-2.0, active): FastAPI-style producers and consumers for Kafka, RabbitMQ, or NATS, with the broker behind a decorator \citep{faststream2026}.
|
|
|
|
\textbf{Study.} Home Assistant (\texttt{home-assistant/core}, $\sim$89k stars, Apache-2.0, very active) is genuinely event-driven Python: \texttt{homeassistant/core.py} defines \texttt{class Event} and \texttt{class EventBus} with its \texttt{async\_fire} method, and \emph{every} state change, automation, and integration in the platform flows through that one in-process bus \citep{homeassistant2026core}. \texttt{pip install homeassistant \&\& hass} runs it easily; the codebase is huge, so read exactly that one file -- the whole pattern is in it.
|
|
\end{examplebox}
|
|
|
|
\paragraph{Anti-patterns and measurable alarms.} \emph{Event soup}: a distributed spaghetti of events with no workflow visibility -- nobody can say what happens after an order is placed. Alarms: the share of events without correlation identifiers, and the time needed to reconstruct the causal trail of one business transaction (an audit drill, the D6 response measure of Table~\ref{tab:dimensions}). \emph{Events as disguised RPCs}: a ``producer'' that blocks waiting for a specific reply event has re-invented synchronous coupling with extra steps; alarm: request/reply latency masquerading as event flow. \emph{Missing idempotency}: at-least-once delivery plus non-idempotent consumers equals silent corruption; alarm: duplicate rate under replay tests. \emph{Event sourcing as default}: a powerful, expensive tool applied where a CRUD table would do; the burden of proof lies with the auditability requirement.
|
|
|
|
\paragraph{Selection signals.} \textbf{Choose when:} high throughput and load spikes dominate, the domain's semantics are natively event-shaped (telemetry streams, order flows, integration hubs), or extensibility-by-new-consumers is a High-weight requirement. \textbf{Avoid when:} strong consistency and synchronous response guarantees govern the core path, or the team does not (yet) command asynchronous debugging -- D8 $= --$ is a statement about people as much as software.
|
|
|
|
\begin{thinkbox}
|
|
D4 $= --$, and yet Part~\ref{part:fit} will rate EDA the \emph{primary} pattern for three application classes. Which classes can afford to make correctness eventual -- and what, precisely, is the response measure that would tell you your class cannot? Formulate it as a six-part quality attribute scenario before looking at Part~\ref{part:applications}.
|
|
\end{thinkbox}
|
|
|
|
\begin{ailinse}[Queues absorb what LLMs are worst at]
|
|
D12 $= ++$. The three operational pathologies of LLM components -- seconds-scale latency, rate limits, provider outages -- are exactly what a queue between caller and callee absorbs; asynchronous integration is the default for non-interactive AI work (Part~\ref{part:ai}). Queues also create natural \emph{batching points} for cost control: batch APIs price roughly 50\,\% below synchronous calls \citep{openai2024batch}. And the durable event log doubles as the audit journal that logging obligations for AI systems demand -- every prompt, every response, every agent step, replayable \citep{kleppmann2017designing}. The one thing EDA does not give an AI component is a synchronous answer; where the user is waiting, the latency budget must be engineered explicitly.
|
|
\end{ailinse}
|
|
|
|
\begin{keypoint}
|
|
EDA packages the tactics ``use an intermediary'' and ``introduce concurrency'' by construction -- hence $++$ on scaling, availability, and evolvability -- and impedes the tactic ``transactions'' by the same construction -- hence $--$ on consistency and the testing bill \citep{bass2021software}. The positive and negative cells are the same coin, seen from two sides; no broker product removes the trade.
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: PF -- PIPES-AND-FILTERS / BATCH PIPELINE
|
|
% ============================================
|
|
\section{PF -- Pipes-and-Filters / Batch Pipeline}
|
|
\label{sec:pat-pf}
|
|
|
|
\emph{The nightly risk run must process millions of rows, reproducibly, by 06:00: what structure is born for exactly that?} This one -- the oldest pattern in the catalogue, and the one most precisely matched to its workload shape.
|
|
|
|
\subsection{Definition and topology}
|
|
|
|
Pipes-and-filters structures computation as a unidirectional chain -- more generally, a directed acyclic graph -- of independent transformation steps. Each \emph{filter} knows only its input and output formats; \emph{pipes} carry data between them. The classical filter roles are producer (source), transformer, tester (routing on a predicate), and consumer (sink) \citep{buschmann1996posa}. The historical root is the Unix pipe; \citet{richards2025fundamentals} rate the style (as ``pipeline architecture'') as technically partitioned with a single quantum. Its modern incarnations dominate the data world: ETL/ELT jobs, DAG orchestration, data-parallel engines, HPC job chains, and -- most recently -- machine-learning and retrieval pipelines.
|
|
|
|
\begin{definitionbox}[Pipes-and-filters / batch pipeline (PF)]
|
|
A macro-structure of independent, composable transformation steps (\emph{filters}) connected by unidirectional data conduits (\emph{pipes}) into a chain or directed acyclic graph. Each filter is self-contained -- ideally stateless and idempotent -- and knows only its own data contracts, never its neighbours; the composition logic is explicit and external to the filters \citep{buschmann1996posa,richards2025fundamentals}.
|
|
\end{definitionbox}
|
|
|
|
Figure~\ref{fig:pat-pf} shows the topology: a unidirectional chain of self-contained filters between source and sink.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
sysbox/.style={rectangle, draw, rounded corners=4pt, align=center, font=\footnotesize\sffamily, line width=0.8pt},
|
|
filter/.style={sysbox, fill=bankgreen!15, draw=bankgreen, minimum width=2.0cm, minimum height=0.9cm},
|
|
extern/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=2.0cm, minimum height=0.9cm},
|
|
arr/.style={-{Stealth[length=2.5mm]}, thick, gray!60!black}
|
|
]
|
|
\node[extern] (src) at (-5.6,0) {Source\\(producer)};
|
|
\node[filter] (f1) at (-2.9,0) {Filter 1\\(transform)};
|
|
\node[filter] (f2a) at (0,1.1) {Filter 2a\\(transform)};
|
|
\node[filter] (f2b) at (0,-1.1) {Filter 2b\\(transform)};
|
|
\node[filter] (f3) at (2.9,0) {Filter 3\\(merge/test)};
|
|
\node[extern] (sink) at (5.7,0) {Output store\\(consumer)};
|
|
\draw[arr] (src) -- node[above, font=\scriptsize\sffamily] {pipe} (f1);
|
|
\draw[arr] (f1) -- (f2a);
|
|
\draw[arr] (f1) -- (f2b);
|
|
\draw[arr] (f2a) -- (f3);
|
|
\draw[arr] (f2b) -- (f3);
|
|
\draw[arr] (f3) -- node[above, font=\scriptsize\sffamily] {pipe} (sink);
|
|
\node[font=\footnotesize\sffamily, text=gray!60!black, align=center] at (0,-2.3)
|
|
{every pipe is a versioned data contract; every filter is stateless, idempotent,\\and independently testable against golden datasets};
|
|
\end{tikzpicture}%
|
|
\caption{Topology of a pipes-and-filters system as a directed acyclic graph: independent filters connected by pipes that carry explicit data contracts. Read serving is delegated to the materialised output store (table note~2); the pipeline itself optimises makespan, not response time.}
|
|
\label{fig:pat-pf}
|
|
\end{figure}
|
|
|
|
\subsection{Variants}
|
|
|
|
(a)~The classical \emph{batch pipeline}: scheduled ETL/ELT runs inside a batch window. (b)~\emph{DAG orchestration}: Apache Airflow -- ``workflows as code'', developed at Airbnb -- externalises composition, scheduling, and retry logic \citep{beauchemin2015airflow,airflow2024docs}. (c)~\emph{Data-parallel engines}: Spark distributes filters over clusters via resilient distributed datasets \citep{zaharia2012resilient}. (d)~\emph{HPC scheduling}: Slurm job chains on compute clusters \citep{yoo2003slurm}. (e)~\emph{Streaming pipelines}: Flink and the dataflow model move the same topology from bounded batches to unbounded streams with event-time windows and exactly-once state -- the structural bridge to EDA \citep{carbone2017state,akidau2015dataflow}. (f)~\emph{ML and retrieval pipelines}: ingestion $\rightarrow$ chunking $\rightarrow$ embedding $\rightarrow$ index is the current incarnation, and the eval harness of Part~\ref{part:ai} is itself a pipeline \citep{lewis2020rag,barnett2024seven}.
|
|
|
|
\subsection{The problem it solves}
|
|
|
|
The nightly batch is the oldest workload in commercial computing: when the business day closes, the day's records must be collected, validated, transformed, and aggregated into ledgers, reports, and forecasts -- reliably, repeatably, and before the next day begins. Payroll runs, bank end-of-day processing, warehouse loads (the ETL of every BI stack), and overnight risk simulations all share one shape: a finite body of data flows through a fixed series of transformations, and nobody sits waiting interactively for the answer. What matters is that the whole run finishes inside its window -- and that the same inputs provably produce the same outputs.
|
|
|
|
The structural answer predates its modern names. The Unix pipe made composing small, single-purpose transformations a one-character operation, and the pattern literature generalised the idea into independent filters, explicit pipes, and external composition \citep{buschmann1996posa}. The shape survives every technology generation because the problem does: the same topology runs today as SQL models in a warehouse, as orchestrated DAGs, as data-parallel jobs on a cluster -- and, most recently, as machine-learning ingestion and evaluation pipelines.
|
|
|
|
Airflow, Spark, dbt, and Slurm are the pattern's industrial workhorses \citep{beauchemin2015airflow,zaharia2012resilient,yoo2003slurm}; the modern ELT data stack -- sources $\rightarrow$ orchestrated DAGs $\rightarrow$ warehouse $\rightarrow$ serving -- is the pattern at organisational scale \citep{kimball2013warehouse}.
|
|
|
|
\subsection{Capability profile}
|
|
|
|
With the nightly batch in mind, Table~\ref{tab:cap-pf} walks column PF of Table~\ref{tab:capability} (assembled at the end of this part) through all twelve dimensions; it describes a \emph{throughput} pattern, whose response measures are the \emph{makespan} -- the wall-clock time from the first filter starting to the last one finishing -- and batch-window adherence, not response time.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{Pipes-and-filters across the twelve dimensions -- column PF of Table~\ref{tab:capability}; note numbers refer to that table's notes.}
|
|
\label{tab:cap-pf}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{3.2cm}cp{8.5cm}@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{Rating} & \textbf{Structural reason} \\
|
|
\midrule
|
|
D1 Read scalability & $\circ$ & the pipeline does not serve interactive reads itself; precomputation delegates read serving to the materialised output store (note~2) \\
|
|
D2 Write scalability \& elasticity & $+$ & data-parallel frameworks are the pattern's standard operating mode for throughput (note~6) \\
|
|
D3 Latency \& predictability & $--$ & answers arrive in makespans, not milliseconds -- by design, in the batch form (note~9) \\
|
|
D4 Consistency \& integrity & $+$ & immutable inputs and idempotent stages give reproducibility, ``as of last run'' (note~11) \\
|
|
D5 Availability \& fault isolation & $-$ & a failed stage stalls the run; recovery is re-execution, not failover \\
|
|
D6 Security \& auditability & $+$ & versioned inputs and deterministic reruns make an audit trail reproducible on demand \\
|
|
D7 Evolvability \& maintainability & $+$ & filters are individually replaceable behind explicit data contracts on every pipe \\
|
|
D8 Simplicity \& time-to-market & $++$ & explicit composition logic over small, self-contained filters -- one repository, shipping this week \\
|
|
D9 Testability \& deployability & $+$ & golden datasets per filter and bit-level assertions on deterministic outputs \\
|
|
D10 Operating cost efficiency & $++$ & compute consumed in schedulable bursts, near-zero platform staff \\
|
|
D11 Team scaling (Conway) & $\circ$ & DAG and filter ownership parallelises data teams moderately, with no independent-deployment payoff beyond that (note~20) \\
|
|
D12 AI integrability & $++$ & ingestion, training, and eval workloads are pipes-and-filters by construction; non-determinism stays localised to the stage that causes it \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & \multicolumn{2}{l}{scheduled batch} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
The table gives the full profile; four cells deviate from or refine the source ratings and carry the intellectual content of the column. \textbf{D1 $= \circ$} (note~2): the source rates scalability at one star, assessing the pipeline as a single deployment; but the pipeline does not serve interactive reads itself -- precomputation \emph{delegates} read serving to the materialised output store, which is exactly how BI stacks work. \textbf{D2 $= +$} (note~6): a flagged deviation -- data-parallel frameworks are this pattern's standard operating mode for throughput, not an exotic extension. \textbf{D3 $= --$} (note~9) rates the batch form; the streaming incarnation escapes it, which is why the fit matrix in Part~\ref{part:fit} treats stream processing as pipes-and-filters over a durable log. \textbf{D4 $= +$} (note~11) is the subtlest cell in the table: the pattern is not ACID, but immutable inputs plus deterministic, \emph{idempotent} stages -- running a stage twice on the same input has exactly the effect of running it once, which is what makes retries safe -- give \emph{reproducibility} -- ``as of last run'' -- a third consistency semantics beside ACID and eventual, and for scientific and regulatory workloads the one that actually matters \citep{glasserman2003monte}. One refinement to \textbf{D9 $= +$}: dimension-for-dimension, this is the most deterministically testable pattern in the catalogue -- \emph{as long as no AI filter sits inside} (AI lens below).
|
|
|
|
\subsection{Software engineering implications}
|
|
|
|
\textbf{Build and CI/CD.} The pipeline versions \emph{three} things: code, data, and schemas -- a schema change without a data migration is a broken build waiting for the next run. \emph{Backfills} -- re-running history through changed logic -- are their own deployment class with their own risk profile and their own runbook. \textbf{Test.} \emph{Golden datasets} -- pinned inputs with known-correct outputs -- per filter; end-to-end runs against pinned inputs as data-contract tests; determinism makes bit-level assertions possible. \textbf{Operations.} Scheduling, retry with backoff, idempotency, and data-quality monitoring; the on-call page is ``the 02:00 run missed its batch window'', and the standing response measures are makespan trend and window adherence. \textbf{Maintenance.} Filters are individually replaceable by construction; the coupling risk hides in the pipes (implicit schemas). \textbf{Team size (Conway).} Small data or platform teams suffice; ownership maps to DAGs and filters.
|
|
|
|
\subsection{Exemplars, anti-patterns, and selection signals}
|
|
|
|
\begin{examplebox}[Build it and study it -- pipes-and-filters]
|
|
\textbf{Build.} Apache Airflow (Python, Apache-2.0, active) is DAG-as-code orchestration; its shipped example DAGs (under \texttt{airflow-core/src/airflow/example\_dags/}) are annotated filters-and-pipes worth reading before writing your own \citep{airflow2024docs}. dbt Core (Apache-2.0) expresses each filter as one SQL model -- with a branch nuance that rewards the maintenance check: the \texttt{main} branch hosts the v2.0 Rust rewrite in alpha, while the Python v1 line lives on the \texttt{1.latest} branch \citep{dbt2026core}. Dagster is an equally active Python alternative.
|
|
|
|
\textbf{Study.} \texttt{dbt-labs/jaffle\_shop\_duckdb} (Apache-2.0, current): every \texttt{.sql} model under \texttt{models/staging/} versus \texttt{marts/} is one filter, \texttt{ref()} wires the pipes, and \texttt{dbt build} materialises the whole DAG \citep{dbtlabs2026jaffleduckdb}. \texttt{pip install dbt-duckdb \&\& dbt build} runs it fully locally on DuckDB: very easy. (The status check again: the classic \texttt{jaffle\_shop} repository was renamed \texttt{jaffle-shop-classic} and archived in 2024, and its dbt-Cloud-oriented successor ships no licence file -- use the DuckDB variant.)
|
|
\end{examplebox}
|
|
|
|
\begin{examplebox}[A Monte-Carlo risk run -- reproducibility by construction]
|
|
A bank's overnight risk simulation is the pattern in its purest form: immutable market-data snapshots and versioned model parameters enter; embarrassingly parallel simulation stages fan out across a cluster; deterministic aggregation produces versioned risk figures \citep{glasserman2003monte}. Fixed seeds and versioned inputs make the run bit-level reproducible -- which is not a nicety but a regulatory duty for risk models \citep{bcbs2013rdarr}. The D4 $= +$ cell (note~11) describes exactly this guarantee: not ACID, but ``the same inputs produce the same books, provably, on demand''.
|
|
\end{examplebox}
|
|
|
|
\paragraph{Anti-patterns and measurable alarms.} \emph{Stateful filters with side effects}: a filter that reads yesterday's leftovers or writes outside its declared outputs destroys reproducibility; alarm: a non-zero \emph{rerun-diff rate} -- re-executing a stage on identical inputs yields different outputs. \emph{Pipeline sprawl}: hundreds of unversioned ad-hoc DAGs; alarms: the count of DAGs without owners, and DAGs absent from version control. \emph{Implicit schema coupling}: filters that agree on column names by folklore; alarm: downstream breakage count per upstream schema change -- the measurable case for explicit data contracts on every pipe.
|
|
|
|
\paragraph{Selection signals.} \textbf{Choose when:} the work is directed data transformation, throughput dominates latency, and reproducible runs are required -- simulation, BI feeds, reporting, model training. \textbf{Avoid when:} interactive latency, bidirectional workflows, or long-lived state between steps dominate -- the shape gate of Part~\ref{part:fit} will exclude the pattern from such cores anyway.
|
|
|
|
\begin{ailinse}[Pipelines are where AI work naturally lives]
|
|
D12 $= ++$. The AI-adjacent workloads of Part~\ref{part:ai} are pipes-and-filters \emph{by construction}: retrieval ingestion (documents $\rightarrow$ chunking $\rightarrow$ embedding $\rightarrow$ index) \citep{lewis2020rag,barnett2024seven}, model training and batch inference, and the eval harness itself -- a versioned pipeline from golden set to statistical verdict. One precise change occurs when an AI filter enters a previously deterministic chain: that stage's test regime switches from golden-dataset equality to \emph{statistical acceptance thresholds} (pass rates, score distributions, judge calibration) \citep{zheng2023judging} -- the rest of the pipeline keeps its deterministic tests. The pattern localises non-determinism to the stage that causes it; few structures contain AI more cheaply.
|
|
\end{ailinse}
|
|
|
|
\begin{keypoint}
|
|
Pipes-and-filters is the most deterministically testable pattern in the catalogue and the natural home of batch, data, and ML workloads. Its $--$ latency cell is not a defect but its definition: the pattern optimises makespan and reproducibility, and delegates interactive serving to the stores it fills (note~2).
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: SL -- SERVERLESS / FAAS
|
|
% ============================================
|
|
\section{SL -- Serverless / FaaS}
|
|
\label{sec:pat-sl}
|
|
|
|
\emph{Your load is zero at night and spikes at noon: why pay for idle servers at 03:00?} The serverless answer is radical -- pay per execution, own no capacity -- and the whole profile of this section is the fine print of that offer.
|
|
|
|
\subsection{Definition and topology}
|
|
|
|
Serverless computing executes short-lived functions in response to events, with the platform -- not the team -- owning resource provisioning, scaling (including \emph{to zero}), and billing per execution. The CNCF whitepaper draws the definitional line between \emph{FaaS} (your code, event-triggered) and \emph{BaaS} (API-consumed third-party services) \citep{cncf2018serverless}; the scientific reference point is the Berkeley view, which predicted the dominance of simplified cloud programming while cataloguing the pattern's structural limits with unusual precision: cold starts, stateless functions, expensive communication through storage, and vendor lock-in \citep{jonas2019berkeley}.
|
|
|
|
\begin{definitionbox}[Serverless / Function-as-a-Service (SL)]
|
|
A macro-structure of event-triggered, short-lived, stateless functions executed on a managed platform that provides provisioning, elastic scaling from zero to massive parallelism, and per-execution billing. State lives outside the functions, in managed backing services; the function is simultaneously the unit of code, of deployment, of scaling, and of cost \citep{cncf2018serverless,jonas2019berkeley}.
|
|
\end{definitionbox}
|
|
|
|
Figure~\ref{fig:pat-sl} shows the topology: event sources triggering managed functions whose state lives in backing services.
|
|
|
|
\begin{figure}[htbp]
|
|
\centering
|
|
\begin{tikzpicture}[
|
|
sysbox/.style={rectangle, draw, rounded corners=4pt, align=center, font=\footnotesize\sffamily, line width=0.8pt},
|
|
evt/.style={sysbox, fill=bankgreen!15, draw=bankgreen, minimum width=2.2cm, minimum height=0.8cm},
|
|
fn/.style={sysbox, fill=bankblue!20, draw=bankblue, minimum width=2.6cm, minimum height=0.85cm},
|
|
mgd/.style={sysbox, fill=gray!15, draw=gray!60!black, minimum width=2.4cm, minimum height=0.8cm},
|
|
arr/.style={-{Stealth[length=2.5mm]}, thick, gray!60!black}
|
|
]
|
|
% event sources
|
|
\node[evt] (http) at (-5.6,1.6) {HTTP request};
|
|
\node[evt] (q) at (-5.6,0) {queue event};
|
|
\node[evt] (sched) at (-5.6,-1.6) {schedule};
|
|
% platform boundary
|
|
\draw[dashed, gray!60!black, rounded corners=6pt] (-2.6,2.6) rectangle (2.6,-2.6);
|
|
\node[font=\footnotesize\sffamily, text=gray!60!black] at (0,2.25) {managed FaaS platform};
|
|
\node[fn] (f1) at (0,1.2) {Function A\\ \scriptsize $0 \dots n$ instances};
|
|
\node[fn] (f2) at (0,-0.3) {Function B\\ \scriptsize $0 \dots n$ instances};
|
|
\node[font=\scriptsize\sffamily, text=bankred, align=center] at (0,-1.75)
|
|
{scale-to-zero $\leftrightarrow$ burst parallelism\\per-invocation billing; cold starts};
|
|
% managed services
|
|
\node[mgd] (obj) at (5.4,1.6) {object storage};
|
|
\node[mgd] (db) at (5.4,0) {managed database};
|
|
\node[mgd] (api) at (5.4,-1.6) {third-party API\\(BaaS)};
|
|
\draw[arr] (http) -- (f1.west);
|
|
\draw[arr] (q) -- (f2.west);
|
|
\draw[arr] (sched) -- ([yshift=-0.25cm]f2.west);
|
|
\draw[arr] (f1.east) -- (obj.west);
|
|
\draw[arr] (f1.east) -- (db.west);
|
|
\draw[arr] (f2.east) -- (db.west);
|
|
\draw[arr] (f2.east) -- (api.west);
|
|
\node[font=\footnotesize\sffamily, text=gray!60!black, align=center] at (1.4,-3.3)
|
|
{functions are stateless: all state lives outside, and inter-function\\communication runs through storage -- the documented cost trap \citep{jonas2019berkeley}};
|
|
\end{tikzpicture}%
|
|
\caption{Topology of a serverless system: event sources trigger short-lived function instances inside a managed platform; state and communication live in managed services outside the functions. Elasticity and idle cost are platform properties; the cost profile inverts under sustained data-intensive load (table note~18).}
|
|
\label{fig:pat-sl}
|
|
\end{figure}
|
|
|
|
\subsection{Variants}
|
|
|
|
(a)~Public-cloud FaaS: AWS Lambda (launched November 2014), Azure Functions, Google Cloud Functions and Cloud Run. (b)~Portable, Kubernetes-based platforms: Knative \citep{knative2021release} and OpenFaaS -- serverless mechanics with self-managed lock-in economics. (c)~\emph{Edge functions} (Cloudflare Workers and peers) trading runtime restrictions for latency proximity. (d)~\emph{Orchestrated function workflows} (Step Functions, Durable Functions): a mediator topology over functions -- the overlap zone with pipes-and-filters and mediator EDA, and the exact architecture of the Prime Video case below.
|
|
|
|
\subsection{The problem it solves}
|
|
|
|
The problem serverless solves is idle capacity -- and the operations bill that comes with it. A great deal of real work is spiky or rare: a thumbnail must be generated when a user uploads an image, a webhook fires when a payment settles, a report is built when somebody asks for one. Served from provisioned machines, such workloads force a bad choice: size for the peak and pay for servers that idle through the night, or size for the average and fail at the peak. And whichever sizing wins, somebody must patch, monitor, and operate a fleet -- for logic that may amount to two hundred lines.
|
|
|
|
The serverless offer, commercially triggered by AWS Lambda's 2014 launch (Variants above), is radical on both counts: hand the platform the code and an event trigger, and the platform owns provisioning, scaling -- including \emph{to zero} -- and billing per execution \citep{cncf2018serverless}. When no events arrive, nothing runs and nothing costs; when a burst arrives, the platform fans out to massive parallelism without a capacity plan. The Berkeley analysis predicted that this simplification of cloud programming would dominate precisely because it removes the two costs above -- idle capacity and operations staffing -- while cataloguing, with unusual precision, the structural fine print the rest of this section prices: cold starts, enforced statelessness, communication through storage, vendor lock-in \citep{jonas2019berkeley}.
|
|
|
|
\subsection{Capability profile}
|
|
|
|
With the idle-server problem in mind, Table~\ref{tab:cap-sl} walks column SL of Table~\ref{tab:capability} (assembled at the end of this part) through all twelve dimensions. The column carries no star anchor -- \citet{richards2025fundamentals} do not rate serverless as a style -- so every cell derives from the Berkeley analysis and the CNCF definition, flagged accordingly \citep{jonas2019berkeley,cncf2018serverless}.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{Serverless across the twelve dimensions -- column SL of Table~\ref{tab:capability}; note numbers refer to that table's notes.}
|
|
\label{tab:cap-sl}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{3.2cm}cp{8.5cm}@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{Rating} & \textbf{Structural reason} \\
|
|
\midrule
|
|
D1 Read scalability & $+$ & wide per-invocation parallelism, tempered by platform concurrency limits and the database-connection bottleneck (note~3) \\
|
|
D2 Write scalability \& elasticity & $++$ & scale-to-zero to mass parallelism as a platform service, with no capacity planning \\
|
|
D3 Latency \& predictability & $-$ & cold starts make tail latency structurally unpredictable (note~10) \\
|
|
D4 Consistency \& integrity & $-$ & stateless functions push all state through external storage; transactions across function boundaries need heavy machinery \\
|
|
D5 Availability \& fault isolation & $+$ & platform-managed redundancy and a naturally small blast radius per function \\
|
|
D6 Security \& auditability & $\circ$ & platform-managed identity and logging per function against an audit trail fragmented across many small units \\
|
|
D7 Evolvability \& maintainability & $\circ$ & fine-grained deployability against vendor lock-in and function sprawl (note~14) \\
|
|
D8 Simplicity \& time-to-market & $\circ$ & no server management against a large configuration surface of triggers, permissions, and quotas (note~15) \\
|
|
D9 Testability \& deployability & $\circ$ & trivial unit tests against cloud wiring that no laptop reproduces (note~17) \\
|
|
D10 Operating cost efficiency & $++$/$--$ & split by load shape: zero idle cost for spiky load, per-execution billing and storage round trips dominant under sustained load (note~18) \\
|
|
D11 Team scaling (Conway) & $+$ & the smallest teams in the catalogue ship to production -- the lowest platform self-investment of all seven patterns \\
|
|
D12 AI integrability & $\circ$ & fine event glue for batch AI jobs, but platform timeout ceilings collide with minutes-long runs (note~21) \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & \multicolumn{2}{l}{event-trig., short-lived} \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
The table gives the full profile; two entries deserve the story behind the number. \textbf{D3 $= -$} (note~10) has a mitigation with a sting: provisioned concurrency removes cold starts at the price of exactly the idle cost the pattern exists to avoid. And \textbf{D10 $= ++/--$} (note~18) is the most workload-sensitive cell in the entire table and is treated as a standing sensitivity point: $++$ for spiky or sporadic load, where scale-to-zero yields zero idle cost; $--$ for sustained throughput with data-intensive inter-function communication through storage, where per-execution billing and storage round trips dominate. The cell does not average to $\circ$ because averaging would erase precisely the information an architect needs. The three $\circ$ cells D7/D8/D9 (notes~14, 15, 17) share one logic: each nets out a genuine tension rather than describing a middling property.
|
|
|
|
\subsection{Software engineering implications}
|
|
|
|
\textbf{Build and CI/CD.} Deployment granularity is excellent -- the function is the deployment unit -- but versioning, alias routing, and IAM policy complexity grow with function count; infrastructure-as-code (SAM, CDK, Terraform) becomes \emph{part of the test subject}, not an ops afterthought. \textbf{Test.} Unit tests are trivially easy; integration tests run against emulators or ephemeral cloud environments because the wiring -- triggers, permissions, quotas -- is not locally reproducible (note~17). \textbf{Operations.} Observability fragments across many small functions; distributed tracing is mandatory from day one. Above all, \textbf{cost monitoring becomes an engineering discipline (FinOps)}: cost is the pattern's primary emergent property, and cost per request belongs on the same dashboards as latency \citep{jonas2019berkeley}. \textbf{Maintenance.} Individual functions are disposable; the system-level risk is sprawl -- an inventory nobody can analyse (note~14). \textbf{Team size (Conway).} Minimal: the platform absorbs what other patterns solve with platform teams.
|
|
|
|
\subsection{Exemplars, anti-patterns, and selection signals}
|
|
|
|
\begin{examplebox}[Build it and study it -- serverless]
|
|
\textbf{Build.} Knative (Go, Apache-2.0, active) is a clean, self-hostable FaaS and serving layer on Kubernetes that runs on a local kind cluster \citep{knative2021release}. OpenFaaS remains usable for study, with an honest caveat: its Community Edition no longer ships under plain MIT but under the OpenFaaS CE EULA -- free for personal and educational use, commercial use requires a licence \citep{openfaas2026}. For AWS-style serverless code, Moto (Python, Apache-2.0, active) mocks the AWS APIs in local tests \citep{moto2026}. Two former defaults are deliberately off this list: LocalStack was archived into a closed, account-required model in March 2026, and Serverless Framework v4 sits under a proprietary customer agreement -- the licence check of the first box, twice over.
|
|
|
|
\textbf{Study.} The Knative Bookstore sample (in \texttt{knative/docs} under \texttt{code-samples/eventing/bookstore-sample-app/}, code samples Apache-2.0), with its \texttt{start/} and \texttt{solution/} directories: each capability -- sentiment analysis, comment handling -- is an independent Knative Service wired to the others by Brokers and Triggers declared in YAML \citep{knative2026bookstore}. It runs on local kind or minikube following the tutorial: the most involved setup of the seven boxes, but genuinely self-hostable end to end. Smaller: \texttt{openfaas/store-functions} (MIT).
|
|
\end{examplebox}
|
|
|
|
\begin{examplebox}[Amazon Prime Video -- a measured cost inversion]
|
|
Prime Video's audio/video-quality monitoring service was built as a distributed serverless orchestration: Step Functions coordinating Lambda-based detectors, with S3 buckets buffering video frames between stages. The design hit a hard scaling limit at roughly 5\,\% of expected load -- state-transition and storage costs dominated -- and the team consolidated the workflow into a single process on ECS, cutting infrastructure cost by more than 90\,\% \citep{kolny2023primevideo}. The correct reading is precise: this is \emph{one} service with a data-intensive, tightly coupled processing flow -- pipes-and-filters pushed across expensive distributed boundaries -- not a verdict on serverless, and not an Amazon-wide retreat from anything \citep{cockcroft2023prime}. It is table note~18 observed in production, with a price tag.
|
|
\end{examplebox}
|
|
|
|
\begin{thinkbox}
|
|
The Prime Video post was widely reported as ``Amazon abandons microservices''. Using only the vocabulary of this chapter -- workload shape, quantum boundaries, the split D10 rating -- state precisely what the case demonstrates and what it does not. Then consider: which \emph{measurement}, taken before the first release, would have predicted the inversion? (Reading engineering blogs against their headlines is a skill this module examines.)
|
|
\end{thinkbox}
|
|
|
|
\paragraph{Anti-patterns and measurable alarms.} \emph{Lambda pinball}: a request ricochets through dozens of functions and nobody can draw the path; alarms: function hops per request, and functions touched per feature change. \emph{The distributed monolith of functions}: functions that deploy together, share implicit state, and fail together -- the Section~\ref{sec:pat-ms} pathology at finer grain. \emph{Cost inversion under load growth}: the $--$ half of note~18 arriving gradually; alarm: cost-per-request trend against load trend, with a computed break-even utilisation as a standing fitness function. \emph{Cold-start denial}: latency-critical paths behind unwarmed functions; alarm: cold-start rate on p99-relevant routes.
|
|
|
|
\paragraph{Selection signals.} \textbf{Choose when:} load is spiky or sporadic, the work is event-shaped integration glue, experiments must ship fast, and operational self-investment must be minimal at low base load. \textbf{Avoid when:} sustained high throughput, strict latency SLOs, long-running stateful processes, or data-intensive inter-function communication dominate -- each is a documented Berkeley limitation, and the third is a platform timeout, not a preference \citep{jonas2019berkeley}.
|
|
|
|
\begin{ailinse}[Serverless and AI: excellent glue -- conditional runtime]
|
|
D12 $= \circ$ (note~21), and the split mirrors the D10 cell. Event glue around \emph{batch} AI APIs fits beautifully: spiky, sporadic advisory load hits the $++$ side of the cost split, and a function that submits, polls, and stores an asynchronous AI job is serverless at its best. But platform timeout ceilings collide with minutes-long LLM and solver runs -- a hard constraint, not a tuning issue -- and per-call cost stacking across functions \emph{plus} tokens is opaque without a single gateway measurement point (Part~\ref{part:ai}). The pattern is a fine chauffeur for AI jobs and a poor place for them to live.
|
|
\end{ailinse}
|
|
|
|
\begin{keypoint}
|
|
The serverless profile is dominated by one variable more than any other pattern's: load shape. Its elasticity and scale-to-zero economics are unmatched for spiky workloads -- and invert into the catalogue's worst cost cell under sustained, data-intensive load. The split D10 rating is the table being honest where an average would lie.
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: STEPPING BACK -- WHAT SEVEN PATTERNS GENERALISE TO
|
|
% ============================================
|
|
\section{Stepping Back: What Seven Patterns Generalise To}
|
|
\label{sec:pat-method}
|
|
|
|
Seven times over, this part has run the same movement: a real problem from production, a topology invented to answer it, a twelve-dimension profile explained cell by cell, the engineering consequences, and code you can run. Each section stood on its own. What no single section could deliver is the view \emph{across} the seven -- and that is what this closing section supplies: the two structural variables that explain most of what you have seen, the evidence base behind the ratings and its honest gaps, and the consolidated table that finally puts all seven profiles side by side.
|
|
|
|
The causal backbone established in Part~\ref{part:foundations} ran underneath every one of those sections: \emph{patterns are bundles of tactics}. A pattern's topology structurally packages certain elementary design decisions -- ``use an intermediary'', ``introduce concurrency'', ``maintain a single copy of state'' -- and structurally impedes others \citep{bass2021software,bachmann2003deriving}. A pattern therefore helps some quality attributes and hurts others \emph{by construction}, not by accident. This is what made the capability ratings of this part explainable rather than merely asserted: behind every cell of the tables above stands a tactic that the pattern bundles or blocks.
|
|
|
|
Because each dimension $D_i$ carries a defined response measure and measurement instrument (Table~\ref{tab:dimensions}), a rating is not an adjective but a testable prediction: $c_2(\mathrm{MS}) = ++$ predicts that a properly cut microservices system will pass a step-profile load test with near-linear throughput gains under horizontal replication; $c_4(\mathrm{EDA}) = --$ predicts that a Jepsen-style consistency test against an event-driven core will surface anomalies unless the design compensates explicitly. The scale is ordinal: the ratings support rankings and exclusions, never sums or percentages -- the formal argument against arithmetic over these profiles was given in Part~\ref{part:foundations} and returns in the matching procedure of Part~\ref{part:fit}. Two cells of the table you watched being derived at full depth even before this part began: EDA's $++$ on evolvability and MS's $--$ on consistency were the method examples of Part~\ref{part:foundations} (Section~\ref{subsec:derivation}). The pattern sections above deepened those two derivations rather than repeating them, and supplied the remaining eighty-two cells.
|
|
|
|
\subsection{The architecture quantum}
|
|
\label{subsec:pat-quantum}
|
|
|
|
Start with the unit you have met in every topology figure of this part -- the dashed boundary marking what deploys, scales, and fails together. Comparing seven topologies requires a common unit of analysis. The unit is not the class, the package, or the server, but the \emph{quantum}: the granule at which a system can be deployed, scaled, and can fail.
|
|
|
|
\begin{definitionbox}[Architecture quantum]
|
|
An \emph{architecture quantum} is an independently deployable unit of a software system that can be deployed, scaled, and can fail independently of the rest of the system -- the joint unit of deployment, of scaling, and of failure \citep{richards2025fundamentals}. A layered application, a modular monolith, and a batch pipeline each form exactly one quantum; an event-driven system forms one or more; a microservices system forms many; a serverless system forms many small, short-lived ones.
|
|
\end{definitionbox}
|
|
|
|
The quantum count is the single most powerful compression of the capability table. Nearly every systematic difference between the left and right halves of Table~\ref{tab:capability} -- cost, simplicity, consistency, and rigidity on one side; elasticity, fault isolation, team autonomy, and operational complexity on the other -- follows from how many independent quanta a pattern creates and where their boundaries run.
|
|
|
|
\begin{keypoint}
|
|
\textbf{Maxim 3.} One quantum $\rightarrow$ cheap, simple, consistent, rigid. Many quanta $\rightarrow$ expensive, complex, eventually consistent, elastic. The number of independently deployable, scalable, failable units explains most of the capability table \citep{richards2025fundamentals}.
|
|
\end{keypoint}
|
|
|
|
\subsection{Partitioning beats distribution}
|
|
\label{subsec:pat-partitioning}
|
|
|
|
The quantum count is one explanatory axis; the second is orthogonal to it: \emph{how} the units are cut. \emph{Technical partitioning} groups code by technical role -- presentation, business rules, persistence in the layered pattern; transformation stages in a pipeline. \emph{Domain partitioning} groups code by business capability -- the modules of a modular monolith, the services of a microservices system. The distinction matters because change requests arrive in the vocabulary of the domain (``add a new instrument type''), not of the technology (``modify all persistence classes''): under technical partitioning a typical feature cuts across every unit, under domain partitioning it lands inside one. This is Parnas's criterion, six decades on -- decompose around the decisions most likely to change, so that change is absorbed by a boundary instead of dispersed across the system \citep{parnas1972criteria}. The empirically measurable consequence is \emph{change dispersion} (dimension D7 of Table~\ref{tab:dimensions}): the number of modules a typical feature touches.
|
|
|
|
This part let you watch the axis operate in isolation. Reading Table~\ref{tab:cap-mm} against Table~\ref{tab:cap-l} holds the quantum count constant at one and flips only the cut: D7 and D9 rise from $-$ to $+$. Reading Table~\ref{tab:cap-ms} against Table~\ref{tab:cap-mm} holds the domain cut constant and multiplies the quanta: the evolvability stays, and the bill arrives on D8 and D10. Domain partitioning, not distribution, is what bought the change absorption.
|
|
|
|
\begin{keypoint}
|
|
\textbf{Maxim 4.} The partitioning axis beats the distribution axis: domain-oriented partitioning (MM, MS), not the number of deployment units, is the strongest single predictor of evolvability -- Parnas's criterion of decomposing around anticipated change, six decades on \citep{parnas1972criteria,richards2025fundamentals}.
|
|
\end{keypoint}
|
|
|
|
The two maxims jointly organise the seven patterns into a two-by-two logic that compresses the seven sections you have just read: L and PF are technically partitioned single quanta; MM is a domain-partitioned single quantum; MS is domain-partitioned across many quanta; EDA and SL multiply quanta along technical (event, function) seams; HX is orthogonal to both axes -- it disciplines dependency direction inside whatever quanta its host pattern creates.
|
|
|
|
\subsection{The evidence base -- and its honest gaps}
|
|
\label{subsec:pat-evidence}
|
|
|
|
Only now, with all seven profiles seen, can the evidence base behind them be discussed properly -- a caveat about a rating means little before the rating itself does.
|
|
|
|
The most systematic public rating of architectural styles is the star scheme of \citet{richards2025fundamentals}, who score each style on eleven architecture characteristics (deployability, elasticity, evolvability, fault tolerance, modularity, overall cost, performance, reliability, scalability, simplicity, testability) on a one-to-five-star scale. Where star ratings exist, this catalogue calibrates against them with a fixed mapping: five stars $\rightarrow$ $++$, four stars $\rightarrow$ $+$, three stars $\rightarrow$ $\circ$, two stars $\rightarrow$ $-$, one star $\rightarrow$ $--$. The scheme's own headline result is worth stating, because it proves that a matching problem exists at all: no style dominates. Microservices achieve the highest aggregate score across the eleven characteristics yet score one star on overall cost and simplicity; the layered architecture scores five stars on cost and simplicity yet one star on deployability, elasticity, and scalability \citep{richards2025fundamentals}. If a dominant pattern existed, this part would be one page long.
|
|
|
|
\begin{hinweisbox}
|
|
Four caveats about the evidence base must be stated before the consolidated table is put to comparative use. (i)~The star ratings are structured \emph{expert judgement}, not measurements; they are transcribed here from the first-edition figures of \citet{richards2020fundamentals} and remain to be reconciled against the revised second-edition figures before this script goes to print. (ii)~The \emph{hexagonal architecture carries no star profile at all} in \citet{richards2025fundamentals} -- correctly so, because it is a pattern of dependency organisation, not a distribution style (Section~\ref{sec:pat-hx}); its column in Table~\ref{tab:capability} is filled from \citet{cockburn2024hexagonal} and flagged as a delta profile. (iii)~\emph{Serverless} likewise has no star profile; its column derives from the Berkeley analysis \citep{jonas2019berkeley} and the CNCF whitepaper \citep{cncf2018serverless}. (iv)~The \emph{modular monolith} appears as a style with its own chapter and rating only since the second edition \citep{richards2025fundamentals,su2024modular}. The catalogue compensates by triangulation -- expert ratings $\times$ documented production cases $\times$ defined response measures -- and by the measurement contract of Part~\ref{part:fit}, which converts every adopted claim into a testable one.
|
|
\end{hinweisbox}
|
|
|
|
\subsection{The consolidated capability table}
|
|
\label{sec:capability-table}
|
|
|
|
Table~\ref{tab:capability} is the centrepiece of this part: the seven capability profiles side by side, one column per pattern, one row per profile dimension -- the seven explanation views you have just walked through, assembled into one comparison view. Each column was unpacked in its own section above; the table exists so that the columns can be \emph{compared}, which is the only operation the matching procedure of Part~\ref{part:fit} ever performs on them. The reading contract between this table and the pattern sections is deliberate: the consolidated table is the \emph{comparison} view, and each pattern section opened its capability-profile subsection with the corresponding \emph{explanation} view -- a per-pattern table (Tables~\ref{tab:cap-l}--\ref{tab:cap-sl}) that repeats that pattern's column cell for cell and adds, for every dimension, the structural reason behind the rating. Ratings live in both views identically; reasons live in the per-pattern tables; deviations from the source ratings live in the notes below Table~\ref{tab:capability}, to which the per-pattern tables refer by note number. Four reading rules apply.
|
|
|
|
\begin{enumerate}
|
|
\item \textbf{HX is a delta pattern.} The hexagonal architecture organises dependencies inside a host pattern rather than distributing a system; the symbol $\diamond$ marks dimensions on which it simply \emph{inherits} its host's rating (a hexagonal modular monolith scales like a modular monolith). Its own cells describe what the ports-and-adapters cut \emph{adds} to any host.
|
|
\item \textbf{Every deviation is footnoted.} Where a cell departs from the star ratings of \citet{richards2025fundamentals} -- because the star conflates two dimensions this theory separates, or because documented production evidence overrides it -- a table note states the reason and the evidence. An unexplained deviation would violate the theory's own rigour standard.
|
|
\item \textbf{One cell is split.} The serverless operating-cost cell (D10) reads $++$/$--$ because it genuinely inverts with load shape; note~18 explains why this is treated as a standing sensitivity point rather than averaged into a meaningless $\circ$.
|
|
\item \textbf{The last row feeds the shape gate.} $S(p)$, the native workload shape, is compared against the dominant workload shape $S(a)$ of an application class during knock-out screening in Part~\ref{part:fit}: a pattern whose native shape does not match the class's core cannot be that core's dominant structure, however good its other cells look.
|
|
\end{enumerate}
|
|
|
|
\begin{keypoint}
|
|
The capability table supports \emph{ordinal} reading only: rankings and exclusions, never weighted sums. Every cell encodes a typical workload and is a \emph{default hypothesis} -- to be replaced by measurement once the system exists. Two documented cells literally invert with context: serverless operating cost flips with load shape (Prime Video, note~18), and layered read scalability flips with cache-friendliness (Stack Overflow, note~1). The matrix predicts the default, not the exception.
|
|
\end{keypoint}
|
|
|
|
\begin{sidewaystable}[p]
|
|
\centering
|
|
\caption{Consolidated capability profiles of the seven architectural patterns over the twelve profile dimensions of Table~\ref{tab:dimensions}. Ratings are calibrated against the star ratings of \citet{richards2025fundamentals} where those exist (five stars $\rightarrow$ $++$, four $\rightarrow$ $+$, three $\rightarrow$ $\circ$, two $\rightarrow$ $-$, one $\rightarrow$ $--$), and against primary literature and documented production systems otherwise. HX is a delta pattern: $\diamond$ marks dimensions inherited from its host pattern. All deviations from the source ratings are explained in the table notes.}
|
|
\label{tab:capability}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{4.4cm}ccccccc@{}}
|
|
\toprule
|
|
\textbf{Dimension} & \textbf{L} & \textbf{MM} & \textbf{HX (delta)} & \textbf{MS} & \textbf{EDA} & \textbf{PF} & \textbf{SL} \\
|
|
& Layered / 3-tier & Modular monolith & Hexagonal & Microservices & Event-driven & Pipes-and-filters & Serverless / FaaS \\
|
|
\midrule
|
|
D1 Read scalability & $\circ$\textsuperscript{1} & $\circ$\textsuperscript{1} & $\diamond$ & $++$ & $++$ & $\circ$\textsuperscript{2} & $+$\textsuperscript{3} \\
|
|
D2 Write scalability \& elasticity & $--$ & $-$\textsuperscript{4} & $\diamond$ & $++$\textsuperscript{5} & $++$ & $+$\textsuperscript{6} & $++$ \\
|
|
D3 Latency \& predictability & $+$\textsuperscript{7} & $+$ & $\diamond$ & $-$ & $+$\textsuperscript{8} & $--$\textsuperscript{9} & $-$\textsuperscript{10} \\
|
|
D4 Consistency \& integrity & $++$ & $++$ & $\diamond$ & $--$ & $--$ & $+$\textsuperscript{11} & $-$ \\
|
|
D5 Availability \& fault isolation & $-$ & $-$ & $\diamond$ & $++$ & $++$ & $-$ & $+$ \\
|
|
D6 Security \& auditability & $+$ & $+$ & $+$ & $\circ$\textsuperscript{12} & $\circ$\textsuperscript{13} & $+$ & $\circ$ \\
|
|
D7 Evolvability \& maintainability & $-$ & $+$ & $++$ & $++$ & $++$ & $+$ & $\circ$\textsuperscript{14} \\
|
|
D8 Simplicity \& time-to-market & $++$ & $+$ & $-$ & $--$ & $--$ & $++$ & $\circ$\textsuperscript{15} \\
|
|
D9 Testability \& deployability & $-$ & $+$ & $++$ & $+$\textsuperscript{16} & $-$ & $+$ & $\circ$\textsuperscript{17} \\
|
|
D10 Operating cost efficiency & $++$ & $++$ & $\diamond$ & $--$ & $\circ$ & $++$ & $++$/$--$\textsuperscript{18} \\
|
|
D11 Team scaling (Conway) & $-$\textsuperscript{19} & $\circ$ & $\diamond$ & $++$ & $+$ & $\circ$\textsuperscript{20} & $+$ \\
|
|
D12 AI integrability & $\circ$ & $+$ & $++$ & $\circ$ & $++$ & $++$ & $\circ$\textsuperscript{21} \\
|
|
\midrule
|
|
Native workload shape $S(p)$ & interactive & interactive & (host's) & interactive & stream / async & scheduled batch & event-trig., short-lived \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
|
|
\vspace{2.5mm}
|
|
\begin{minipage}{21cm}
|
|
\footnotesize
|
|
\textbf{Table notes (deviations from the source ratings and split ratings):}
|
|
\textsuperscript{1}~Deviation from the Richards \& Ford scalability rating (one star), which conflates read and write scaling: stateless replication of the application tier plus caching demonstrably scales reads in this pattern class (Stack Overflow: $\sim$9 web servers, $\sim$12\,ms render times \citep{craver2016stackoverflow}; Instagram \citep{woodruff2019static}); the single write path remains the bottleneck -- hence D2~$=$~$--$ for L.
|
|
\textsuperscript{2}~Deviation from the one-star scalability rating (assessed as a single deployment): the pipeline does not serve interactive reads itself; precomputation \emph{delegates} read serving to the materialised output store, and data-parallel frameworks scale throughput horizontally \citep{zaharia2012resilient,yoo2003slurm}.
|
|
\textsuperscript{3}~Wide per-invocation parallelism, tempered by platform concurrency limits and the documented database-connection bottleneck \citep{jonas2019berkeley}.
|
|
\textsuperscript{4}~$-$ rather than $--$ because tenant/pod sharding of whole-monolith instances is a documented write-scaling path (Shopify BFCM peaks of $\sim$280 million requests per minute \citep{westeinde2019deconstructing,shopify2024bfcm}).
|
|
\textsuperscript{5}~Richards \& Ford rate microservices scalability five stars but elasticity only three to four stars (fine-grained provisioning, not scale-to-zero); the $++$ reads the pair jointly and notes that the elasticity half is the weaker one.
|
|
\textsuperscript{6}~The base style is rated one star (single deployment); the $+$ reflects that data-parallel execution frameworks (Spark \citep{zaharia2012resilient}, Slurm \citep{yoo2003slurm}) are the standard operating mode of this pattern for throughput workloads.
|
|
\textsuperscript{7}~Deviation from the two-star performance rating: in-process calls give low and -- more importantly -- \emph{predictable} latency; the two stars reflect throughput under load, captured separately by D1/D2.
|
|
\textsuperscript{8}~The five-star performance rating concerns throughput and responsiveness; request/response round trips \emph{through} asynchronous flows are not this pattern's mode; $+$ is the honest average.
|
|
\textsuperscript{9}~Rating of the batch form; the streaming incarnation (dataflow over a durable log) escapes it \citep{carbone2017state} -- see the C8 column of the fit matrix in Part~\ref{part:fit}.
|
|
\textsuperscript{10}~Cold starts make tail latency structurally unpredictable \citep{jonas2019berkeley}.
|
|
\textsuperscript{11}~Not ACID: immutable inputs and idempotent stages give \emph{reproducibility} -- a third consistency semantics (``as of last run'') beside ACID and eventual consistency.
|
|
\textsuperscript{12}~Expanded network attack surface and distributed audit trails ($-$) versus per-service isolation and compliance-scope carving, e.g.\ an isolated PCI payment service ($+$) \citep{pcissc2022pcidss}; net $\circ$.
|
|
\textsuperscript{13}~A durable event log supports audit ($+$), but reconstructing a causal trail across asynchronous hops requires disciplined correlation identifiers ($-$); net $\circ$.
|
|
\textsuperscript{14}~Fine-grained deployment units aid local change, but vendor lock-in and function sprawl impair system-level analysability \citep{jonas2019berkeley}.
|
|
\textsuperscript{15}~Operational simplicity (no server management, function $=$ deployment unit) offset by configuration surface and many small units; net $\circ$.
|
|
\textsuperscript{16}~Per-service testability and deployability rate four to five stars; \emph{system-level} integration verification shifts towards production (consumer-driven contracts, canaries, progressive delivery) \citep{newman2021building,humble2010continuous}; the $+$ averages this split -- a flagged deviation.
|
|
\textsuperscript{17}~Unit tests are easy; the cloud wiring is not locally reproducible (emulators, ephemeral environments); deployment granularity is excellent; net $\circ$.
|
|
\textsuperscript{18}~Split rating, load-dependent: $++$ for spiky or sporadic load (scale-to-zero, zero idle cost), $--$ for sustained throughput and data-intensive inter-function communication through storage -- the Prime Video lesson ($\sim$90\,\% cost cut after consolidation) \citep{kolny2023primevideo,jonas2019berkeley}. This is the single most workload-sensitive cell in the table and is treated as a standing sensitivity point.
|
|
\textsuperscript{19}~One quantum, one release train: effectively a single-team pattern ($\leq\sim$8--10 developers) \citep{skelton2019team}; kept at $-$ (not $--$) to preserve ordinal distance to MM's $\circ$ ($\sim$3--5 teams with release coordination).
|
|
\textsuperscript{20}~Filter/DAG ownership parallelises data teams moderately (dbt/Airflow ownership models); no independent-deployment payoff beyond that.
|
|
\textsuperscript{21}~Event glue around batch AI APIs fits; platform timeout ceilings collide with minutes-long LLM/solver runs; net $\circ$.
|
|
\end{minipage}
|
|
\end{sidewaystable}
|
|
|
|
One exercise closes the generalisation -- the consolidated table read against two production extremes you have met in full in this part:
|
|
|
|
\begin{thinkbox}
|
|
Stack Overflow serves on the order of 1.3~billion page views per month from \emph{one} architecture quantum on roughly nine web servers \citep{craver2016stackoverflow,infoq2015stackoverflow}. Monzo, a licensed bank, operates roughly 2{,}800 quanta \citep{sewell2024monzo}. Both are documented successes in demanding domains. What exactly does this pair of cases falsify about the sentence ``microservices scale better than monoliths'' -- and what does it \emph{not} falsify? Which profile dimensions of Table~\ref{tab:dimensions} would you have to measure to predict whether either company would be harmed by adopting the other's architecture?
|
|
\end{thinkbox}
|
|
|
|
% ============================================
|
|
% SECTION: READING THE CATALOGUE AS A WHOLE
|
|
% ============================================
|
|
\section{Reading the Catalogue as a Whole}
|
|
\label{sec:pat-synthesis}
|
|
|
|
One service table first. Table~\ref{tab:osscatalog} collects the ``Build it and study it'' material of the seven pattern sections into a single reference card -- the place to look when you want to \emph{see} a pattern run rather than reread it. Then three cross-cutting observations compress the seven columns of Table~\ref{tab:capability} into usable judgement.
|
|
|
|
\begin{table}[htbp]
|
|
\centering
|
|
\caption{Open-source reference card of this part: for each pattern, a framework to build it with and a system to study it in, with the local-run difficulty as verified for this module (July 2026). Licence and maintenance caveats are stated in the pattern sections' boxes -- check both again before relying on any entry.}
|
|
\label{tab:osscatalog}
|
|
\footnotesize
|
|
\begin{tabular}{@{}p{1.2cm}p{4.6cm}p{5.6cm}p{2.6cm}@{}}
|
|
\toprule
|
|
\textbf{Pattern} & \textbf{Build with} & \textbf{Study (repository)} & \textbf{Run difficulty} \\
|
|
\midrule
|
|
L & Django (Python); Spring Boot MVC (Java) & \texttt{healthchecks/healthchecks} -- four cleanly layered Django apps under \texttt{hc/} & very easy \\
|
|
MM & Spring Modulith (Java); \texttt{import-linter} (Python) & \texttt{apache/fineract} -- 34 modules, one deployable; smaller: \texttt{spring-modular-monolith} & moderate \\
|
|
HX & ArchUnit (Java); \texttt{import-linter} (Python) & \texttt{cosmicpython/code} -- \texttt{src/allocation/}: domain, service layer, adapters, entrypoints & easy--moderate \\
|
|
MS & FastAPI (Python) $+$ Kubernetes; Spring Cloud (Java) & \nolinkurl{GoogleCloudPlatform/microservices-demo} -- twelve polyglot services; alt.\ \texttt{dotnet/eShop} & moderate--hard \\
|
|
EDA & Kafka; RabbitMQ; FastStream (Python) & \texttt{home-assistant/core} -- the \texttt{EventBus} in \texttt{homeassistant/core.py} & easy to run, huge to read \\
|
|
PF & Airflow (Python); dbt Core; Dagster & \texttt{dbt-labs/jaffle\_shop\_duckdb} -- models as filters, \texttt{ref()} as pipes & very easy \\
|
|
SL & Knative; Moto (Python, test emulation) & Knative Bookstore sample (\texttt{knative/docs}) -- services wired by Brokers/Triggers & most involved \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
\textbf{No column dominates -- so a matching problem exists.} That is the headline result of the evidence base (Section~\ref{subsec:pat-evidence}), and it is assumption A2 (Part~\ref{part:foundations}) made visible in data: every column that is strong at the top of Table~\ref{tab:capability} is weak at the bottom, and vice versa. Maxims~3 and~4 explain \emph{why}: the quantum count sets the cost/elasticity/consistency trade, and the partitioning axis sets evolvability -- two variables account for most of eighty-four cells.
|
|
|
|
\textbf{The cost curves cross.} Monolithic quanta have low fixed costs and maintenance costs that grow superlinearly with size and team count unless boundary governance holds (Lehman); many-quanta patterns have high fixed costs -- platform staffing is the microservice premium's real currency \citep{fowler2015premium,portainer2024k8scost} -- or load-proportional costs (serverless), with flatter marginal cost per additional team. The five case studies of this part are the empirical support points at which fit and misfit were \emph{measured in money}: Stack Overflow and Shopify mark where single quanta carry extreme load \citep{craver2016stackoverflow,shopify2024bfcm}; Segment and Prime Video mark where distribution was rolled back at a measured saving \citep{noonan2018goodbye,kolny2023primevideo}; Uber DOMA marks where too many quanta forced a second structuring level \citep{gluck2020doma}.
|
|
|
|
\textbf{Conway is a decision filter, not a footnote.} Every column presupposes a team topology: L one team, MM three to five coordinated teams, MS stream-aligned teams plus a platform \citep{skelton2019team,conway1968committees}. The strongest single empirical finding in the field points the same way: loosely coupled architectures \emph{and teams} are the strongest predictor of continuous delivery -- with the honest caveat that the evidence is survey-based prediction, not experimental causation \citep{dora2017sodr,forsgren2018accelerate}. Checking the organisation before choosing the column is not optional; Part~\ref{part:fit} builds the check into the procedure.
|
|
|
|
What this part has deliberately \emph{not} done is decide anything: capability profiles are half of a comparison. Part~\ref{part:applications} builds the other half -- the requirements profiles of ten application classes -- and Part~\ref{part:fit} performs the match, cell by cell, with the veto rules and the measurement contract that keep it honest.
|
|
|
|
\begin{keypoint}
|
|
No column dominates, the cost curves cross, and every column presupposes a team topology. Two variables -- quantum count (Maxim~3) and partitioning axis (Maxim~4) -- explain most of the eighty-four cells of Table~\ref{tab:capability}, and the five case studies are the points at which fit and misfit were measured in money \citep{richards2025fundamentals, skelton2019team}.
|
|
\end{keypoint}
|
|
|
|
% ============================================
|
|
% SECTION: OUTLOOK: AGENT ORCHESTRATION
|
|
% ============================================
|
|
\section{Outlook: Agent Orchestration as an Emergent Composition Pattern}
|
|
\label{sec:pat-agents}
|
|
|
|
One candidate for an ``eighth pattern'' must be named now, if only to file it correctly. Agent orchestration -- chaining, routing, parallelising, and supervising LLM calls -- is \emph{not} a new architectural style: it is a composition pattern for non-deterministic runtime components that reuses the topologies of this catalogue. The load-bearing distinction is between \emph{workflows}, which orchestrate LLM calls along predefined code paths, and \emph{agents}, in which the model steers its own process and tool use \citep{anthropic2024agents}. The catalogue's topologies reappear one-for-one: prompt chaining is pipes-and-filters (Section~\ref{sec:pat-pf}); routing is a dispatch layer; parallelisation with voting is broker-style fan-out; orchestrator--workers is mediator EDA (Section~\ref{sec:pat-eda}); the evaluator--optimizer loop is a feedback control loop; and multi-agent systems are broker fan-out of autonomous quanta \citep{anthropic2024agents}. The exact mapping -- with structure, use, and the failure modes each classical analogue predicts -- is Table~\ref{tab:agenttopology} in Part~\ref{part:ai}; graph-based execution frameworks make the correspondence literal by modelling agent systems as stateful graphs with checkpointing \citep{langchain2024langgraph}.
|
|
|
|
Because the topologies are inherited, the evaluation logic of this part applies unchanged -- sharpened along three axes: non-determinism (testability becomes an evals problem), token economics (cost per request, not per infrastructure), and fallibility (fault tolerance becomes guardrails, evaluator loops, and deterministic fallbacks behind ports). The rigour case for restraint is already quantified: a multi-agent research system beat a single agent by 90.2\,\% on an internal evaluation -- at roughly fifteen times the token consumption of a chat, with token use alone explaining 80\,\% of the performance variance \citep{anthropic2025multiagent}. Autonomy is bought with cost, latency, and error accumulation; the default rule of the module is \emph{workflows before agents}, and every escalation is an ADR with a measurement contract (Part~\ref{part:ai}).
|
|
|
|
\begin{keypoint}
|
|
Chain, tree, graph, and multi-agent are re-incarnations of pipeline, routing layer, mediator, and broker. Nothing in this part becomes obsolete when the components stop being deterministic -- the capability logic applies unchanged, sharpened by non-determinism, token economics, and fallibility \citep{anthropic2024agents}.
|
|
\end{keypoint}
|
|
|
|
\begin{projektbox}
|
|
The course project composes its architecture from this catalogue -- deliberately, and traceably to the profiles of this part. Its core is exactly the verdict you watched the mini-match compute in Part~\ref{part:foundations} (Table~\ref{tab:minimatch}): MM won C10 at $++$ with no veto firing, and the HX discipline lifted D9 and D12 to $++$ -- this part has now shown, cell by cell, \emph{why} those ratings hold. The deterministic core -- portfolio data access, computation, validation, audit -- is therefore a \emph{hexagonal modular monolith} (MM $+$ HX, Sections~\ref{sec:pat-mm} and~\ref{sec:pat-hx}): domain modules with CI-verified boundaries, the LLM behind a port. The \emph{event-driven edges} (EDA, Section~\ref{sec:pat-eda}) carry the AI job spine: analysis requests queue asynchronously, and the event journal doubles as the audit trail. \emph{Pipes-and-filters} (Section~\ref{sec:pat-pf}) carries document ingestion, embedding and index builds, and the eval harness. The agent layer is an orchestrator--workers \emph{workflow} behind a port (Section~\ref{sec:pat-agents}). Each subsystem is its own architecture quantum with its own fitness functions; \emph{why} this composition matches the C10 requirements profile is derived, not asserted, in Parts~\ref{part:applications} and~\ref{part:fit}.
|
|
\end{projektbox}
|
|
|
|
% END OF PART II
|