938 lines
83 KiB
JSON
938 lines
83 KiB
JSON
{
|
|
"lecture": 13,
|
|
"week": 13,
|
|
"lessons": 3,
|
|
"title": "Lecture 13: Threats, the Shifted Matrix, Agent Orchestration -- and Synthesis",
|
|
"script_reference": "Script: Part V, Sections 42.6--42.7, 43--45",
|
|
"agenda": [
|
|
"Axis B completed: a new threat class (OWASP LLM Top 10, prompt injection) and regulation as a hard constraint (EU AI Act)",
|
|
"How AI shifts the matrix: the C10 row cell by cell, the D12 column, five shifted cells, MLOps maturity -- A6 restated",
|
|
"Agent orchestration -- the emergent eighth pattern: the advisor workflow, agent vs.\\ workflow, topologies and their classical analogues",
|
|
"The economics of autonomy and the default rule; a capability-profile sketch",
|
|
"Synthesis: one theory, five parts -- Maxim 8 and the symmetry of the two axes",
|
|
"Exam orientation: the facts, the six learning objectives applied to the five parts, what to have at hand",
|
|
"This week's exercise: M5 closes -- eval harness in CI, guard, cost observability; threat model incl. prompt injection via news and basic hardening; optional distinction work (semester plan: Kür)"
|
|
],
|
|
"recap": [
|
|
"Part V is the framework's stress test on two axes: \\textbf{Axis A} -- AI in the process (agentic tools, verification bottleneck, guardrails); \\textbf{Axis B} -- AI in the product (non-deterministic, fallible, per-call-priced runtime components)",
|
|
"Lecture 12 (§40--41, §42.1--42.5): Axis-A evidence -- two contradictory RCTs and their resolution; the \\textbf{verification bottleneck} (Maxim 7: generation is cheap, verification and architecture are binding -- D7 and D9 gain weight); Axis A compact: documentation as control interface, fitness functions as operating licence, who merges answers; Axis B: sentiment call wired wrong and right; three component types; SE4AI classics; reference architecture (\\textbf{LLM gateway} as single measurement point); eval-harness basics",
|
|
"Already computed three times: the C10 verdict -- mini-match (Lecture 3), profile and real systems (Lecture 6), the full row with its cell rationales (Lecture 10; the $7 \\times 10$ matrix itself was read in Lecture 7)",
|
|
"Filed in advance (Lecture 6 outlook): agent orchestration is a \\emph{composition pattern} reusing the catalogue's topologies; workflows before agents ($15\\times$ token finding) -- today that claim is paid out in full",
|
|
"Today completes Axis B (threats, regulation), shows how AI \\emph{shifts} the matrix, develops the eighth pattern with its profile -- and closes the module: synthesis (Maxim 8) and exam orientation. This is the last lecture of new material; week 14: one lesson synthesis and exam hints, three lessons final presentations, architecture defence and peer reviews (A3, M6)"
|
|
],
|
|
"frames": [
|
|
{
|
|
"no": 1,
|
|
"deck_section": "Title",
|
|
"title": "AISE502: AI in Software Engineering II -- Lecture 13: Threats, the Shifted Matrix, Agent Orchestration -- and Synthesis",
|
|
"kind": "content",
|
|
"script_ref": "Title slide; subtitle line: Script: Part V, Sections 42.6--42.7, 43--45",
|
|
"content": [
|
|
"\\FHGRTitlePage with subtitle as in deck 6: \\subtitle{Lecture 13: Threats, the Shifted Matrix, Agent Orchestration -- and Synthesis\\\\[0.4ex]{\\small Script: Part V, Sections 42.6--42.7, 43--45}}"
|
|
],
|
|
"elements": [
|
|
"FHGR title page (theme)"
|
|
],
|
|
"minutes": 0,
|
|
"notes": "Identical metadata block to deck 6 (author, shortname, fullname). Subtitle shortened to one clause (~70 characters) so it does not wrap to three lines on the FHGR title page."
|
|
},
|
|
{
|
|
"no": 2,
|
|
"deck_section": "Agenda",
|
|
"title": "Agenda",
|
|
"kind": "agenda",
|
|
"script_ref": "Semester plan row week 13 (Semesterplan_AISE502_HS26.md line 24)",
|
|
"content": [
|
|
"1. Axis B completed: a new threat class -- OWASP LLM Top 10, prompt injection; regulation as a hard constraint -- the EU AI Act",
|
|
"2. How AI shifts the matrix: the C10 row cell by cell; D12 across the seven patterns; five shifted cells; MLOps maturity; A6 restated",
|
|
"3. Agent orchestration -- the emergent eighth pattern: the advisor workflow; agent vs.\\ workflow; topologies and classical analogues",
|
|
"4. The economics of autonomy and the default rule; a capability-profile sketch",
|
|
"5. Synthesis: one theory, five parts -- Maxim 8; one discipline at two binding sites",
|
|
"6. Exam orientation",
|
|
"7. This week's exercise: \\textbf{M5 closes} -- eval harness in CI, guard, cost observability; threat model incl.\\ prompt injection via news $+$ basic hardening; optional distinction work (K\\\"ur)"
|
|
],
|
|
"elements": [
|
|
"\\small enumerate with \\itemsep 1pt, as in deck 6"
|
|
],
|
|
"minutes": 1
|
|
},
|
|
{
|
|
"no": 3,
|
|
"deck_section": "Agenda",
|
|
"title": "Recap: where we are",
|
|
"kind": "recap",
|
|
"script_ref": "Deck 12 (§40--41, §42.1--42.5 per task description; L12 plan summary; semester plan row 12); Maxim 7 and the verification bottleneck: part5_ai_dimension.tex 142-154 (line 142: 'code generation became cheap; specification, verification, and architecture became the binding constraints'; line 148: D7 and D9 gain weight in every requirements profile; lines 149-150: documentation as control interface, fitness functions as operating licence; line 234: 'the person who merges code answers for it'); deck 6 outlook frame (AISE502_Vorlesung_6_Folien.tex 498-521); deck 3/6/10 C10 computations; semester plan rows 7, 10, 14",
|
|
"content": [
|
|
"Part V is the framework's stress test on two axes: \\textbf{Axis A} -- AI in the process; \\textbf{Axis B} -- AI in the product (components that are non-deterministic, fallible, latency-heavy, priced per call)",
|
|
"Lecture 12 (§40--41, §42.1--42.5): Axis-A evidence -- two contradictory RCTs and their resolution; the \\textbf{verification bottleneck} (Maxim 7: generation is cheap, verification and architecture are binding -- D7 and D9 gain weight); Axis A compact: documentation as control interface, fitness functions as operating licence, who merges answers; Axis B: sentiment call wired wrong and right; three component types; SE4AI classics; reference architecture (\\textbf{LLM gateway} as single measurement point); eval-harness basics",
|
|
"The C10 verdict has been computed three times: mini-match (L, MM, MS -- Lecture 3), profile and real systems (Lecture 6), the full row with its cell rationales (Lecture 10; the $7 \\times 10$ matrix itself was read in Lecture 7)",
|
|
"Lecture 6 filed the outlook: agent orchestration is a \\emph{composition pattern} reusing the catalogue's topologies -- workflows before agents ($15\\times$ tokens). Today that claim is paid out",
|
|
"Today: Axis B completed (threats, regulation) $\\to$ how AI \\emph{shifts} the matrix $\\to$ the eighth pattern with a profile sketch $\\to$ synthesis and exam orientation. \\textbf{The last lecture of new material}; week 14: one lesson synthesis and exam hints, three lessons final presentations, architecture defence and peer reviews (A3, M6)"
|
|
],
|
|
"elements": [
|
|
"\\footnotesize itemize; one-line running map below the bullets: 'tenth class (done) -- twelfth dimension (done) -- shifted cells (today) -- one composition pattern (today)' -- the four absorption forms of §43.4 closing paragraph (line 556); frame 9 refers back to this map"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Emitted between Agenda and the first \\section, as in decks 2, 3, 5 -- no \\section{Recap}. Do not re-teach the gateway figure; name it only. The week-14 wording follows the semester plan row 14 ('1 L Synthese + 3 L Präsentationen'), deck 1's 'semester at a glance' row 14 and the exercise sheet. Bullet 2 now names Maxim 7 (the verification bottleneck) and the Axis-A compact block, because frame 33 ('one discipline at two binding sites') and Maxim 8 build directly on Maxim 7; it must stay within four lines at \\footnotesize -- if it overflows, drop 'two contradictory' and the parenthetical '(§40--41, §42.1--42.5)' before touching the Maxim-7 clause."
|
|
},
|
|
{
|
|
"no": 4,
|
|
"deck_section": "Axis B Completed -- Threats and Regulation",
|
|
"title": "A new threat class -- OWASP Top 10 for LLM Applications (1/2)",
|
|
"kind": "table",
|
|
"script_ref": "§42.6 (part5_ai_dimension.tex 455-481)",
|
|
"content": [
|
|
"\\emph{\\textcolor{bankblue}{AI components add an attack surface that classical threat models do not cover.}} (first sentence of §42.6, verbatim, line 458)",
|
|
"The OWASP Top 10 for LLM Applications 2025 codifies it; the script pairs each risk with its \\textbf{architectural} counter-measure -- deliberately, because the defence is \\emph{structural, not model-internal}",
|
|
"Table rows LLM01--LLM05: LLM01 Prompt injection (direct and indirect) -- defence in depth: privilege separation, output validation, human-in-the-loop for sensitive actions",
|
|
"LLM02 Sensitive information disclosure -- data minimisation in prompts; output filtering at the gateway",
|
|
"LLM03 Supply chain -- vetting of models, weights, and dependencies; registry discipline",
|
|
"LLM04 Data and model poisoning -- data governance and provenance for training/index data",
|
|
"LLM05 Improper output handling -- treat output as untrusted input: schema validation, encoding, ontology guard"
|
|
],
|
|
"elements": [
|
|
"Table (footnotesize, booktabs, p{1.2cm}p{4.2cm}p{7.2cm}): ID | Risk | Architectural counter-measure -- rows LLM01--LLM05 from tab:owasp, lines 460-481"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "Split the 10-row table into two frames (5 rows each) so the counter-measure column keeps its full wording. §42.6 has no italic leading question in the script; the opener is the section's declarative first sentence verbatim. Speaker question (not on the slide): 'where does the defence live, if not in the model?' -- the table answers it row by row. The remark that most counter-measures are elements of the Lecture-12 gateway figure belongs to the footer of frame 5 only."
|
|
},
|
|
{
|
|
"no": 5,
|
|
"deck_section": "Axis B Completed -- Threats and Regulation",
|
|
"title": "A new threat class -- OWASP Top 10 for LLM Applications (2/2)",
|
|
"kind": "table",
|
|
"script_ref": "§42.6, tab:owasp (part5_ai_dimension.tex 460-481)",
|
|
"content": [
|
|
"LLM06 Excessive agency -- least-privilege tool design; deterministic services own irreversible actions",
|
|
"LLM07 System prompt leakage -- no secrets or authorisation logic in prompts",
|
|
"LLM08 Vector and embedding weaknesses -- access control and tenant isolation on the retrieval index",
|
|
"LLM09 Misinformation -- grounding with mandatory citations; domain-axiom checks",
|
|
"LLM10 Unbounded consumption -- rate limits, token budgets, cost circuit breakers per tenant",
|
|
"Footer line (\\footnotesize): \\emph{most counter-measures are elements of the gateway architecture of Lecture 12} (caption of tab:owasp, line 462)"
|
|
],
|
|
"elements": [
|
|
"Table (footnotesize, booktabs, same column widths as 1/2): rows LLM06--LLM10 from tab:owasp, lines 460-481; one-line footer"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Point out that three of the four defence elements of the next frame carry OWASP IDs (LLM06, LLM05, LLM10). Project link (verbal): this table is the checklist for this week's threat model."
|
|
},
|
|
{
|
|
"no": 6,
|
|
"deck_section": "Axis B Completed -- Threats and Regulation",
|
|
"title": "Prompt injection -- why the model cannot solve it",
|
|
"kind": "keyconcept",
|
|
"script_ref": "§42.6 hinweisbox (part5_ai_dimension.tex 483-485)",
|
|
"content": [
|
|
"\\textbf{Important Note (hinweisbox, condensed):} prompt injection is not fully solvable inside the model, because LLMs process instructions and data in the \\emph{same channel}",
|
|
"Any document, e-mail, or web page the system reads can carry instructions (``ignore your previous rules and \\dots'') -- and no reliable in-model separator exists",
|
|
"The defence is therefore \\textbf{defence in depth at the system level}: least-privilege tools (LLM06), output validation (LLM05), human approval for consequential actions, consumption limits (LLM10)",
|
|
"This is the security-flavoured restatement of the section's design rule: \\textbf{the architecture, not the model, is the trust boundary}",
|
|
"For the course project, concretely: no LLM output may reach the booking path without passing the \\textbf{ontology guard}; no agent tool may perform an \\textbf{irreversible action}"
|
|
],
|
|
"elements": [
|
|
"hinweisbox 'Important Note' with the condensed text (lines 483-485); the project consequence as the last line inside the box (it is part of the hinweisbox)"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Connect to deck 6 'C10 -- the binding scenarios' footer (defence in depth is constitutive) without re-teaching it. Keep the frame to the hinweisbox plus nothing else."
|
|
},
|
|
{
|
|
"no": 7,
|
|
"deck_section": "Axis B Completed -- Threats and Regulation",
|
|
"title": "Regulation as a hard constraint: the EU AI Act",
|
|
"kind": "table",
|
|
"script_ref": "§42.7 (part5_ai_dimension.tex 487-490)",
|
|
"content": [
|
|
"Lead-in: regulation closes the quality-attribute loop with legal force -- \\textbf{Regulation (EU) 2024/1689}, the AI Act, entered into force on 1 August 2024; a risk-based approach with four classes:",
|
|
"Left table (Class | Examples / duties): \\textbf{Unacceptable risk} | prohibited practices, e.g.\\ social scoring",
|
|
"\\textbf{High risk} | Annex III use cases: creditworthiness assessment, employment, critical infrastructure -- duties: risk management, data governance, documentation, logging, oversight, accuracy/robustness/cybersecurity",
|
|
"\\textbf{Limited risk} | transparency duties: labelling AI interaction and generated content",
|
|
"\\textbf{Minimal risk} | --",
|
|
"Right table (Date | What applies): 2 Feb 2025 -- prohibitions; AI-literacy duties $\\cdot$ 2 Aug 2025 -- governance; GPAI duties $\\cdot$ 2 Aug 2026 -- general applicability incl.\\ Annex III high-risk $\\cdot$ 2 Aug 2027 -- high-risk AI in regulated products"
|
|
],
|
|
"elements": [
|
|
"Two-column layout (0.55 / 0.42): left = 4-row \\scriptsize table 'Class | Examples / duties' from line 490; right = 4-row \\scriptsize timetable 'Date | What applies', one line per row, from line 490"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "The script gives 'minimal risk' without a gloss -- print it without one; the lecturer may say 'no specific duties' as own knowledge. Spoken, not on the slide: the 2 Aug 2026 general-applicability date has passed by the time of this lecture."
|
|
},
|
|
{
|
|
"no": 8,
|
|
"deck_section": "Axis B Completed -- Threats and Regulation",
|
|
"title": "AI Act obligations -- K(a), not weights",
|
|
"kind": "table",
|
|
"script_ref": "§42.7 (part5_ai_dimension.tex 492)",
|
|
"content": [
|
|
"For this theory the AI Act has a precise, limited role: its obligations are \\textbf{quality attributes with legal force} that enter the requirements profile as \\textbf{hard constraints K(a), not as weights} -- Part I: constraints are knock-out filters, never averaged away",
|
|
"A finance-related advisory platform -- class C10, particularly with any \\emph{creditworthiness} bearing -- can fall into the \\textbf{high-risk} class",
|
|
"Then logging of agent steps, technical documentation, human oversight, and demonstrated robustness stop being engineering preferences and become \\textbf{conditions of legal operation}",
|
|
"Table: \\textbf{logging} | gateway telemetry $+$ audit journal $\\cdot$ \\textbf{human oversight} | human-in-the-loop interfaces at the determinism boundary $\\cdot$ \\textbf{robustness} | fallback chains $+$ eval harness",
|
|
"\\textbf{Key Concept (one line):} compliance, correctly designed, is not a parallel work stream -- it is the same architecture, documented"
|
|
],
|
|
"elements": [
|
|
"Three short bullets (\\footnotesize); 3-row \\footnotesize table 'Obligation | Architectural element it lands on' from line 492; one-line keypoint"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Tie back to deck 6 C10 profile row 'Constraints K(a): EU AI Act 2024/1689 (logging, oversight; potentially high-risk); GDPR' -- this frame explains why it sits in K(a) and not in the weights. The landing points are elements of fig:llmgateway (Lecture 12) -- name the figure, do not reprint it."
|
|
},
|
|
{
|
|
"no": 9,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "How AI shifts the matrix",
|
|
"kind": "content",
|
|
"script_ref": "§43 intro (part5_ai_dimension.tex 497-500)",
|
|
"content": [
|
|
"\\emph{\\textcolor{bankblue}{You have computed the C10 verdict three times -- what were those computations doing to the rest of the matrix?}}",
|
|
"The verdict itself needs no fourth derivation -- the three computations of the recap (Lectures 3, 6, 10)",
|
|
"Two of the four absorption forms on the recap's map -- the \\textbf{tenth class} and the \\textbf{twelfth dimension} -- are exactly the artefacts those computations used",
|
|
"This section supplies the generalisation in three steps: (i) the \\emph{supply-side} reading of the row you own; (ii) the full \\textbf{D12 column} it exercised; (iii) the \\textbf{cells that moved} -- cell by cell, with stated and measurable reasons",
|
|
"Then: when the pipeline promise is real (MLOps maturity) -- and A6 restated as a falsifiable claim"
|
|
],
|
|
"elements": [
|
|
"Leading question in italics (bankblue) as in decks 4-6; a small three-step roadmap (i)-(iii)"
|
|
],
|
|
"minutes": 2,
|
|
"notes": "Section opener; the computation list is not repeated here -- point at the recap's map (frame 3)."
|
|
},
|
|
{
|
|
"no": 10,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "The C10 row -- which D12 mechanism each cell exercises",
|
|
"kind": "table",
|
|
"script_ref": "§43.1 (part5_ai_dimension.tex 502-507)",
|
|
"content": [
|
|
"Opener: start from the row you own (tab:fit-c10, Lecture 10) -- the one reading the three computations used but never stated in one place: \\emph{which D12 mechanism each cell exercises}",
|
|
"Table: MM, HX | $++$ | \\textbf{boundary and port}: a CI-verifiable module boundary and an anti-corruption adapter on a port contain a fallible, entangled component",
|
|
"EDA, PF | $+$ | \\textbf{queue}: asynchronous absorption of latency, rate limits, and outages; pipeline-shaped ingestion and evals",
|
|
"L | $-$ | none of the three -- no queue, no port, no measurement point",
|
|
"MS | $\\circ$ | seconds-scale, fallible calls in synchronous chains (the missing queue)",
|
|
"SL | $\\circ$ | minutes-long LLM and solver work against platform timeout ceilings",
|
|
"\\textbf{Key Concept (recommendation, Part IV and ADR-007):} a hexagonal modular monolith plus pipelines and an orchestrated agent workflow, with EDA as the secondary job/audit spine -- governed by token and latency budgets and the determinism boundary: \\emph{agents propose; deterministic services decide and book}"
|
|
],
|
|
"elements": [
|
|
"5-row \\footnotesize table 'Cells | Fit | D12 mechanism exercised', p{1.6cm}cp{9.6cm}, from lines 505 (mechanisms) with the Fit ratings of tab:fit-c10 (part4_fit.tex 439-445: L $-$, MM $++$, HX $++$, MS $\\circ$, EDA $+$, PF $+$, SL $\\circ$); recommendation from line 507 alone in a keypoint below the table"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "Only opener + table + keypoint -- the bullets of the earlier draft duplicated the table rows. The ratings of the capped cells (L, MS, SL) are not restated in §43.1 ('capped'); they are taken from tab:fit-c10, taught in Lecture 10, so that the Fit column follows the rating convention of decks 4-6 instead of mixing math ratings with the word 'capped'."
|
|
},
|
|
{
|
|
"no": 11,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "D12 across the seven patterns",
|
|
"kind": "table",
|
|
"script_ref": "§43.2, tab:d12row (part5_ai_dimension.tex 509-532)",
|
|
"content": [
|
|
"Intro line: the D12 row of the Lecture-6 consolidated table, now with tactic-level rationales -- D12 (AI integrability) measures whether the pattern naturally provides \\textbf{the queue, the port, and the measurement point} a slow, fallible, per-call-priced component requires",
|
|
"L -- Layered $\\circ$: technical layers give the non-deterministic component no boundary, no queue, and no measurement point of its own",
|
|
"MM -- Modular monolith $+$: a dedicated AI module with a hard, CI-verifiable interface contains the component cheaply",
|
|
"HX -- Hexagonal $++$: the LLM is an adapter on a port -- swappable, mockable, contract-guarded; the ACL discipline structurally built in",
|
|
"MS -- Microservices $\\circ$: per-service isolation helps; synchronous chains through seconds-scale calls hurt -- net neutral",
|
|
"EDA -- Event-driven $++$: queues absorb exactly what LLMs are worst at -- latency, rate limits, outages; natural batching points",
|
|
"PF -- Pipes-and-filters $++$: ingestion, training, and eval pipelines are pipes-and-filters by construction",
|
|
"SL -- Serverless $\\circ$: event-glue around batch AI APIs fits; platform timeout ceilings collide with minutes-long LLM/solver runs"
|
|
],
|
|
"elements": [
|
|
"7-row \\scriptsize table, \\renewcommand{\\arraystretch}{0.8}, tabular{@{}p{2.6cm}cp{8.4cm}@{}} 'Pattern | D12 | Rationale', from tab:d12row lines 514-532"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "This is the D12 row of deck 6's consolidated table, now with tactic-level rationales -- say so in the intro line only, do not re-show the whole table."
|
|
},
|
|
{
|
|
"no": 12,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "Which existing cells shift, and why (1/2)",
|
|
"kind": "content",
|
|
"script_ref": "§43.3 (part5_ai_dimension.tex 534-542)",
|
|
"content": [
|
|
"Beyond the new row and column, AI as a runtime component moves \\emph{existing} evaluations in stated directions -- all five visible in the D12 ratings, each carrying a measurable reason",
|
|
"\\textbf{1. Asynchronous patterns gain (EDA, PF $\\uparrow$).} Queues and pipelines absorb what LLMs are worst at -- latency, rate limits, outage -- and ingestion and eval pipelines are pipes-and-filters by construction",
|
|
"\\textbf{2. Hexagonal gains most (HX $\\uparrow$).} The ACL/port discipline is exactly what the CACE problem demands; Assumption A1's cost-of-change criterion bites hardest at \\emph{model replacement}; testing against deterministic fakes is the only way to keep the deterministic 95\\,\\% of the system deterministic",
|
|
"\\textbf{3. Synchronous distributed chains lose (MS $\\downarrow$ where LLM calls sit in the request path).} Seconds-scale latency and per-hop failure probability multiply along the chain; without constitutive stability patterns this is a cascade design"
|
|
],
|
|
"elements": [
|
|
"Numbered list 1-3 (\\footnotesize), each movement in bold with its arrow; from lines 537-542"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "Ask the class to name the tactic behind each movement before revealing it (keypoint of frame 13 demands exactly that)."
|
|
},
|
|
{
|
|
"no": 13,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "Which existing cells shift, and why (2/2)",
|
|
"kind": "keyconcept",
|
|
"script_ref": "§43.3 + keypoint (part5_ai_dimension.tex 543-549)",
|
|
"content": [
|
|
"\\textbf{4. Serverless is conditional (SL $\\sim$).} Platform timeout ceilings against minutes-long LLM and solver runs cap it; event-glue around batch APIs remains a fit",
|
|
"\\textbf{5. A cost dimension becomes load-bearing everywhere.} Cost per request, feature, and tenant is a runtime quality attribute with no counterpart in classical profiles; it belongs in the gateway and in CI budgets",
|
|
"Routing across cheap and expensive models is the new incarnation of a classical resource-arbitration tactic: \\textbf{cascades} up to $\\sim$98\\,\\% cost reduction at comparable quality; \\textbf{learned routers} more than $2\\times$ cheaper without quality loss",
|
|
"\\textbf{Key Concept:} the matrix does not get \\emph{rewritten} by AI; it gets \\emph{shifted} -- in five stated directions, for five stated and measurable reasons",
|
|
"A student who can name, for any cell movement, the quality-attribute mechanism behind it (which tactic the pattern bundles or impedes for a slow, fallible, per-call-priced component) has understood both Part IV and Part V"
|
|
],
|
|
"elements": [
|
|
"Numbered list 4-5 (continuing, lines 543-544), then keypoint box from lines 547-549 (condensed)"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "The keypoint is exam-relevant: it states precisely what 'understood' means for Parts IV-V. It is quoted again on frame 37."
|
|
},
|
|
{
|
|
"no": 14,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "MLOps maturity: the three-level ladder",
|
|
"kind": "table",
|
|
"script_ref": "§43.4 (part5_ai_dimension.tex 551-554)",
|
|
"content": [
|
|
"For type-(b) components the PF cells' promise (the D12 $++$) is realised only at sufficient process maturity -- the canonical three-level ladder:",
|
|
"\\textbf{Level 0} | manual, script-driven, interactive; data science and operations separated; releases rare, no CI/CD, minimal monitoring -- \\emph{the documented reality of many teams} | model handed ``over the fence'' as an artefact",
|
|
"\\textbf{Level 1} | automated ML pipeline with continuous training; automated data and model validation, triggers, metadata store, feature store | \\emph{the pipeline, not the model, is the deployment artefact}",
|
|
"\\textbf{Level 2} | CI/CD automation of the pipeline components themselves | fast, reliable experiment-to-production cycles",
|
|
"Footer (\\scriptsize, grey): nine consolidated MLOps principles -- CI/CD automation, workflow orchestration, reproducibility, versioning of data/model/code, collaboration, continuous training and evaluation, metadata tracking, monitoring, feedback loops"
|
|
],
|
|
"elements": [
|
|
"3-row \\scriptsize table 'Level | Characteristics | Key trait', one-line cells, from line 554; the nine principles as one \\scriptsize grey footer line (or speaker notes if the frame is tight)"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "The third column is headed 'Key trait', not 'Deployment artefact', because the script states a deployment artefact for Levels 0 and 1 only. The PF promise is the D12 row's $++$ (§43.2); in the C10 row PF rates $+$."
|
|
},
|
|
{
|
|
"no": 15,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "When the pipeline promise is real: the fit-theoretical reading",
|
|
"kind": "keyconcept",
|
|
"script_ref": "§43.4 (part5_ai_dimension.tex 554)",
|
|
"content": [
|
|
"\\textbf{Fit-theoretical reading:} the MLOps level describes how much of \\textbf{D9} (testability/deployability) and \\textbf{D12} the organisation can actually \\emph{cash in}",
|
|
"A Level-0 team holding a $++$ pattern rating realises little of it",
|
|
"This is the \\textbf{Axis-B echo of DORA's Axis-A finding} (Lecture 12): \\emph{guardrail maturity, not tool adoption}, converts potential into performance"
|
|
],
|
|
"elements": [
|
|
"Two-line keypoint-style line for the D9/D12 cash-in sentence; the Level-0 example and the DORA echo as two short bullets (\\footnotesize); from line 554"
|
|
],
|
|
"minutes": 2,
|
|
"notes": "Short frame, deliberately: this sentence is the exam-relevant one of §43.4 and must not be the sixth element of a crowded slide. The DORA finding was taught on Axis A in Lecture 12 -- name it, do not re-teach it."
|
|
},
|
|
{
|
|
"no": 16,
|
|
"deck_section": "How AI Shifts the Matrix",
|
|
"title": "Assumption A6 restated as a falsifiable claim",
|
|
"kind": "keyconcept",
|
|
"script_ref": "§43.4 closing + keypoint (part5_ai_dimension.tex 556-560)",
|
|
"content": [
|
|
"Three of the four absorption forms are now on the table, each \\emph{computed rather than asserted}: the tenth class, the twelfth dimension, the shifted cells -- the fourth, the emergent composition pattern, follows next",
|
|
"\\textbf{Key Concept -- A6 restated:} runtime AI components are non-deterministic, fallible, latency-heavy, per-call-priced, and subject to drift and vendor deprecation",
|
|
"They \\emph{stretch} existing quality dimensions by orders of magnitude and add sub-attributes: token cost per request, eval pass rate, provider deprecation risk, prompt-injection resistance",
|
|
"What does \\emph{not} change is the method: scenarios with response measures, tactics, trade-off analysis, ADRs, fitness functions",
|
|
"The theory absorbs AI -- as a tenth application class, a twelfth profile dimension, shifted cell values, and one emergent composition pattern -- instead of being reinvented for it",
|
|
"The quality gate is carried by the one new test-artefact class A6 named from the start: the \\textbf{eval harness}"
|
|
],
|
|
"elements": [
|
|
"keypoint box from lines 558-560 (condensed); above it the one-sentence status line from line 556"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Bridge frame: closes §43 and opens §44 -- 'the assumption is now a falsifiable claim with evidence attached'."
|
|
},
|
|
{
|
|
"no": 17,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Agent orchestration -- the emergent eighth pattern",
|
|
"kind": "content",
|
|
"script_ref": "§44 intro (part5_ai_dimension.tex 565-568)",
|
|
"content": [
|
|
"\\emph{\\textcolor{bankblue}{``Should I reduce my exposure to European industrials?'' -- no single model call can answer that responsibly, so what structure can?}}",
|
|
"The final structural novelty of the AI era is the orchestration of agents",
|
|
"As everywhere in this script, \\textbf{the concrete system comes before the catalogue}: first the advisory workflow the course project actually builds, then the name of what it is an instance of",
|
|
"Rhythm of this block (the pattern rhythm of decks 4-6, adapted): case $\\to$ what an agent is and is not $\\to$ topologies and classical analogues $\\to$ choosing a topology $\\to$ economics and the default rule $\\to$ capability-profile sketch $\\to$ project link"
|
|
],
|
|
"elements": [
|
|
"Leading question in italics (bankblue); short roadmap line"
|
|
],
|
|
"minutes": 2
|
|
},
|
|
{
|
|
"no": 18,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Case: the course project's advisor workflow",
|
|
"kind": "case",
|
|
"script_ref": "§44.1 (part5_ai_dimension.tex 570-573)",
|
|
"content": [
|
|
"Diagram, full width: \\textbf{Orchestrator} (an LLM call with a fixed system prompt) splits the request into sub-tasks and delegates to three specialists -- \\textbf{Document analyst}, \\textbf{Portfolio quant}, \\textbf{Compliance checker}; the portfolio quant calls \\emph{deterministic analytics services}; the merged answer passes the \\textbf{guard} before reaching user or books; every call passes through the \\textbf{LLM gateway}",
|
|
"Footer line 1: orchestrator splits and delegates, then merges the results into one grounded answer",
|
|
"Footer line 2: document analyst returns extracted findings from the ingestion corpus \\emph{with citations}; portfolio quant produces exposure and concentration numbers by calling only deterministic analytics services -- \\emph{arithmetic is not a job for a language model}",
|
|
"Footer line 3: compliance checker verifies the draft -- every cited passage exists, every entity resolves against the ontology, every mandate constraint holds"
|
|
],
|
|
"elements": [
|
|
"tikz diagram drawn from the prose of line 573 (no figure exists in the script), full width in \\resizebox{0.75\\textwidth}{!}{...}: Orchestrator (violet agentbox) fanning out to Document analyst / Portfolio quant / Compliance checker (violet); Portfolio quant with an arrow to 'deterministic analytics services' (green gatebox); a dashed 'LLM gateway' band around all LLM calls; a green 'guard' gate between the merged answer and 'user / books'. Colours as fig:agenttopologies: violet = non-deterministic, green = deterministic. Below: a \\footnotesize footer of at most three lines"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "Deck-6 topology-frame form (diagram + three-line footer), not a two-column layout. The gateway/guard sentence ('every call flows through the LLM gateway -- routed, cached, budgeted, logged -- and nothing any agent produces reaches the user or the books without passing the guard: agents propose; deterministic services decide and book') opens frame 19. Label nothing the prose does not name."
|
|
},
|
|
{
|
|
"no": 19,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Strip the labels: a shape you know cell by cell",
|
|
"kind": "content",
|
|
"script_ref": "§44.1 (part5_ai_dimension.tex 573-575)",
|
|
"content": [
|
|
"Opening line: every call by every participant flows through the \\textbf{LLM gateway} -- routed, cached, budgeted, logged -- and nothing any agent produces reaches the user or the books without passing the \\textbf{guard}: \\emph{agents propose; deterministic services decide and book}",
|
|
"Now strip the labels: a coordinator decomposing work for specialised workers is the \\textbf{mediator topology of event-driven architecture} (Lecture 5); the fixed retrieve--extract--check sequence inside each specialist is a \\textbf{pipeline} (Lecture 6); peers coordinating over shared context would be the \\textbf{broker topology}",
|
|
"The section's deliberately \\emph{deflationary} claim: agent orchestration is \\textbf{not a new architectural style} but a \\textbf{composition pattern for non-deterministic runtime components} that reuses the topologies of the seven patterns you already know -- which is why it can be evaluated with the \\textbf{rating grid you already have}",
|
|
"This is the claim the Part II outlook (Lecture 6) filed in advance; this section pays it out -- the full six-row mapping follows on frame 22"
|
|
],
|
|
"elements": [
|
|
"Four bullets (\\footnotesize); the deflationary claim as a keypoint-styled line; no mini-table (the six-row table of frame 22 supersedes it)"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Trimmed to the gateway/guard rule, the three analogues named in line 575, and the deflationary claim; the mapping table is not pre-empted here."
|
|
},
|
|
{
|
|
"no": 20,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "What an agent is -- and is not",
|
|
"kind": "definition",
|
|
"script_ref": "§44.2 definitionbox + guidance (part5_ai_dimension.tex 577-584)",
|
|
"content": [
|
|
"\\textbf{Definition: Agent; workflow vs.\\ agent} -- an \\emph{agent} is an LLM running in a loop with tools and state: at each iteration the model observes the current state (conversation, tool results, working memory), selects an action (a tool call or an answer), and the loop executes it and feeds the result back -- until a termination condition holds",
|
|
"Schematically: \\textbf{agent $=$ loop $+$ tools $+$ state}",
|
|
"The load-bearing distinction: a \\textbf{workflow} orchestrates LLM calls and tools along \\emph{predefined code paths} -- deterministic structure, non-deterministic building blocks; an \\textbf{agent} lets the model \\emph{steer its own process and tool use} -- the control flow itself becomes model output",
|
|
"Autonomy is therefore not a binary but a \\textbf{dial}, and every notch on it costs latency, tokens, and testability",
|
|
"Below the box -- engineering guidance of the source that defined this vocabulary, matching the module's philosophy verbatim: \\textbf{find the simplest solution possible} $\\cdot$ \\textbf{prefer simple, composable patterns over frameworks} $\\cdot$ \\textbf{escalate to agents only when the task genuinely requires open decision paths}",
|
|
"Orchestration frameworks that model workflows as \\emph{explicit graphs} make the topology \\textbf{inspectable} -- an architectural virtue for the same reason a C4 diagram is (Lecture 3)"
|
|
],
|
|
"elements": [
|
|
"definitionbox[Agent; workflow vs.\\ agent] from lines 580-582 (condensed, with the schematic 'agent = loop + tools + state' as a centred line inside); below it the three guidance rules of line 584 as three bold one-liners and the explicit-graph sentence"
|
|
],
|
|
"minutes": 5,
|
|
"notes": "Frames 19 and 20 of the earlier draft merged: definitionbox uncrowded (four lines), guidance as three bold one-liners below. Deck 6 outlook stated the workflow/agent distinction in one line; here it gets its full definition. Emphasise 'the control flow itself becomes model output' -- that is what changes testability. The ADR-per-escalation rule is NOT stated here -- it is the §44.4 keypoint and is paid out on frame 25."
|
|
},
|
|
{
|
|
"no": 21,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Topologies and their classical analogues",
|
|
"kind": "table",
|
|
"script_ref": "§44.3, tab:agenttopology (part5_ai_dimension.tex 586-608)",
|
|
"content": [
|
|
"The Lecture-6 outlook table, now with its third column -- structure and use: every workflow topology maps exactly onto a structure from Part II that you know cell by cell, and every property the classical pattern is known for (and every weakness) \\emph{transfers}",
|
|
"Prompt chaining (chain) | Pipes-and-filters (PF) | each call processes the previous output; programmatic gates between stages; fixed decomposition",
|
|
"Routing | Routing layer / mediator | a classification step directs inputs to specialised prompts or models; the runtime sibling of model routing",
|
|
"Parallelisation (sectioning, voting) | Broker-style fan-out | independent subtasks in parallel, or repeated runs with majority vote",
|
|
"Orchestrator--workers (tree/graph) | Mediator EDA | a lead model decomposes the task dynamically and delegates to workers; central workflow control",
|
|
"Evaluator--optimizer | Feedback control loop | generator and evaluator model iterate until a quality criterion holds",
|
|
"Autonomous multi-agent | Broker topology | peer agents coordinate over shared context; maximal flexibility, minimal central control"
|
|
],
|
|
"elements": [
|
|
"6-row \\footnotesize table 'Topology | Classical analogue | Structure and use', p{3.2cm}p{3.0cm}p{6.4cm}, from tab:agenttopology lines 591-608, analogue column in the script's exact wording"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "Same mapping as the Lecture-6 outlook -- the analogue column is now the script's exact wording (deck 6 said 'dispatch layer' for routing and 'broker fan-out of autonomous quanta' for multi-agent systems; the script says 'Routing layer / mediator' and 'Broker topology' -- mention the two renamed cells verbally), and the third column is new. Say: 'the classical analogue predicts both the strengths and the failure modes' (caption, line 593)."
|
|
},
|
|
{
|
|
"no": 22,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Four topologies, drawn -- gates in code, judgement in the model",
|
|
"kind": "diagram",
|
|
"script_ref": "§44.3, fig:agenttopologies (part5_ai_dimension.tex 610-651)",
|
|
"content": [
|
|
"Chain $\\hat{=}$ pipes-and-filters: LLM 1 $\\to$ \\textcolor{bankgreen}{gate} $\\to$ LLM 2 $\\to$ LLM 3",
|
|
"Orchestrator--workers $\\hat{=}$ mediator EDA: Orchestrator $\\to$ Worker A / Worker B / Worker C",
|
|
"Evaluator--optimizer $\\hat{=}$ control loop: Generator $\\rightleftarrows$ Evaluator (feedback)",
|
|
"Multi-agent $\\hat{=}$ broker topology: Agent 1, Agent 2, Agent 3 over a shared context / bus",
|
|
"Caption: deterministic gates (green) between non-deterministic stages (violet) are the \\textbf{workflow discipline}: \\emph{structure stays in code, judgement stays in the model}"
|
|
],
|
|
"elements": [
|
|
"tikz figure redrawn from lines 610-651 with the deck's styles (agentbox violet fill aiviolet!15, gatebox bankgreen!15, rounded corners 3pt, Stealth arrows); wrap in \\resizebox{0.85\\textwidth}{!}{...}; caption line (line 649) in \\footnotesize below"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Show the advisor workflow of frame 18 as an instance of the second topology (orchestrator--workers) -- point, don't redraw."
|
|
},
|
|
{
|
|
"no": 23,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "The evaluation logic applies unchanged -- sharpened along three axes",
|
|
"kind": "content",
|
|
"script_ref": "§44.3 (part5_ai_dimension.tex 653)",
|
|
"content": [
|
|
"Because the topologies are the old topologies, the evaluation logic of this theory applies unchanged -- sharpened along exactly three axes:",
|
|
"\\textbf{Non-determinism} -- testability is read as \\emph{eval coverage} (the eval harness of Lecture 12)",
|
|
"\\textbf{Token economics} -- cost per \\emph{request}, not per infrastructure",
|
|
"\\textbf{Fallibility} -- fault tolerance is read as guardrails, evaluator loops, and deterministic fallbacks behind ports",
|
|
"Consequence: the choice of topology can be compressed into the same style of decision aid the matrix provides -- next frame"
|
|
],
|
|
"elements": [
|
|
"Three axes as three short bold-headed items (\\footnotesize); no dimension numbers on the slide -- the script names the axes, not D-numbers"
|
|
],
|
|
"minutes": 2,
|
|
"notes": "The mapping of the three axes onto D9, D10, D5 is the lecturer's own and may be spoken, not printed."
|
|
},
|
|
{
|
|
"no": 24,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Choosing a topology from the task signature",
|
|
"kind": "table",
|
|
"script_ref": "§44.3, tab:agentchoice (part5_ai_dimension.tex 655-671)",
|
|
"content": [
|
|
"Read top-down and \\textbf{stop at the first matching row} -- the ordering encodes ``simplest structure first''",
|
|
"Fixed decomposition; every intermediate result verifiable | Prompt chain (workflow) | cheapest and most testable; deterministic gates between stages",
|
|
"Heterogeneous input categories with specialised handling | Routing | cheap; the router itself needs its own eval",
|
|
"Broad, parallelisable subtasks; breadth-first search | Parallelisation or orchestrator--workers | token cost multiplies with worker count ($\\sim$15$\\times$ class)",
|
|
"Output must clear a measurable quality bar | Evaluator--optimizer | latency and cost grow per iteration; needs a reliable evaluator",
|
|
"Path genuinely unknown; open-ended tool use | Agent | highest cost and risk; guardrails, budgets, and oversight mandatory",
|
|
"Project pointer: your Axis-B ADR must justify your topology \\emph{against this table}"
|
|
],
|
|
"elements": [
|
|
"5-row \\footnotesize table 'Task signature | Topology | Cost/risk note', p{4.6cm}p{3.2cm}p{4.8cm}, from tab:agentchoice lines 655-671; one-line project pointer (projektbox, line 715)"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "Ask: which row does the advisor workflow of frame 18 match? (orchestrator--workers, row 3 -- with the 15x cost class attached)."
|
|
},
|
|
{
|
|
"no": 25,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "The economics of autonomy -- and the default rule",
|
|
"kind": "keyconcept",
|
|
"script_ref": "§44.4 + keypoint (part5_ai_dimension.tex 673-680)",
|
|
"content": [
|
|
"The rigour case for restraint, quantified by the best-documented production account (previewed in the Lecture-6 outlook): Anthropic's multi-agent research system -- an orchestrator--workers design -- beat a single agent by \\textbf{90.2\\,\\%} at roughly \\textbf{15$\\times$ the tokens} of a chat interaction (simple agents $\\sim$4$\\times$); token use alone explains \\textbf{80\\,\\%} of the variance",
|
|
"A textbook \\textbf{CBAM decision} in the sense of Part IV: autonomy is bought with cost, latency, and error accumulation -- justified only where the task's utility-response curve clears the price: broad, parallelisable research questions do; a form-filling workflow does not",
|
|
"Regulation closes the loop from the other side: the AI Act's logging, documentation, human-oversight, and robustness duties attach to \\emph{exactly the autonomy this section prices}",
|
|
"\\textbf{Key Concept -- the default rule for agent architecture:} workflows before agents; the simplest structure first; autonomy only on demonstrated need; every escalation an ADR with a measurement contract. \\emph{An orchestration decision without a token budget and an eval threshold is an opinion -- Maxim 6 applies to agents without modification}"
|
|
],
|
|
"elements": [
|
|
"Three bullets (\\footnotesize, the first at two lines), then keypoint box from lines 678-680 with the Maxim-6 sentence inside it, as in the script"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "The numbers are recognised from deck 6 (the script says so, line 676) -- spend the time on the CBAM reading and the AI Act closure, which are new. This is where the 'dial' of frame 20 is paid out: every notch of autonomy is an escalation, and every escalation an ADR with a measurement contract."
|
|
},
|
|
{
|
|
"no": 26,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Agent orchestration -- capability-profile sketch",
|
|
"kind": "table",
|
|
"script_ref": "§44.5, tab:agentprofile (part5_ai_dimension.tex 682-710)",
|
|
"content": [
|
|
"Header note: provisional profile (state 2025/26) -- every cell carries its own temporal fitness function",
|
|
"D1 Read scalability | $\\circ$ | orchestration state small and replicable; throughput bounded by provider rate limits",
|
|
"D2 Write scalability \\& elasticity | $\\circ$ | fan-out cheap in structure, expensive in tokens; sustained volume quota-bound",
|
|
"D3 Latency \\& predictability | $--$ | seconds per step, steps multiply along the loop; open-ended loops have no tail bound",
|
|
"D4 Consistency \\& integrity | $--$ | non-deterministic by construction; integrity exists only outside, in deterministic validation",
|
|
"D5 Availability \\& fault isolation | $-$ | every step adds provider failure modes and error accumulation; evaluator loops and fallbacks mitigate",
|
|
"D6 Security \\& auditability | $\\circ$ | step and tool logging natural ($+$), legally required in high-risk uses; injection and excessive agency widen the surface ($-$)",
|
|
"D7 Evolvability | $+$ | prompts, tools, models swap behind contracts; framework and model churn claws part back",
|
|
"D8 Simplicity \\& time-to-market | $-$ | a chain workflow is genuinely simple; every notch of autonomy is not",
|
|
"D9 Testability \\& deployability | $--$ | exhaustive testing impossible; evals with statistical thresholds replace assertions",
|
|
"D10 Operating cost efficiency | $--$ | token cost per request orders of magnitude above classical calls; multi-agent $\\sim$15$\\times$ chat",
|
|
"D11 Team scaling | $\\circ$ | sub-agent and tool ownership parallelises teams moderately -- the PF filter-ownership analogy",
|
|
"D12 AI integrability | $++$ | it \\emph{is} the composition pattern for AI components -- trivially its own best host",
|
|
"Status row (in place of the 'Native shape' row of decks 4-6): default hypotheses; re-verified on every model generation"
|
|
],
|
|
"elements": [
|
|
"One 12-row \\scriptsize table, \\renewcommand{\\arraystretch}{0.8}, tabular{@{}p{2.9cm}cp{7.8cm}@{}} 'Dimension | Rating | Ground', each Ground condensed to $\\le$ 12 words as deck 6 does for PF/SL, from tab:agentprofile lines 687-710; a 'Status' row instead of 'Native shape'"
|
|
],
|
|
"minutes": 5,
|
|
"notes": "The eighth profile in the exact form of the seven profiles of decks 4-6 (one 12-row scriptsize table, arraystretch 0.8) -- the rhythm students have seen seven times. The three caveats and the reading follow on frame 27."
|
|
},
|
|
{
|
|
"no": 27,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Agent orchestration -- reading the sketch",
|
|
"kind": "keyconcept",
|
|
"script_ref": "§44.5 (part5_ai_dimension.tex 685, 712)",
|
|
"content": [
|
|
"Why only a \\emph{sketch} -- three caveats: no Richards--Ford star ratings; no decade of production case studies; every cell is a \\textbf{default hypothesis} in the sense of the theory's own limits discussion -- to be replaced by measurement and \\textbf{re-verified on every model generation}",
|
|
"The cells read, as always, ``as the dominant structure of the subsystem it governs'' -- here the \\textbf{AI subsystem}, not the whole platform",
|
|
"\\textbf{The reading:} the profile explains at a glance why agent orchestration can never be the dominant structure of a whole platform of class C1--C9 -- it is \\textbf{vetoed by every High weight on D3, D4, D9, or D10} (stage 2 of the three-stage match, Lecture 7)",
|
|
"\\textbf{Key Concept:} it is, and remains, an \\textbf{edge pattern} -- hosted behind the ports of a deterministic core, exactly where the C10 recommendation places it"
|
|
],
|
|
"elements": [
|
|
"Three bullets (\\footnotesize) from line 685 and line 712; the edge-pattern sentence as a keypoint"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Link the veto reading to the three-stage match (stage 2, veto rule): four $--$ cells against Highs -- the veto logic that capped MS for C1 in Lecture 7 (lifted to $\\circ$ only under the documented Monzo condition, part4_fit.tex line 55)."
|
|
},
|
|
{
|
|
"no": 28,
|
|
"deck_section": "Agent Orchestration -- The Emergent Eighth Pattern",
|
|
"title": "Project link: Axis B is what you build -- and what you build it as",
|
|
"kind": "content",
|
|
"script_ref": "§44.5 projektbox (part5_ai_dimension.tex 714-716)",
|
|
"content": [
|
|
"\\textbf{Project Link (projektbox):} Axis B is \\emph{what} you build; the advisor workflow -- an orchestrator with two to three specialised sub-agents (document analyst, portfolio quant, compliance checker), all behind the LLM gateway -- is what you build it \\emph{as}",
|
|
"The graded Axis-B deliverables are the \\textbf{containment artefacts}:",
|
|
"(i) the \\textbf{gateway} with model routing, fallback chain, and per-request \\textbf{cost observability} -- cost per request, per feature, reported on a dashboard and enforced as a CI budget",
|
|
"(ii) the \\textbf{ontology guard} -- every extracted entity resolves against the deterministic data store, every cited passage exists, portfolio axioms hold",
|
|
"(iii) the \\textbf{eval harness} (Lecture 12 listing) wired as a \\textbf{CI gate}",
|
|
"(iv) one \\textbf{ADR} that justifies your chosen orchestration topology against the task-signature table, with its \\textbf{token budget and eval threshold} as the measurement contract",
|
|
"\\emph{Sub-agents propose; your deterministic services decide and book.}"
|
|
],
|
|
"elements": [
|
|
"projektbox from lines 714-716, items (i)-(iv) as an enumerate inside the box"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "This is the grading rubric of Axis B in the students' own words -- it feeds the exercise frame (frame 38) and the week-14 defence."
|
|
},
|
|
{
|
|
"no": 29,
|
|
"deck_section": "Synthesis -- One Theory, Five Parts",
|
|
"title": "What did AI change? Two temptations, one position",
|
|
"kind": "content",
|
|
"script_ref": "§45 intro (part5_ai_dimension.tex 721-724)",
|
|
"content": [
|
|
"\\emph{\\textcolor{bankblue}{What, in the end, did AI change?}} -- the bet of the opening can now be settled",
|
|
"Architecture teaching in the AI era faces two symmetric temptations:",
|
|
"\\textbf{Denial} -- generative AI as a passing tool fashion that leaves software engineering untouched: falsified by the adoption data alone (\\textbf{90\\,\\%} of practitioners report using AI at work by 2025)",
|
|
"\\textbf{Exceptionalism} -- AI systems as a new discipline with new vocabulary, new roles, new decision logic: rejected on the assembled evidence -- nothing AI does, on either axis, required a decision no ADR can record, a correctness no response measure can capture, or a structure no tactic vocabulary describes",
|
|
"Between the temptations lies the position defended since Part I: \\textbf{absorption}"
|
|
],
|
|
"elements": [
|
|
"Two columns (Denial | Exceptionalism) with the refutation under each; 'absorption' centred as a keypoint line"
|
|
],
|
|
"minutes": 3
|
|
},
|
|
{
|
|
"no": 30,
|
|
"deck_section": "Synthesis -- One Theory, Five Parts",
|
|
"title": "The script read backwards as a single argument (1/2): Parts I--III",
|
|
"kind": "content",
|
|
"script_ref": "§45 (part5_ai_dimension.tex 726)",
|
|
"content": [
|
|
"\\textbf{Part I} established that architecture is the set of significant, hard-to-reverse decisions (\\textbf{A1}), that everything is a trade-off (\\textbf{A2}), that quality attributes -- not features -- drive structure (\\textbf{A3}), and that requirements decide anything only as measurable scenarios (\\textbf{A4}); it fixed the \\textbf{twelve dimensions} on which all later judgements run",
|
|
"\\textbf{Part II} turned seven patterns into \\textbf{capability profiles} by explaining every rating through the tactics a pattern bundles or impedes (the consolidated capability table)",
|
|
"\\textbf{Part III} turned ten application classes into \\textbf{requirements profiles} -- recurring bundles of architecturally significant requirements with response measures and hard constraints (the requirements table)"
|
|
],
|
|
"elements": [
|
|
"Three-row table or three stacked blocks (Part | What it established | Artefact): I | A1-A4 | twelve dimensions; II | tactics $\\to$ ratings | capability table; III | ASR bundles + response measures + K | requirements table -- from line 726"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Use the deck-1 'pipeline + map of the script' vocabulary; each part's table named exactly as the script does."
|
|
},
|
|
{
|
|
"no": 31,
|
|
"deck_section": "Synthesis -- One Theory, Five Parts",
|
|
"title": "The script read backwards as a single argument (2/2): Parts IV--V",
|
|
"kind": "content",
|
|
"script_ref": "§45 (part5_ai_dimension.tex 726)",
|
|
"content": [
|
|
"\\textbf{Part IV} matched them: knock-out screening, veto rule, holistic ordinal reading (the $7 \\times 10$ fit matrix); it insisted that \\textbf{hybrids are the normal case}, that the decision procedure ends in an \\textbf{ADR}, and that every ADR ends in a \\textbf{measurement contract} -- because a decision is a hypothesis tested over the life cycle (\\textbf{A5})",
|
|
"\\textbf{Part V} subjected the whole construction to its hardest contemporary stress test -- and the construction held (\\textbf{A6})",
|
|
"\\textbf{Axis A} raised the value of the theory's artefacts -- documentation as agent context, fitness functions as operating licence -- rather than obsoleting them",
|
|
"\\textbf{Axis B} was absorbed as one class, one dimension, five cell shifts, and one composition pattern whose profile the theory's own grid can express"
|
|
],
|
|
"elements": [
|
|
"Continuation of the Part table (IV | three-stage match, hybrids, ADR, measurement contract, A5 | fit matrix; V | stress test held, A6 | Axis A / Axis B outcomes) -- from line 726"
|
|
],
|
|
"minutes": 3
|
|
},
|
|
{
|
|
"no": 32,
|
|
"deck_section": "Synthesis -- One Theory, Five Parts",
|
|
"title": "Maxim 8 -- the pipeline of this script in one sentence",
|
|
"kind": "keyconcept",
|
|
"script_ref": "§45 keypoint (part5_ai_dimension.tex 728-730)",
|
|
"content": [
|
|
"\\textbf{Key Concept -- Maxim 8.} The theory absorbs AI: a tenth application class, a twelfth dimension, shifted cells, one emergent composition pattern -- \\emph{same assumptions, same procedure, same contract}",
|
|
"The pipeline of this script is one sentence long:",
|
|
"\\emph{scenarios with numbers (Part I) meet capability profiles (Part II) and requirements profiles (Part III) in a non-compensatory match (Part IV) whose result is an ADR with a measurement contract -- and nothing about AI, on either axis, changes a single step of it (Part V)}"
|
|
],
|
|
"elements": [
|
|
"keypoint box from lines 728-730, the one-sentence pipeline set in italics on its own; optionally a five-box horizontal tikz strip (Part I $\\to$ II $+$ III $\\to$ IV $\\to$ ADR + contract, with Part V as a bracket underneath) redrawn from the sentence"
|
|
],
|
|
"minutes": 2,
|
|
"notes": "This sentence returns on the exam-orientation frame 37 as the students' map -- say so."
|
|
},
|
|
{
|
|
"no": 33,
|
|
"deck_section": "Synthesis -- One Theory, Five Parts",
|
|
"title": "The deepest result: one discipline at two binding sites",
|
|
"kind": "content",
|
|
"script_ref": "§45 (part5_ai_dimension.tex 732)",
|
|
"content": [
|
|
"One symmetry deserves to be made explicit -- the deepest result of Part V:",
|
|
"\\textbf{Axis A} concluded: \\emph{generation} is cheap and \\emph{verification} is the bottleneck, so the SDLC must be rebuilt around machine-checkable specifications, guardrails, and human accountability",
|
|
"\\textbf{Axis B} concluded: \\emph{model output} is cheap and \\emph{validation} is the bottleneck, so the runtime must be rebuilt around contracts, ontology guards, eval harnesses, and a human-owned determinism boundary",
|
|
"These are the \\textbf{same conclusion at two different binding sites}: whether the LLM sits in the toolchain or in the product, the discipline it demands is \\textbf{containing cheap, fallible generation behind expensive, explicit verification}",
|
|
"-- which is, on reflection, what this module has called \\emph{architecture} all along"
|
|
],
|
|
"elements": [
|
|
"Two columns (Axis A | Axis B) with 'cheap ... / bottleneck ... / rebuilt around ...' aligned line by line; the shared conclusion as a keypoint-styled line spanning both columns"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Axis A's line is Maxim 7 / the verification bottleneck of Lecture 12, recalled in recap bullet 2 (frame 3) -- point back to it; the two 'cheap ... / bottleneck ...' lines are the script's wording (line 732), not a paraphrase."
|
|
},
|
|
{
|
|
"no": 34,
|
|
"deck_section": "Synthesis -- One Theory, Five Parts",
|
|
"title": "Discussion: is it one discipline?",
|
|
"kind": "discussion",
|
|
"script_ref": "§45 thinkbox (part5_ai_dimension.tex 734-736)",
|
|
"content": [
|
|
"Close the module with the symmetry. Consider the artefact pairs across the two axes:",
|
|
"\\texttt{AGENTS.md} vs.\\ the system prompt and ontology $\\cdot$ the CI fitness function vs.\\ the runtime ontology guard $\\cdot$ the code-review obligation vs.\\ the human-oversight duty of the AI Act $\\cdot$ the test suite as the agent's operating licence vs.\\ the eval harness as the model's operating licence",
|
|
"For each pair: is this genuinely \\emph{one} engineering discipline observed at two binding sites -- or are there differences of kind, in \\textbf{reversibility}, in \\textbf{accountability}, in \\textbf{failure semantics}, that the symmetry hides?",
|
|
"And if it is one discipline: which parts of this script would survive the next order-of-magnitude improvement in model capability -- and which would you expect to rewrite?"
|
|
],
|
|
"elements": [
|
|
"thinkbox 'Discussion' from lines 734-736; the four artefact pairs as a 4-row two-column mini-table (Axis A artefact | Axis B artefact) inside or above the box"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "Run as a 4-minute plenary; the second question is the natural hand-over to the week-14 defence ('reflect on where AI helped and where it hurt')."
|
|
},
|
|
{
|
|
"no": 35,
|
|
"deck_section": "Exam Orientation",
|
|
"title": "The written examination -- the facts",
|
|
"kind": "content",
|
|
"script_ref": "Deck 1, 'Assessment' frame (AISE502_Vorlesung_1_Folien.tex 179-203)",
|
|
"content": [
|
|
"\\textbf{Written examination: 50\\,\\%} of the module grade (the project is the other 50\\,\\%)",
|
|
"\\textbf{End of semester, 60 minutes}",
|
|
"\\textbf{Open book:} script and own notes, \\emph{on paper}",
|
|
"\\textbf{Closed internet}",
|
|
"\\textbf{Focus: architecture reasoning -- profiles, matching, trade-offs}",
|
|
"Language: all materials, the script, and the exam in English"
|
|
],
|
|
"elements": [
|
|
"Reuse the deck-1 'Written examination (50\\,\\%)' block verbatim (right column of the Assessment frame, lines 193-200), full width"
|
|
],
|
|
"minutes": 2,
|
|
"notes": "Only deck-1 facts; the exam date is not stated in deck 1 -- announce it verbally (open_issues)."
|
|
},
|
|
{
|
|
"no": 36,
|
|
"deck_section": "Exam Orientation",
|
|
"title": "Six learning objectives, five parts",
|
|
"kind": "table",
|
|
"script_ref": "Deck 1 'Learning objectives' frame (AISE502_Vorlesung_1_Folien.tex 167-177) mapped onto §45 synthesis (part5_ai_dimension.tex 726-730)",
|
|
"content": [
|
|
"1. explain why architecture selection is a \\textbf{matching problem} -- no pattern is good or bad in itself | Part I: A2 everything is a trade-off, A3 quality attributes drive structure; Part II: every rating explained through the tactics a pattern bundles or impedes",
|
|
"2. construct a \\textbf{requirements profile} R(a): scenarios with response measures, utility tree, weights | Part I: A4 requirements decide only as measurable scenarios; Part III: ten classes as ASR bundles with response measures and hard constraints K(a)",
|
|
"3. derive the \\textbf{capability profile} C(p) of seven patterns from their tactics | Part II: capability profiles, every rating explained by tactics (the consolidated capability table)",
|
|
"4. run the \\textbf{three-stage, non-compensatory match} and defend the result | Part IV: knock-out screening, veto rule, holistic ordinal reading; hybrids are the normal case",
|
|
"5. record decisions as \\textbf{ADRs} and attach a \\textbf{measurement contract} | Part IV: the procedure ends in an ADR, every ADR in a measurement contract -- A5, a decision is a hypothesis tested over the life cycle",
|
|
"6. engineer \\textbf{AI components} into a system (Axis B) and use \\textbf{AI tools} in the process (Axis A) with the same discipline | Part V: A6 held -- one class, one dimension, five cell shifts, one composition pattern; Axis A raised the value of the artefacts"
|
|
],
|
|
"elements": [
|
|
"6-row \\scriptsize table, \\renewcommand{\\arraystretch}{0.85}, two columns 'Learning objective (Lecture 1)' | 'Where the script settles it', p{5.4cm}p{7.2cm}, each cell $\\le$ 2 lines; sources: deck 1 lines 167-177 (objectives verbatim) and §45 line 726 (settlement wording)"
|
|
],
|
|
"minutes": 4,
|
|
"notes": "No exam questions are invented; the right column is the §45 wording applied to each objective. The focus line is on frame 35, not repeated here."
|
|
},
|
|
{
|
|
"no": 37,
|
|
"deck_section": "Exam Orientation",
|
|
"title": "Open book: the map and the four tables",
|
|
"kind": "content",
|
|
"script_ref": "§45 keypoint (part5_ai_dimension.tex 728-730) and line 726; Part V additions from §43-44 tables; §43.3 keypoint (547-549); deck 1 assessment facts (179-203)",
|
|
"content": [
|
|
"Your map is Maxim 8's one sentence: scenarios with numbers $\\to$ capability profiles $+$ requirements profiles $\\to$ non-compensatory match $\\to$ ADR with measurement contract -- and AI changes no step of it",
|
|
"The four tables the argument runs on: the \\textbf{twelve dimensions} (Part I) $\\cdot$ the \\textbf{consolidated capability table} (Part II) $\\cdot$ the \\textbf{requirements table} of the ten classes (Part III) $\\cdot$ the \\textbf{$7 \\times 10$ fit matrix} (Part IV)",
|
|
"Part V's additions: the \\textbf{D12 row with rationales}, the \\textbf{agent capability-profile sketch}, the \\textbf{task-signature table}, the \\textbf{OWASP table}",
|
|
"The spine of the argument: the six load-bearing assumptions \\textbf{A1--A6} -- and the Maxims, up to Maxim 8",
|
|
"Focus (Lecture 1): \\textbf{architecture reasoning -- profiles, matching, trade-offs}; in the words of the §43 keypoint: \\emph{name, for any cell movement, the quality-attribute mechanism behind it}"
|
|
],
|
|
"elements": [
|
|
"Two columns: left = the one-sentence map as a small five-box strip (reuse frame 32 strip); right = the list of tables and the A1-A6 / Maxims spine; last bullet full width"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "No study advice beyond the deck-1 focus line and the §43 keypoint quotation; add nothing procedural about the exam."
|
|
},
|
|
{
|
|
"no": 38,
|
|
"deck_section": "Closing",
|
|
"title": "This week's exercise: M5 closes -- eval harness in CI, guard, cost observability; threat model, hardening, distinction work",
|
|
"kind": "exercise",
|
|
"script_ref": "Exercise sheet M5 taskbox week 13 and hintbox (project_exercise/project_exercise.tex 432-442, 451-464; week table lines 385-386); semester plan row 13 (Semesterplan_AISE502_HS26.md line 24: 'Hardening; Kür (autonomes Planning, Self-Repair, Model-Routing)')",
|
|
"content": [
|
|
"\\textbf{Project Link (projektbox):} week 13 of M5 -- Multi-Agent Orchestration, Evaluation, and Hardening; coaching session (1 lesson):",
|
|
"1. \\textbf{Evaluation harness as a CI gate}; report accuracy and failure modes",
|
|
"2. \\textbf{Observability} of token cost and latency per request",
|
|
"3. \\textbf{Threat model} incl.\\ prompt injection via news (today's §42.6: the OWASP table, frames 4-5; the same-channel rule, frame 6) $+$ \\textbf{basic hardening}",
|
|
"4. \\textbf{Scaling/optimisation}: caching, batching",
|
|
"\\textbf{Optional distinction} (semester plan: \\emph{K\\\"ur}): autonomous planning, self-repair, model routing, CI/CD, tracing -- every topology escalation justified against the task-signature table (frame 24) and priced per the default rule (frame 25)",
|
|
"\\textbf{Milestone M5: eval harness in CI $+$ guard $+$ cost observability.} Keep the deterministic core free of LLM calls -- this is the line that is graded"
|
|
],
|
|
"elements": [
|
|
"projektbox with an enumerate of the four week-13 tasks in the exercise-sheet wording (M5 taskbox, project_exercise.tex 432-442: week-13 item, threat-model item, distinction item), the distinction line as a separate bold-headed line, the milestone in bold, the hintbox line (project_exercise.tex 451-464); pointers to frames 4-6, 24 and 25 instead of restated rules"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "The threat model is the direct project application of §42.6 taught on frames 4-6 of this deck -- say so, and hand the OWASP table over as the checklist. The distinction work is the semester plan's 'Kür' (row 13) and the exercise sheet's 'Optional Distinction work' -- name both so the two documents are visibly the same list. The milestone line is unchanged ('eval harness in CI + guard + cost observability'). The week-14 announcement is the Next-week frame's job -- not repeated here."
|
|
},
|
|
{
|
|
"no": 39,
|
|
"deck_section": "Closing",
|
|
"title": "Summary",
|
|
"kind": "summary",
|
|
"script_ref": "§42.6-42.7, §43, §44, §45 (part5_ai_dimension.tex 455-736)",
|
|
"content": [
|
|
"1. \\textbf{OWASP LLM Top 10}: prompt injection cannot be solved in the model (same channel) -- defence in depth at the system level; \\emph{the architecture, not the model, is the trust boundary}",
|
|
"2. \\textbf{EU AI Act}: quality attributes with legal force enter R(a) as hard constraints K(a), never weights -- compliance is the same architecture, documented",
|
|
"3. \\textbf{C10 row and five shifts}: boundary/port, queue and capped cells; EDA/PF $\\uparrow$, HX $\\uparrow$ most, MS $\\downarrow$ in synchronous chains, SL conditional, cost load-bearing -- the matrix is \\emph{shifted}, not rewritten; MLOps level = how much of D9/D12 a team can cash in",
|
|
"4. \\textbf{Agent orchestration} is a composition pattern, not a style: agent $=$ loop $+$ tools $+$ state; workflows vs.\\ agents; topologies map onto PF, mediator EDA, broker, control loop",
|
|
"5. \\textbf{Economics and profile}: 90.2\\,\\% at $\\sim$15$\\times$ tokens is a CBAM decision; $--$ on D3/D4/D9/D10 makes it an edge pattern -- workflows before agents, every escalation an ADR with token budget and eval threshold",
|
|
"6. \\textbf{Maxim 8}: the theory absorbs AI -- same assumptions, same procedure, same contract; one discipline at two binding sites: containing cheap, fallible generation behind expensive, explicit verification"
|
|
],
|
|
"elements": [
|
|
"\\footnotesize enumerate, \\itemsep 2pt, as in deck 6; six points of at most two lines each"
|
|
],
|
|
"minutes": 3,
|
|
"notes": "Cut from eight to six points; the parenthetical numbers (98 %, >2x, 4x, 80 %) are on the frames, not in the summary."
|
|
},
|
|
{
|
|
"no": 40,
|
|
"deck_section": "Closing",
|
|
"title": "Next week",
|
|
"kind": "nextweek",
|
|
"script_ref": "Semester plan rows 13-14 (Semesterplan_AISE502_HS26.md lines 6, 24-25); deck 1 'semester at a glance' row 14 (line 215) and assessment frame (179-203); exercise sheet M6 taskbox (project_exercise/project_exercise.tex 444-449)",
|
|
"content": [
|
|
"Left column -- \\textbf{Week 14}: one lesson synthesis and exam hints; three lessons final presentations, architecture defence and peer reviews (deliverable A3, milestone M6); then the written examination (open book, 60 min)",
|
|
"The week-14 synthesis lesson is a \\textbf{recap of today's synthesis and exam orientation (frames 29-37) plus Q\\&A} -- \\emph{no new material}; today's frames are the reference",
|
|
"Present the system and \\textbf{defend your architectural trade-offs}; reflect on where AI helped and where it hurt -- in building (A) and in the system (B)",
|
|
"Right column -- \\textbf{Reading}: this week: Part V, §42.6--42.7, 43--45; ahead: \\textbf{the whole script, Parts I--V} -- with the four tables and Maxim 8's one sentence as the map",
|
|
"\\textbf{Exercise / deliverable}: M5 closes this week (eval harness in CI $+$ guard $+$ cost observability; threat model $+$ hardening; optional distinction work); \\textbf{A3: final presentation with architecture defence, week 14}"
|
|
],
|
|
"elements": [
|
|
"Two-column layout as in deck 6 (0.55 / 0.42): left = week 14 (structure, the recap-plus-Q\\&A line, the defence) and the exam; right = Reading + Exercise/deliverable"
|
|
],
|
|
"minutes": 1,
|
|
"notes": "Wording of week 14 follows the semester plan ('1 L Synthese + 3 L Präsentationen', lines 6 and 25), deck 1 row 14 ('Synthesis, exam preparation | Presentations + defence | A3') and the exercise sheet's week table (lines 385-386) -- not 'no lecture'. The synthesis lesson of week 14 is announced explicitly as a recap of frames 29-37 plus Q&A so that nobody expects new material there; the task brief's 'Week 14 -- no lecture' constraint must be reconciled with the semester plan by the lecturer before typesetting (open_issues)."
|
|
},
|
|
{
|
|
"no": 41,
|
|
"deck_section": "Closing",
|
|
"title": "Closing page",
|
|
"kind": "content",
|
|
"script_ref": "Deck 6 closing page (theme)",
|
|
"content": [
|
|
"\\FHGRClosingPage as in deck 6: 'Thank you!' -- Dr.\\ Florian Herzog, Fachhochschule Graub\\\"unden, Chur -- AISE502 -- AI in Software Engineering II"
|
|
],
|
|
"elements": [
|
|
"FHGR closing page (theme)"
|
|
],
|
|
"minutes": 0,
|
|
"notes": "Last lecture of new material -- the lecturer may wish to add a one-line farewell; keep the theme page otherwise unchanged."
|
|
}
|
|
],
|
|
"exercise_frame": {
|
|
"title": "This week's exercise: M5 closes -- eval harness in CI, guard, cost observability; threat model, hardening, distinction work",
|
|
"content": [
|
|
"Week 13 of M5 (Multi-Agent Orchestration, Evaluation, and Hardening); coaching session (1 lesson)",
|
|
"Evaluation harness as a CI gate; report accuracy and failure modes",
|
|
"Observability of token cost and latency per request",
|
|
"Threat model incl. prompt injection via news (today's §42.6: the OWASP table, frames 4-5; the same-channel rule, frame 6) + basic hardening",
|
|
"Scaling/optimisation: caching, batching",
|
|
"Optional distinction work (semester plan: Kür): autonomous planning, self-repair, model routing, CI/CD, tracing -- topology escalations justified against the task-signature table (frame 24) and priced per the default rule (frame 25)",
|
|
"Milestone M5: eval harness in CI + guard + cost observability; keep the deterministic core free of LLM calls -- the line that is graded"
|
|
]
|
|
},
|
|
"summary": [
|
|
"OWASP LLM Top 10: prompt injection cannot be solved in the model (same channel) -- defence in depth at the system level; the architecture, not the model, is the trust boundary",
|
|
"EU AI Act (Regulation 2024/1689): quality attributes with legal force enter R(a) as hard constraints K(a), never weights -- compliance is the same architecture, documented",
|
|
"C10 row and five shifts: boundary/port, queue and capped cells; EDA/PF gain, HX gains most, MS loses in synchronous chains, SL conditional, cost load-bearing everywhere -- the matrix is shifted, not rewritten; MLOps level = how much of D9/D12 a team can cash in",
|
|
"Agent orchestration is a composition pattern, not a style: agent = loop + tools + state; workflows (predefined code paths) vs. agents (model steers its own process); topologies map onto PF, mediator EDA, broker, feedback control loop",
|
|
"Economics and profile: 90.2 % better at ~15x tokens is a CBAM decision; $--$ on D3/D4/D9/D10 makes it an edge pattern behind the ports of a deterministic core -- workflows before agents, every escalation an ADR with token budget and eval threshold (Maxim 6)",
|
|
"Maxim 8: the theory absorbs AI -- tenth class, twelfth dimension, shifted cells, one composition pattern -- same assumptions, same procedure, same contract; Axis A and Axis B are one discipline at two binding sites: containing cheap, fallible generation behind expensive, explicit verification"
|
|
],
|
|
"next_week": {
|
|
"lecture_line": "Week 14: one lesson synthesis and exam hints -- a recap of today's synthesis and exam orientation (frames 29-37) plus Q&A, no new material; three lessons final presentations, architecture defence and peer reviews (deliverable A3, milestone M6); then the written examination (open book, 60 min)",
|
|
"topics": [
|
|
"Final presentations: present the system and defend your architectural trade-offs (M6)",
|
|
"Peer reviews",
|
|
"Reflect on where AI helped and where it hurt -- in building (Axis A) and in the system (Axis B)",
|
|
"Written examination: end of semester, 60 minutes, open book (script and own notes on paper), closed internet; focus: architecture reasoning -- profiles, matching, trade-offs"
|
|
],
|
|
"reading": [
|
|
"this week: Part V, Sections 42.6--42.7, 43--45",
|
|
"ahead: the whole script, Parts I--V -- with the four tables (twelve dimensions, capability table, requirements table, fit matrix) and Maxim 8's one-sentence pipeline as the map"
|
|
],
|
|
"exercise": [
|
|
"M5 closes this week: eval harness in CI + guard + cost observability; threat model incl. prompt injection via news + basic hardening; optional distinction work (Kür)",
|
|
"A3: final presentation with architecture defence, week 14"
|
|
]
|
|
},
|
|
"script_boxes_used": [
|
|
{
|
|
"box": "Table tab:owasp -- OWASP Top 10 for LLM Applications 2025 with architectural counter-measures",
|
|
"location": "§42.6, part5_ai_dimension.tex 460-481",
|
|
"used_in_frame": "Frames 4 and 5 (split LLM01-05 / LLM06-10)"
|
|
},
|
|
{
|
|
"box": "hinweisbox -- prompt injection not solvable inside the model; defence in depth; project consequence",
|
|
"location": "§42.6, part5_ai_dimension.tex 483-485",
|
|
"used_in_frame": "Frame 6"
|
|
},
|
|
{
|
|
"box": "Table tab:d12row -- the D12 row of the capability table with tactic-level rationales",
|
|
"location": "§43.2, part5_ai_dimension.tex 514-532",
|
|
"used_in_frame": "Frame 11"
|
|
},
|
|
{
|
|
"box": "keypoint -- the matrix is shifted, not rewritten (five directions, five reasons)",
|
|
"location": "§43.3, part5_ai_dimension.tex 547-549",
|
|
"used_in_frame": "Frame 13 (quoted again on frame 37)"
|
|
},
|
|
{
|
|
"box": "keypoint -- Assumption A6 restated as a falsifiable claim",
|
|
"location": "§43.4, part5_ai_dimension.tex 558-560",
|
|
"used_in_frame": "Frame 16"
|
|
},
|
|
{
|
|
"box": "definitionbox[Agent; workflow vs. agent]",
|
|
"location": "§44.2, part5_ai_dimension.tex 580-582",
|
|
"used_in_frame": "Frame 20"
|
|
},
|
|
{
|
|
"box": "Table tab:agenttopology -- topologies mapped to classical patterns",
|
|
"location": "§44.3, part5_ai_dimension.tex 591-608",
|
|
"used_in_frame": "Frame 21"
|
|
},
|
|
{
|
|
"box": "Figure fig:agenttopologies -- tikz: four topologies and their classical analogues",
|
|
"location": "§44.3, part5_ai_dimension.tex 610-651",
|
|
"used_in_frame": "Frame 22 (redrawn with deck styles)"
|
|
},
|
|
{
|
|
"box": "Table tab:agentchoice -- choosing a topology from the task signature",
|
|
"location": "§44.3, part5_ai_dimension.tex 655-671",
|
|
"used_in_frame": "Frame 24"
|
|
},
|
|
{
|
|
"box": "keypoint -- the default rule for agent architecture (Maxim 6 applies)",
|
|
"location": "§44.4, part5_ai_dimension.tex 678-680",
|
|
"used_in_frame": "Frame 25"
|
|
},
|
|
{
|
|
"box": "Table tab:agentprofile -- capability-profile sketch of agent orchestration",
|
|
"location": "§44.5, part5_ai_dimension.tex 687-710",
|
|
"used_in_frame": "Frame 26 (one 12-row table); reading on frame 27"
|
|
},
|
|
{
|
|
"box": "projektbox -- Axis B is what you build; the four graded containment artefacts",
|
|
"location": "§44.5, part5_ai_dimension.tex 714-716",
|
|
"used_in_frame": "Frame 28 (pointer on frame 24; items referenced on the exercise frame 38)"
|
|
},
|
|
{
|
|
"box": "keypoint -- Maxim 8, the pipeline in one sentence",
|
|
"location": "§45, part5_ai_dimension.tex 728-730",
|
|
"used_in_frame": "Frame 32 (reused as the map on frame 37)"
|
|
},
|
|
{
|
|
"box": "thinkbox -- artefact pairs across the two axes; what survives the next model generation",
|
|
"location": "§45, part5_ai_dimension.tex 734-736",
|
|
"used_in_frame": "Frame 34 (Discussion)"
|
|
}
|
|
],
|
|
"script_boxes_dropped": [],
|
|
"open_issues": [
|
|
"Week 14 wording: the task constraint says the Next-week frame must announce 'Week 14 -- no lecture'. Three reference documents contradict this (semester plan row 14 and phase logic, lines 6 and 25: '1 L Synthese + 3 L Präsentationen'; deck 1 'semester at a glance' row 14: 'Synthesis, exam preparation | Presentations + defence | A3'; exercise sheet week table, lines 385-386). The plan follows the reference documents on frames 3 and 40 ('one lesson synthesis and exam hints; three lessons presentations, defence, peer reviews -- A3, M6') and keeps the full synthesis and exam orientation in this deck (frames 29-37, 27 minutes); frame 40 announces the week-14 synthesis lesson as a recap of frames 29-37 plus Q&A, not new material. The lecturer must reconcile the task brief's 'no lecture' constraint with the semester plan before typesetting; if 'no lecture' is in fact intended, only the week-14 line on frames 3 and 40 and next_week.lecture_line change -- the deck's content does not.",
|
|
"§43.1 names the capped cells of the C10 row (L, MS, SL) only as 'capped'; frame 10 prints their Fit ratings from tab:fit-c10 (part4_fit.tex 439-445: L $-$, MS $\\circ$, SL $\\circ$), taught in Lecture 10 but outside the assigned passage, so that the Fit column follows the rating convention of decks 4-6.",
|
|
"The reference architecture figure fig:llmgateway (part5_ai_dimension.tex line 384, taught in Lecture 12) is named on frames 5, 8, 18-19 and 28. The plan only names it; a miniature reprint of the deck-12 figure on frame 8 is optional if space allows.",
|
|
"§44.1 (advisor workflow) has no figure in the script; the diagram on frame 18 is drawn from the prose of line 573 only (orchestrator, three specialists, gateway, guard, deterministic analytics services). Label nothing the prose does not name.",
|
|
"§42.6 has no italic leading question; frame 4 opens with the section's declarative first sentence verbatim (line 458). §43, §44 and §45 have their own leading questions (lines 499, 567, 723) and use them verbatim.",
|
|
"No thinkbox in §42.6-44 and no ailinse box anywhere in the assigned passages: the deck has exactly one Discussion frame (§45) and no AI Lens frames -- consistent with the whole lecture being Axis B.",
|
|
"Density: the eighth capability profile (frame 26) is one 12-row scriptsize table with grounds condensed to $\\le$ 12 words -- the typesetter must keep each ground to one line; frame 36 is a two-column scriptsize table with cells of at most two lines; frame 20 (definitionbox + three one-liners) must keep the definitionbox to four lines.",
|
|
"Exam orientation uses only deck-1 facts and §45 (plus the §43 keypoint quotation on frame 37); deck 1 states 'end of semester' without a date -- the exam date must be announced verbally.",
|
|
"The recap is built from the task's description of deck 12, the Lecture-12 plan's summary (L12.v1.json) and semester-plan row 12 (decks 7-12 do not exist as files in Folien/); recap bullet 2 now carries the verification bottleneck / Maxim 7 and the Axis-A compact block in the script's wording (part5_ai_dimension.tex 142-154, 234) because frame 33 and Maxim 8 build on Maxim 7. At \\footnotesize the bullet must stay within four lines -- the typesetter drops 'two contradictory' and the section-number parenthetical first, never the Maxim-7 clause. The wording 'three component types' is kept generic because only 'type-(b)' appears in the assigned passage (§43.4).",
|
|
"Frame 21 uses the script's analogue wording ('Routing layer / mediator', 'Broker topology'); deck 6's outlook table said 'dispatch layer' and 'broker fan-out of autonomous quanta' -- the two renamed cells are to be mentioned verbally, not corrected in deck 6.",
|
|
"AI Act timetable: the 2 Aug 2026 general-applicability date has passed by the time of this lecture; the script text is unchanged, but the lecturer may want to say 'now applicable' on frame 7. 'Minimal risk' is printed without a gloss because the script gives none.",
|
|
"Minutes: frames 2-37 (agenda, recap, content) sum to 116; exercise 3, summary 3, next-week 1 on top -- total 123 for the 135-minute slot, leaving about 12 minutes for the two lesson breaks and questions on the last lecture of new material.",
|
|
"Exercise sheet location: project_exercise.tex lives at AISE502/project_exercise/project_exercise.tex, not under skript/chapters; the cited line numbers (M5 taskbox 432-442, M6 444-449, hintbox 451-464, week table rows 13-14 at lines 385-386) refer to that file. Frame 38 lists all three week-13 items of the M5 taskbox (harness + observability; threat model + hardening + scaling; optional distinction) and names the semester plan's 'Kür' -- the milestone line is unchanged."
|
|
],
|
|
"total_frames": 41
|
|
}
|