60 lines
7.0 KiB
HTML
60 lines
7.0 KiB
HTML
<!-- ==========================================================================
|
||
AISE502 · Moodle course page · Week 13
|
||
Section title (Edit section → Section name): Week 13: Threats, the Shifted Matrix, Agent Orchestration – and Synthesis
|
||
Three blocks follow, matching the FHGR section template. Paste each block into the
|
||
corresponding 'Text and media area' (editor → source code view), or all three into one.
|
||
The slide set of lecture 13 is added by the lecturer as a file resource below block 2.
|
||
Generated by src/build_moodle.py from src/content/week_13.json – edit the JSON, not this file.
|
||
========================================================================== -->
|
||
|
||
<!-- BLOCK 1 · Learning objectives -->
|
||
<h4>🎯 Learning objectives</h4>
|
||
<ul>
|
||
<li>I can pair each risk of the OWASP Top 10 for LLM Applications with its architectural counter-measure.</li>
|
||
<li>I can explain why prompt injection cannot be solved inside the model and why the architecture is the trust boundary.</li>
|
||
<li>I know the EU AI Act's four risk classes and why its obligations enter R(a) as hard constraints K(a).</li>
|
||
<li>I can derive each pattern's D12 rating from the queue, port and measurement point it provides or lacks.</li>
|
||
<li>I can name the five cell shifts AI causes in the matrix and the quality-attribute mechanism behind each.</li>
|
||
<li>I can distinguish a workflow from an agent and explain why autonomy is a dial, not a binary.</li>
|
||
<li>I can map the six workflow topologies onto their classical analogues from Part II.</li>
|
||
<li>I can choose a topology from the task signature and justify it with a token budget and eval threshold.</li>
|
||
<li>I can explain, from the profile sketch and the economics of autonomy, why agent orchestration remains an edge pattern.</li>
|
||
<li>I can state Maxim 8 and explain why Axis A and Axis B are one discipline at two binding sites.</li>
|
||
</ul>
|
||
|
||
<!-- BLOCK 2 · Theory -->
|
||
<h4>🧑🏫 Theory</h4>
|
||
<p><strong>Lecture 13: Threats, the Shifted Matrix, Agent Orchestration – and Synthesis</strong> · 3 lessons lecture + 1 lesson standup/coaching<br>
|
||
The last lecture of new material: Axis B completed (threats, regulation), the shifted matrix, agent orchestration as the eighth pattern, synthesis with Maxim 8, and exam orientation.</p>
|
||
<ul>
|
||
<li>The OWASP LLM Top 10 pairs risks with architectural counter-measures; prompt injection is not solvable in the model (same channel), so the architecture, not the model, is the trust boundary.</li>
|
||
<li>EU AI Act obligations (C10 can be high-risk) are quality attributes with legal force; they enter R(a) as hard constraints K(a), never weights: compliance is the same architecture, documented.</li>
|
||
<li>The matrix is shifted, not rewritten: patterns providing the queue, port and measurement point (D12) gain (EDA, PF; HX most); synchronous chains lose (MS); SL is conditional; cost becomes load-bearing.</li>
|
||
<li>The three-level MLOps ladder says how much of D9 and D12 a team can cash in; A6, restated as a falsifiable claim: AI stretches the dimensions, the method is unchanged.</li>
|
||
<li>Agent orchestration is a composition pattern, not a new style; agent = loop + tools + state; workflows follow predefined code paths, agents steer their own process and tool use.</li>
|
||
<li>The six topologies map onto Part II: chain (PF), routing (routing layer/mediator), parallelisation (broker-style fan-out), orchestrator–workers (mediator EDA), evaluator–optimizer (feedback control loop), multi-agent (broker topology); strengths and weaknesses transfer.</li>
|
||
<li>The task-signature table selects the simplest matching topology first; the default rule: workflows before agents, every escalation an ADR with token budget and eval threshold (Maxim 6 applies unchanged).</li>
|
||
<li>Autonomy is a CBAM (cost–benefit) decision (90.2 % better at ~15× the tokens); with −− on D3/D4/D9/D10 in its profile sketch, agent orchestration remains an edge pattern behind a deterministic core.</li>
|
||
<li>Maxim 8: the theory absorbs AI (same assumptions, same procedure, same contract); the two axes are one discipline at two binding sites: cheap, fallible generation contained behind expensive, explicit verification.</li>
|
||
<li>Exam orientation (open book, 60 minutes; focus: architecture reasoning): the map is Maxim 8's one sentence, the four tables, Part V's additions and the spine A1–A6 with the Maxims.</li>
|
||
</ul>
|
||
<p>📎 <strong>Materials:</strong> Slide set of Lecture 13 <em>(added below by the lecturer)</em> · Script: Part V – Sections 42 (Axis B: AI as a Runtime Component), 43 (How AI Shifts the Matrix), 44 (Agent Orchestration: The Emergent Eighth Pattern) and 45 (Synthesis: One Theory, Five Parts). Section 42: subsections 42.6–42.7 only.</p>
|
||
|
||
<!-- BLOCK 3 · Self-study and assignments -->
|
||
<h4>🧑💻 Self-study and assignments</h4>
|
||
<p><strong>📖 Reading before the lecture:</strong> Part V – Sections 42 (Axis B: AI as a Runtime Component), 43 (How AI Shifts the Matrix), 44 (Agent Orchestration: The Emergent Eighth Pattern) and 45 (Synthesis: One Theory, Five Parts). Section 42: subsections 42.6–42.7 only.</p>
|
||
<p><strong>Also before the lecture:</strong></p>
|
||
<ul>
|
||
<li>Bring the current state of your M5 work to the coaching session: eval harness as CI gate, cost/latency observability, hardening and any distinction work.</li>
|
||
</ul>
|
||
<p><strong>🧩 Exercise session:</strong> One lesson of standup and coaching on the closing M5 work: eval harness as a CI gate, token-cost and latency observability, threat model and hardening, caching and batching, optional distinction work; M5 is checked at the end of the week.</p>
|
||
<p><strong>🛠️ Project work this week</strong> · Milestone M5 – Multi-Agent Orchestration, Evaluation, and Hardening (weeks 12–13)</p>
|
||
<ul>
|
||
<li>Wire the evaluation harness into CI as a gate; report accuracy and failure modes.</li>
|
||
<li>Add observability of token cost and latency per request – cost per request and per feature on a dashboard, enforced as a CI budget; then scale and optimise with caching and batching.</li>
|
||
<li>Write the threat model including prompt injection via news, using the OWASP table as your checklist and the same-channel rule, and apply basic hardening: no LLM output reaches the booking path without the ontology guard, no agent tool performs an irreversible action.</li>
|
||
<li>Record the ADR that justifies your orchestration topology against the task-signature table, with its token budget and eval threshold as the measurement contract.</li>
|
||
<li>Take on optional distinction work (Kür) – autonomous planning, self-repair, model routing, CI/CD, tracing – justifying every topology escalation against the task-signature table and pricing it per the default rule; keep the deterministic core free of LLM calls – this is the line that is graded.</li>
|
||
</ul>
|
||
<div style="border-left:4px solid #1b6ec2; background:#eef5fc; padding:8px 12px; margin:10px 0;">📌 <strong>Due this week:</strong> Milestone check (end of week 13): eval harness in CI + guard + cost observability</div>
|