60 lines
7.0 KiB
HTML
60 lines
7.0 KiB
HTML
<!-- ==========================================================================
|
||
AISE502 · Moodle course page · Week 12
|
||
Section title (Edit section → Section name): Week 12: The AI Dimension I – Axis A Evidence, Axis B Foundations
|
||
Three blocks follow, matching the FHGR section template. Paste each block into the
|
||
corresponding 'Text and media area' (editor → source code view), or all three into one.
|
||
The slide set of lecture 12 is added by the lecturer as a file resource below block 2.
|
||
Generated by src/build_moodle.py from src/content/week_12.json – edit the JSON, not this file.
|
||
========================================================================== -->
|
||
|
||
<!-- BLOCK 1 · Learning objectives -->
|
||
<h4>🎯 Learning objectives</h4>
|
||
<ul>
|
||
<li>I can explain assumption A6 – two axes, one method – and distinguish Axis A (tool) from Axis B (component).</li>
|
||
<li>I can reconcile the Copilot RCT (+55.8 %) and the METR RCT (−19 %) with the five moderator variables.</li>
|
||
<li>I know the system-level record – DORA's amplifier finding, GitClear's duplication and churn trend – and the METR perception gap.</li>
|
||
<li>I can explain the verification bottleneck and derive Maxim 7 from it: AI raises the value of architecture.</li>
|
||
<li>I can explain how ADRs, agent instruction files and fitness functions become the control interface and operating licence for agents.</li>
|
||
<li>I know that patterns are examinable and products expire, that MCP is ports-and-adapters, and who answers for merged code.</li>
|
||
<li>I can name the five failures of the synchronously wired sentiment call and their dimensions D3, D10, D12, D7, D6.</li>
|
||
<li>I can distinguish the three AI component types and explain why adding AI is a per-component matching problem.</li>
|
||
<li>I can explain why CACE demands containment and justify each element of the reference architecture (port, gateway, queue, guard).</li>
|
||
<li>I can define the eval harness, name its four evaluation strategies, and place its thresholds in the measurement contract.</li>
|
||
</ul>
|
||
|
||
<!-- BLOCK 2 · Theory -->
|
||
<h4>🧑🏫 Theory</h4>
|
||
<p><strong>Lecture 12: The AI Dimension I – Axis A Evidence, Axis B Foundations</strong> · 3 lessons lecture + 1 lesson standup/coaching<br>
|
||
Part V opens as Assumption A6 falls due: the Axis A evidence and its resolution, then the Axis B foundations from the wrongly wired sentiment call to the eval harness.</p>
|
||
<ul>
|
||
<li>Assumption A6 falls due: Axis A (AI as tool) changes how systems are built, Axis B (AI as component) what they contain – two independent axes, one method, nothing new.</li>
|
||
<li>Copilot RCT: +55.8 % on a greenfield task; METR RCT: experts 19 % slower in their own mature codebases, estimating +20 % – the perception gap is the didactic core.</li>
|
||
<li>Neither study is wrong – five moderators reconcile the record; at system level DORA finds AI an amplifier (instability persists); GitClear: duplication 4×, refactoring signature collapsed; adoption up, trust down.</li>
|
||
<li>The verification bottleneck: code generation became cheap; specification, verification and architecture became the binding constraints – AI lowers the cost of producing code, not of taking responsibility for it.</li>
|
||
<li>Maxim 7: AI raises the change rate by an order of magnitude and therefore raises, not lowers, the value of architecture; D7 and D9 gain weight in every requirements profile.</li>
|
||
<li>ADRs and agent instruction files are the control interface, fitness functions the operating licence for agents; whoever merges code answers for it; patterns are examinable, products expire; MCP is ports-and-adapters.</li>
|
||
<li>Wired synchronously, the news-sentiment call fails on latency (D3), cost (D10), non-determinism (D12), drift (D7) and injection (D6) – properties of the wiring, not bugs in the model.</li>
|
||
<li>Three component types (LLM, ML, optimisation) differ where the twelve dimensions measure: adding AI is a per-component matching problem; CACE (changing anything changes everything) means decomposing around the component.</li>
|
||
<li>Reference architecture (ADR-011 made structural): typed port → gateway (routing, semantic cache, stability patterns, cost telemetry) → queue → ontology guard; LLM agents propose, deterministic services decide and book.</li>
|
||
<li>Eval harness (versioned cases, scoring logic, statistical thresholds, executed in CI) replaces exact assertion by statistical acceptance: golden-set regression, LLM-as-judge, domain axioms, online evaluation; thresholds belong in the measurement contract.</li>
|
||
</ul>
|
||
<p>📎 <strong>Materials:</strong> Slide set of Lecture 12 <em>(added below by the lecturer)</em> · Script: Part V – Sections 40 (Two Axes, One Method), 41 (Axis A: AI as a Tool Shifts the Economics of the SDLC) and 42 (Axis B: AI as a Runtime Component). Section 42: subsections 42.1–42.5 only.</p>
|
||
|
||
<!-- BLOCK 3 · Self-study and assignments -->
|
||
<h4>🧑💻 Self-study and assignments</h4>
|
||
<p><strong>📖 Reading before the lecture:</strong> Part V – Sections 40 (Two Axes, One Method), 41 (Axis A: AI as a Tool Shifts the Economics of the SDLC) and 42 (Axis B: AI as a Runtime Component). Section 42: subsections 42.1–42.5 only.</p>
|
||
<p><strong>Also before the lecture:</strong></p>
|
||
<ul>
|
||
<li>Come to the coaching slot ready to start M5: the AdvisorAgent with 2–3 sub-agents behind the gateway (mandatory) and the ontology guard active on all insights.</li>
|
||
</ul>
|
||
<p><strong>🧩 Exercise session:</strong> One-lesson standup/coaching slot on milestone M5: teams wire the AdvisorAgent and its 2–3 sub-agents into the lecture's reference architecture – typed port → gateway → queue → ontology guard – with every LLM call through the gateway and the ontology guard active on all insights.</p>
|
||
<p><strong>🛠️ Project work this week</strong> · Milestone M5 – Multi-Agent Orchestration, Evaluation, and Hardening (weeks 12–13)</p>
|
||
<ul>
|
||
<li>Wire the AdvisorAgent to orchestrate 2–3 sub-agents through contracts – every LLM call through the gateway port (ADR-011), no provider SDK import in domain code; this is the mandatory part of the week.</li>
|
||
<li>Activate the ontology guard on all insights: entity resolution against the deterministic store (valid ticker, sector, event type), domain axioms, citation check – reject unresolvable references rather than passing them on.</li>
|
||
<li>Keep the deterministic core free of LLM calls – LLM agents propose, deterministic services decide and book; this is the line that is graded.</li>
|
||
<li>Apply the Axis A discipline: keep AGENTS.md / CLAUDE.md as current as code, let agents draft ADRs that a named team member signs, and merge agent-generated changes only through the CI gate (module-boundary fitness functions, test suite, eval harness for anything touching prompts or the gateway).</li>
|
||
<li>Put a one-page AI policy in the project handbook (permitted tools, provenance labelling, review rules); week 13 turns the eval harness into a CI gate for the M5 check.</li>
|
||
</ul>
|
||
<p><em>No deliverable is due this week.</em></p>
|