diff --git a/Folien/AISE502_Vorlesung_11_Folien.pdf b/Folien/AISE502_Vorlesung_11_Folien.pdf index 6203b7f..4f92352 100644 Binary files a/Folien/AISE502_Vorlesung_11_Folien.pdf and b/Folien/AISE502_Vorlesung_11_Folien.pdf differ diff --git a/Folien/AISE502_Vorlesung_11_Folien.tex b/Folien/AISE502_Vorlesung_11_Folien.tex index 8bbb9b7..2443603 100644 --- a/Folien/AISE502_Vorlesung_11_Folien.tex +++ b/Folien/AISE502_Vorlesung_11_Folien.tex @@ -548,7 +548,7 @@ Cost per request is a runtime quality attribute with \textbf{no counterpart in c \begin{projektbox} \footnotesize Your project submission must ship its \textbf{measurement contract}, not just its architecture -- the repository must contain, \emph{wired into CI}: \begin{enumerate}\setlength\itemsep{1pt} - \item module-boundary verification with \textbf{zero violations} (ArchUnit or Spring Modulith verify), including the determinism-boundary rule: no domain service reaches the LLM gateway except via its declared port + \item module-boundary verification with \textbf{zero violations} (Python: import-linter; Java: ArchUnit or Spring Modulith verify), including the determinism-boundary rule: no domain service reaches the LLM gateway except via its declared port \item an eval harness with a versioned golden set and a \textbf{pass rate $\geq 95\,\%$} gating every prompt or model change \item a \textbf{token-cost budget per request} enforced as a pipeline gate and reported per feature \item a p95 latency budget for the advisory scenario (\textbf{$< 20$\,s} end-to-end) diff --git a/Folien/AISE502_Vorlesung_12_Folien.pdf b/Folien/AISE502_Vorlesung_12_Folien.pdf index 456b4d3..d70da44 100644 Binary files a/Folien/AISE502_Vorlesung_12_Folien.pdf and b/Folien/AISE502_Vorlesung_12_Folien.pdf differ diff --git a/Folien/AISE502_Vorlesung_12_Folien.tex b/Folien/AISE502_Vorlesung_12_Folien.tex index 1e84310..61361da 100644 --- a/Folien/AISE502_Vorlesung_12_Folien.tex +++ b/Folien/AISE502_Vorlesung_12_Folien.tex @@ -849,7 +849,7 @@ def test\_judge\_is\_calibrated(judge, human\_labels):\\ \footnotesize \textbf{Coaching slot (1 lesson). Milestone M5} -- Multi-Agent Orchestration, Evaluation, and Hardening (weeks 12--13): \begin{enumerate}\setlength\itemsep{2pt} \item \textbf{Mandatory this week}: the AdvisorAgent orchestrates 2--3 sub-agents \emph{through contracts} -- every LLM call through the gateway port (ADR-011); no provider SDK import in domain code - \item \textbf{Ontology guard active on all insights}: entity resolution against the deterministic store, domain axioms, citation check -- unresolvable references are \emph{rejected}, not passed on + \item \textbf{Ontology guard active on all insights}: entity resolution against the deterministic store (valid ticker, sector, event type), domain axioms, citation check -- unresolvable references are \emph{rejected}, not passed on \item \textbf{LLM agents propose; deterministic services decide and book} -- keep the deterministic core free of LLM calls \emph{(the line that is graded)} \end{enumerate} \vspace{0.05cm} diff --git a/Folien/AISE502_Vorlesung_13_Folien.pdf b/Folien/AISE502_Vorlesung_13_Folien.pdf index 8aef9a4..1037fda 100644 Binary files a/Folien/AISE502_Vorlesung_13_Folien.pdf and b/Folien/AISE502_Vorlesung_13_Folien.pdf differ diff --git a/Folien/AISE502_Vorlesung_13_Folien.tex b/Folien/AISE502_Vorlesung_13_Folien.tex index 0ae5c83..fdb6a05 100644 --- a/Folien/AISE502_Vorlesung_13_Folien.tex +++ b/Folien/AISE502_Vorlesung_13_Folien.tex @@ -678,9 +678,9 @@ Status & \multicolumn{2}{@{}>{\raggedright\arraybackslash}p{10.1cm}@{}}{provisio \begin{frame}{Project link: Axis B is what you build -- and what you build it as} \begin{projektbox} -\footnotesize Axis B is \emph{what} you build; the advisor workflow -- an orchestrator with two to three specialised sub-agents (document analyst, portfolio quant, compliance checker), all behind the LLM gateway -- is what you build it \emph{as}. The graded Axis-B deliverables are the \textbf{containment artefacts}: +\footnotesize Axis B is \emph{what} you build; the advisor workflow -- an orchestrator with two to three specialised sub-agents (document analyst, portfolio quant, compliance checker -- in the exercise sheet: \texttt{ResearchAgent}, \texttt{RiskAgent}\,/\,\texttt{OptimizationAgent}, and the ontology guard), all behind the LLM gateway -- is what you build it \emph{as}. The graded Axis-B deliverables are the \textbf{containment artefacts}: \begin{enumerate}\setlength\itemsep{1pt} - \item[(i)] the \textbf{gateway} with model routing, fallback chain, and per-request \textbf{cost observability} -- cost per request, per feature, reported on a dashboard and enforced as a CI budget + \item[(i)] the \textbf{gateway} with fallback chain and per-request \textbf{cost observability} -- cost per request, per feature, reported on a dashboard and enforced as a CI budget (model routing itself is distinction work) \item[(ii)] the \textbf{ontology guard} -- every extracted entity resolves against the deterministic data store, every cited passage exists, portfolio axioms hold \item[(iii)] the \textbf{eval harness} (Lecture 12 listing) wired as a \textbf{CI gate} \item[(iv)] one \textbf{ADR} that justifies your chosen orchestration topology against the task-signature table, with its \textbf{token budget and eval threshold} as the measurement contract diff --git a/Folien/AISE502_Vorlesung_1_Folien.pdf b/Folien/AISE502_Vorlesung_1_Folien.pdf index f0bc8dc..f3606c1 100644 Binary files a/Folien/AISE502_Vorlesung_1_Folien.pdf and b/Folien/AISE502_Vorlesung_1_Folien.pdf differ diff --git a/Folien/AISE502_Vorlesung_1_Folien.tex b/Folien/AISE502_Vorlesung_1_Folien.tex index d01e99f..fce1751 100644 --- a/Folien/AISE502_Vorlesung_1_Folien.tex +++ b/Folien/AISE502_Vorlesung_1_Folien.tex @@ -224,7 +224,7 @@ After this module you can \dots \begin{frame}{The course project} \begin{projektbox} -You will design and build an \textbf{AI-native advisory platform}: answers generated by LLMs over a grounded knowledge base, with deterministic services for portfolio analytics around them. Everything this module teaches -- profiles, matching, ADRs, measurement -- you will apply to \emph{this} system, and defend at the end. +You will design and build an \textbf{AI-augmented portfolio intelligence platform} (exercise sheet): market prices and company news in; deterministic risk, performance, and optimisation services; an LLM component turning news into validated \texttt{Insight}s; a multi-agent advisor that explains -- API-first. Everything this module teaches -- profiles, matching, ADRs, measurement -- you will apply to \emph{this} system, and defend at the end. \end{projektbox} \vspace{0.2cm} @@ -235,7 +235,7 @@ You will design and build an \textbf{AI-native advisory platform}: answers gener \item collect \emph{raw} stakeholder wishes -- unfiltered, unweighted \end{itemize} -\vspace{0.2cm} +\vspace{0.05cm} \textcolor{codegray}{\small Next week those wishes become \emph{scenarios}; in week 3 they become your requirements profile (deliverable A1).} \end{frame} diff --git a/Folien/AISE502_Vorlesung_9_Folien.pdf b/Folien/AISE502_Vorlesung_9_Folien.pdf index 6fe4661..fd68217 100644 Binary files a/Folien/AISE502_Vorlesung_9_Folien.pdf and b/Folien/AISE502_Vorlesung_9_Folien.pdf differ diff --git a/Folien/AISE502_Vorlesung_9_Folien.tex b/Folien/AISE502_Vorlesung_9_Folien.tex index 1f29b82..b29decf 100644 --- a/Folien/AISE502_Vorlesung_9_Folien.tex +++ b/Folien/AISE502_Vorlesung_9_Folien.tex @@ -353,13 +353,11 @@ Constraints $K(a)$ & \multicolumn{2}{l}{BCBS 239 lineage (banks)} \\ \begin{frame}{Closing the loop: C6 and C7 inside your C10 project} \small Week 6's inheritance diagram: C10 inherits C6, C7 and C3 -- today the two batch parents are taught. - -\vspace{0.05cm} \begin{projektbox} \footnotesize \begin{columns}[T,totalwidth=\linewidth] \begin{column}{0.48\textwidth} -\textcolor{bankblue}{\textbf{The C6 subsystems}} -- ingestion (chunking, embedding, indexing) and the eval pipeline +\textcolor{bankblue}{\textbf{The C6 subsystems}} -- ingestion (chunking, embedding, indexing -- in the sheet: news ingestion $\to$ \texttt{ResearchAgent}) and the eval pipeline \begin{itemize}\setlength\itemsep{1pt} \item the C6 discipline transfers literally: versioned inputs, idempotent stages, deterministic reruns, golden datasets \item note 14 is the bridge: for batch pipelines, \emph{reproducibility is the operative meaning of testability} @@ -375,8 +373,6 @@ Constraints $K(a)$ & \multicolumn{2}{l}{BCBS 239 lineage (banks)} \\ \end{column} \end{columns} \end{projektbox} - -\vspace{0.05cm} \footnotesize \textbf{Key concept:} the freshness contract appears \textbf{twice} in your measurement contract -- as a pipeline fitness function (C7 discipline) and as a grounding rule for generated answers (C10 discipline). \end{frame} diff --git a/Semesterplan_AISE502_HS26.md b/Semesterplan_AISE502_HS26.md index d26b908..1a275a3 100644 --- a/Semesterplan_AISE502_HS26.md +++ b/Semesterplan_AISE502_HS26.md @@ -32,17 +32,15 @@ 4. **Nichts Blockierendes zu spät:** C1–C9 (Prüfungsstoff, aber nicht design-relevant) sowie Achse A/B kommen in den Wochen 8–13 — Gateway- und Eval-Harness-Grundlagen (V Woche 12) genau dann, wenn Gateway/Agenten gebaut werden (Ü Woche 12) und der Eval-Harness ins CI geht (Ü Woche 13). Teil IV erhält drei Vorlesungen (Wochen 7, 10, 11) — als Herz des Moduls und Prüfungsschwerpunkt; Teil V kompakt in zwei (Wochen 12–13). 5. **Belastung:** Ab Woche 8 nur 3 L Vorlesung (nie 4); die 1-L-Übung ist Standup/Coaching, die Implementierung läuft im Selbststudium (78 h-Budget). -## Notwendige Anpassungen an project_exercise.tex +## Meilensteine (Stand 7. September 2026 — identisch mit project_exercise.tex und den Folien 1–14) -Die Meilenstein-Wochen im Aufgabenblatt stammen aus dem alten Ablauf und müssen auf die neue Phasenlogik gezogen werden: - -| Meilenstein | Bisher | Neu | +| Meilenstein | Wochen | Abgabe / Check | |---|---|---| -| M1 Domain, Ontologie, Architektur | Wochen 1–5 | **Wochen 1–7** (inkl. Requirements-Dossier W3, Architektur-Dossier + Design-Gate W7) | -| M2 Walking Skeleton | Woche 6 | **Wochen 8–9** | -| M3 Resilienz + deterministischer Kern | Wochen 7–9 | **Wochen 10–11** | -| M4 Multi-Agent + Evaluation | Wochen 8–10 | **Wochen 12–13** | -| M5 Operate/Scale/Harden | Wochen 11–13 | **Woche 13** (Kür parallel) | -| M6 Präsentation | Woche 14 | Woche 14 (unverändert) | +| M1 Requirements und Ontologie | 1–3 | **A1 Requirements-Dossier** (Ende Woche 3) | +| M2 Architekturentscheid und Lösungsdesign | 4–7 | **A2 Architektur-Dossier + Design-Review-Gate** (Ende Woche 7) | +| M3 Walking Skeleton | 8–9 | läuft End-to-End (Check Woche 9) | +| M4 Deterministischer Kern und Resilienz | 10–11 | Kern vollständig getestet + resilient (Ende Woche 11) | +| M5 Multi-Agent-Orchestrierung, Evaluation, Hardening | 12–13 | Eval-Harness im CI + Guard + Kosten-Observability (Ende Woche 13) | +| M6 Präsentation und Architektur-Verteidigung | 14 | **A3 Abschlusspräsentation**, Peer-Reviews | -Zudem: Titelseite sagt „Spring Semester" — das Modul läuft im Herbstsemester (5. Semester); bei der Anpassung korrigieren. +Die Titelseite des Aufgabenblatts nennt das Herbstsemester (5. Semester); die Meilenstein-Wochen sind auf die Phasenlogik oben gezogen. Die Zeilen 9/11/13 der Tabelle oben entsprechen M3/M4/M5. diff --git a/project_exercise/project_exercise.pdf b/project_exercise/project_exercise.pdf index 81e67e1..d0b4cd5 100644 Binary files a/project_exercise/project_exercise.pdf and b/project_exercise/project_exercise.pdf differ diff --git a/project_exercise/project_exercise.tex b/project_exercise/project_exercise.tex index c9d86a7..d0960e2 100644 --- a/project_exercise/project_exercise.tex +++ b/project_exercise/project_exercise.tex @@ -73,7 +73,7 @@ \vspace{1.2cm} {\large Semester Group Project\par} \vspace{0.4cm} - {\normalsize Language: Python 3.11+ \quad$\cdot$\quad Architecture-first \quad$\cdot$\quad Distributed services + AI agents\par} + {\normalsize Language: Python 3.11+ \quad$\cdot$\quad Architecture-first \quad$\cdot$\quad Services with contracts + AI agents\par} \vfill {\small Analysis \& advisory only --- no trading, no real money.\par} \end{titlepage} @@ -81,12 +81,15 @@ %==================================================================== \section{Overview and Goals} -The goal of this project is to design, build, and operate a \textbf{distributed, +The goal of this project is to design, build, and operate a \textbf{modular, AI-augmented analysis platform for stock portfolios}. The platform ingests \emph{structured} external data (market prices) and \emph{unstructured} external data (company news and web reports), turns both into decision-relevant information, computes quantitative risk, performance, and optimisation figures, and exposes the results through -a small set of cooperating services driven by an orchestrated multi-agent layer. +a small set of cooperating services driven by an orchestrated multi-agent layer. Whether those +services ship as one modular monolith or as several deployables is not prescribed here: it is +the architecture decision you take in week~6 with the lecture's three-stage match and defend +in week~14. This is a \textbf{Software Engineering II} project: the centre of gravity is \emph{architecture}---how you structure a system so that it meets its quality attributes @@ -210,6 +213,10 @@ are guidance, not a prescription. insights), \texttt{RiskAgent} (calls \texttt{RiskService}, interprets), and \texttt{OptimizationAgent} (calls \texttt{OptimizationService}, explains). \item Agents \emph{interpret and explain} numbers; services \emph{compute} them. + \item The lecture script's worked example (Part~V, Section~44.1) describes the same + workflow with the role names \emph{document analyst} ($\approx$ \texttt{ResearchAgent}), + \emph{portfolio quant} ($\approx$ \texttt{RiskAgent} / \texttt{OptimizationAgent}) and + \emph{compliance checker} ($\approx$ the ontology guard) -- one design, two vocabularies. \end{itemize} \end{taskbox} @@ -237,7 +244,9 @@ observability, hardening. \end{conceptbox} \subsection{Reference architecture} -The system follows a service-oriented split. Deterministic services form a trustworthy core; +The system follows a service-oriented split -- a \emph{logical} split into services with +contracts; the deployment cut (one modular monolith, several deployables, or a hybrid) is the +outcome of your match. Deterministic services form a trustworthy core; the non-deterministic AI layer sits on top and may only \emph{read} the core through its contracts. @@ -252,6 +261,8 @@ contracts. | |-- ResearchAgent (news -> Insight) | | |-- RiskAgent (calls RiskService) | | +-- OptimizationAgent (calls OptimizationService) | +| LLM Gateway: ONE port for every model call -- | +| cost/latency per request, fallback chain (ADR-011) | +-----------------------------------------------------------+ | DETERMINISTIC SERVICES (NO LLM INSIDE) | | MarketDataService (live API + cache fallback) | @@ -285,6 +296,13 @@ contracts. \item Use structured output / schema validation; reject or repair non-conforming responses. \item Keep system prompts concise; separate persona, task, and data (use the prompting techniques from AISE501). + \item \textbf{One LLM gateway}: every model call -- the \texttt{ResearchAgent}'s extraction as + well as the advisor's and sub-agents' calls -- goes through a single gateway behind a + port owned by the domain (the lecture's ADR-011). Domain code never imports a provider + SDK; the gateway records tokens, cost, and latency per request and holds the fallback + chain. + \item Every \texttt{Insight} carries a provenance reference to the stored news item, so that + the advisor's answers can cite their sources. \end{itemize} \end{taskbox} @@ -310,6 +328,22 @@ contracts. \end{itemize} \end{taskbox} +\subsection{Building with AI tools (Axis A)} +\begin{taskbox}[Requirements] +\begin{itemize}[nosep] + \item The repository carries an agent instruction file (\texttt{AGENTS.md} / \texttt{CLAUDE.md}) + that states the architecture rules an agent must respect; keep it as current as code. + \item Every architecture decision is an ADR: agents may draft it, a named team member signs it. + \item Agent-generated changes reach the main branch only through the CI gate: module-boundary + checks, the test suite, and -- for anything touching prompts or the gateway -- the eval + harness. + \item Your project handbook contains a one-page AI policy: permitted tools, provenance + labelling of generated code, review rules. +\end{itemize} +The graded artefact is not the generated code but the control system around it (Lecture~12); +the M6 reflection on where AI helped and hurt \emph{in building} draws on exactly this. +\end{taskbox} + \begin{warningbox} \begin{itemize}[nosep] \item \textbf{Never} let an agent compute or invent a numeric figure --- always route @@ -326,6 +360,7 @@ contracts. %==================================================================== \section{Mandatory vs.\ Distinction (Pflicht / K\"ur)} +\label{sec:distinction} \begin{conceptbox}[Mandatory --- required to pass] \begin{itemize}[nosep] @@ -334,7 +369,10 @@ contracts. \item Orchestrated advisor with \textbf{2--3 specialised sub-agents} cooperating only via service contracts. \item Ontology guard on all AI insights; resilience against external-data outages. - \item Evaluation harness for the AI component; basic observability (cost and latency). + \item Evaluation harness for the AI component; basic observability (cost and latency) + through the single LLM gateway. + \item The measurement contract of deliverable A2 (module-boundary check, eval threshold, + token-cost budget) wired into CI. \end{itemize} \end{conceptbox} @@ -383,7 +421,7 @@ measurement contract. 11 & Part IV: measurement contract in depth (fitness functions, DORA) & Resilience complete; \textbf{core fully tested} \\ 12 & Part V: Axis A; Axis B (gateway, eval basics) & Advisor + sub-agents behind the gateway \\ 13 & Part V: Axis B (security, orchestration economics) & \textbf{Eval harness in CI}; hardening; distinction work \\ -14 & Synthesis & \textbf{Presentations and architecture defence} \\ +14 & Synthesis (1 lesson) & \textbf{Presentations, architecture defence, peer reviews} (A3) \\ \bottomrule \end{tabular} \end{center} @@ -391,7 +429,9 @@ measurement contract. \begin{taskbox}[M1 --- Requirements and Ontology (Weeks 1--3) --- Deliverable A1] \begin{itemize}[nosep] \item Domain model and ontology of the investment domain. - \item Quality attribute scenarios with response measures; utility tree; the requirements + \item Quality attribute scenarios with response measures (at least eight scenarios, at + least three of them for the AI components: answer correctness, token cost per + request, provider migration -- Lecture~2); utility tree; the requirements profile $R(\mathrm{platform})$ with weights, workload shape, and hard constraints. \item Project and tooling setup (repository, environment, agentic dev tools). \item \textbf{Deliverable A1 (end of week 3): requirements dossier.} @@ -408,6 +448,10 @@ measurement contract. \item Bounded contexts $\rightarrow$ service decomposition and contracts; measurement contract (token budget, eval threshold, module-boundary checks); walking-skeleton plan. + \item Set numbers in the contract. Reference values from the lecture's worked ADRs + (Lectures~3 and~11): eval pass rate $\geq 95\,\%$ on a versioned golden set, p95 + latency $\leq 20$\,s for the advisory scenario, a token-cost budget per request at + p95, zero module-boundary violations. \item \textbf{Deliverable A2 (end of week 7): architecture dossier + design-review gate.} Production code starts only after the gate (exploratory spikes are allowed). \end{itemize} @@ -431,10 +475,12 @@ measurement contract. \begin{taskbox}[M5 --- Multi-Agent Orchestration, Evaluation, and Hardening (Weeks 12--13)] \begin{itemize}[nosep] - \item Week 12: advisor orchestrates 2--3 sub-agents through contracts (mandatory); - ontology guard active on all insights. + \item Week 12: advisor orchestrates 2--3 sub-agents through contracts, every LLM call + through the gateway (mandatory); ontology guard active on all insights. \item Week 13: evaluation harness as a CI gate; report accuracy and failure modes; - observability of token cost and latency per request. + observability of token cost and latency per request; an ADR that justifies the + chosen orchestration topology (task-signature table, Lecture~13) with its token + budget and eval threshold. \item Threat model including prompt injection via news; basic hardening; scaling/optimisation (caching, batching). \item Optional Distinction work: autonomy, self-repair, model routing, CI/CD, tracing. @@ -445,6 +491,9 @@ measurement contract. \begin{itemize}[nosep] \item Present the system and \textbf{defend your architectural trade-offs}. \item Reflect on where AI helped and where it hurt --- in building (A) and in the system (B). + \item Peer reviews: each team reviews the other teams' presentations and defences. + \item Be prepared to show one measurement-contract violation being caught by CI + (Lecture~11): a contract that has never failed has never been tested. \end{itemize} \end{taskbox} @@ -478,7 +527,7 @@ Evaluation of the project emphasises: \item \textbf{Quality} --- tests for deterministic services, eval harness for AI. \item \textbf{AI integration} --- correct anti-corruption layering and ontology guarding. \item \textbf{Operation} --- observability of cost/latency. - \item Distinction criteria for top marks (see Section~6). + \item Distinction criteria for top marks (see Section~\ref{sec:distinction}). \end{itemize} \vfill