Auto-commit 2026-09-07 15:59: 13 files changed, 77 insertions(+), 34 deletions(-)
This commit is contained in:
herzogflorian
parent
14514eba4a
commit
a1e51df7bb
Binary file not shown.
@ -548,7 +548,7 @@ Cost per request is a runtime quality attribute with \textbf{no counterpart in c
|
||||
\begin{projektbox}
|
||||
\footnotesize Your project submission must ship its \textbf{measurement contract}, not just its architecture -- the repository must contain, \emph{wired into CI}:
|
||||
\begin{enumerate}\setlength\itemsep{1pt}
|
||||
\item module-boundary verification with \textbf{zero violations} (ArchUnit or Spring Modulith verify), including the determinism-boundary rule: no domain service reaches the LLM gateway except via its declared port
|
||||
\item module-boundary verification with \textbf{zero violations} (Python: import-linter; Java: ArchUnit or Spring Modulith verify), including the determinism-boundary rule: no domain service reaches the LLM gateway except via its declared port
|
||||
\item an eval harness with a versioned golden set and a \textbf{pass rate $\geq 95\,\%$} gating every prompt or model change
|
||||
\item a \textbf{token-cost budget per request} enforced as a pipeline gate and reported per feature
|
||||
\item a p95 latency budget for the advisory scenario (\textbf{$< 20$\,s} end-to-end)
|
||||
|
||||
Binary file not shown.
@ -849,7 +849,7 @@ def test\_judge\_is\_calibrated(judge, human\_labels):\\
|
||||
\footnotesize \textbf{Coaching slot (1 lesson). Milestone M5} -- Multi-Agent Orchestration, Evaluation, and Hardening (weeks 12--13):
|
||||
\begin{enumerate}\setlength\itemsep{2pt}
|
||||
\item \textbf{Mandatory this week}: the AdvisorAgent orchestrates 2--3 sub-agents \emph{through contracts} -- every LLM call through the gateway port (ADR-011); no provider SDK import in domain code
|
||||
\item \textbf{Ontology guard active on all insights}: entity resolution against the deterministic store, domain axioms, citation check -- unresolvable references are \emph{rejected}, not passed on
|
||||
\item \textbf{Ontology guard active on all insights}: entity resolution against the deterministic store (valid ticker, sector, event type), domain axioms, citation check -- unresolvable references are \emph{rejected}, not passed on
|
||||
\item \textbf{LLM agents propose; deterministic services decide and book} -- keep the deterministic core free of LLM calls \emph{(the line that is graded)}
|
||||
\end{enumerate}
|
||||
\vspace{0.05cm}
|
||||
|
||||
Binary file not shown.
@ -678,9 +678,9 @@ Status & \multicolumn{2}{@{}>{\raggedright\arraybackslash}p{10.1cm}@{}}{provisio
|
||||
|
||||
\begin{frame}{Project link: Axis B is what you build -- and what you build it as}
|
||||
\begin{projektbox}
|
||||
\footnotesize Axis B is \emph{what} you build; the advisor workflow -- an orchestrator with two to three specialised sub-agents (document analyst, portfolio quant, compliance checker), all behind the LLM gateway -- is what you build it \emph{as}. The graded Axis-B deliverables are the \textbf{containment artefacts}:
|
||||
\footnotesize Axis B is \emph{what} you build; the advisor workflow -- an orchestrator with two to three specialised sub-agents (document analyst, portfolio quant, compliance checker -- in the exercise sheet: \texttt{ResearchAgent}, \texttt{RiskAgent}\,/\,\texttt{OptimizationAgent}, and the ontology guard), all behind the LLM gateway -- is what you build it \emph{as}. The graded Axis-B deliverables are the \textbf{containment artefacts}:
|
||||
\begin{enumerate}\setlength\itemsep{1pt}
|
||||
\item[(i)] the \textbf{gateway} with model routing, fallback chain, and per-request \textbf{cost observability} -- cost per request, per feature, reported on a dashboard and enforced as a CI budget
|
||||
\item[(i)] the \textbf{gateway} with fallback chain and per-request \textbf{cost observability} -- cost per request, per feature, reported on a dashboard and enforced as a CI budget (model routing itself is distinction work)
|
||||
\item[(ii)] the \textbf{ontology guard} -- every extracted entity resolves against the deterministic data store, every cited passage exists, portfolio axioms hold
|
||||
\item[(iii)] the \textbf{eval harness} (Lecture 12 listing) wired as a \textbf{CI gate}
|
||||
\item[(iv)] one \textbf{ADR} that justifies your chosen orchestration topology against the task-signature table, with its \textbf{token budget and eval threshold} as the measurement contract
|
||||
|
||||
Binary file not shown.
@ -224,7 +224,7 @@ After this module you can \dots
|
||||
|
||||
\begin{frame}{The course project}
|
||||
\begin{projektbox}
|
||||
You will design and build an \textbf{AI-native advisory platform}: answers generated by LLMs over a grounded knowledge base, with deterministic services for portfolio analytics around them. Everything this module teaches -- profiles, matching, ADRs, measurement -- you will apply to \emph{this} system, and defend at the end.
|
||||
You will design and build an \textbf{AI-augmented portfolio intelligence platform} (exercise sheet): market prices and company news in; deterministic risk, performance, and optimisation services; an LLM component turning news into validated \texttt{Insight}s; a multi-agent advisor that explains -- API-first. Everything this module teaches -- profiles, matching, ADRs, measurement -- you will apply to \emph{this} system, and defend at the end.
|
||||
\end{projektbox}
|
||||
|
||||
\vspace{0.2cm}
|
||||
@ -235,7 +235,7 @@ You will design and build an \textbf{AI-native advisory platform}: answers gener
|
||||
\item collect \emph{raw} stakeholder wishes -- unfiltered, unweighted
|
||||
\end{itemize}
|
||||
|
||||
\vspace{0.2cm}
|
||||
\vspace{0.05cm}
|
||||
\textcolor{codegray}{\small Next week those wishes become \emph{scenarios}; in week 3 they become your requirements profile (deliverable A1).}
|
||||
\end{frame}
|
||||
|
||||
|
||||
Binary file not shown.
@ -353,13 +353,11 @@ Constraints $K(a)$ & \multicolumn{2}{l}{BCBS 239 lineage (banks)} \\
|
||||
|
||||
\begin{frame}{Closing the loop: C6 and C7 inside your C10 project}
|
||||
\small Week 6's inheritance diagram: C10 inherits C6, C7 and C3 -- today the two batch parents are taught.
|
||||
|
||||
\vspace{0.05cm}
|
||||
\begin{projektbox}
|
||||
\footnotesize
|
||||
\begin{columns}[T,totalwidth=\linewidth]
|
||||
\begin{column}{0.48\textwidth}
|
||||
\textcolor{bankblue}{\textbf{The C6 subsystems}} -- ingestion (chunking, embedding, indexing) and the eval pipeline
|
||||
\textcolor{bankblue}{\textbf{The C6 subsystems}} -- ingestion (chunking, embedding, indexing -- in the sheet: news ingestion $\to$ \texttt{ResearchAgent}) and the eval pipeline
|
||||
\begin{itemize}\setlength\itemsep{1pt}
|
||||
\item the C6 discipline transfers literally: versioned inputs, idempotent stages, deterministic reruns, golden datasets
|
||||
\item note 14 is the bridge: for batch pipelines, \emph{reproducibility is the operative meaning of testability}
|
||||
@ -375,8 +373,6 @@ Constraints $K(a)$ & \multicolumn{2}{l}{BCBS 239 lineage (banks)} \\
|
||||
\end{column}
|
||||
\end{columns}
|
||||
\end{projektbox}
|
||||
|
||||
\vspace{0.05cm}
|
||||
\footnotesize \textbf{Key concept:} the freshness contract appears \textbf{twice} in your measurement contract -- as a pipeline fitness function (C7 discipline) and as a grounding rule for generated answers (C10 discipline).
|
||||
\end{frame}
|
||||
|
||||
|
||||
@ -32,17 +32,15 @@
|
||||
4. **Nichts Blockierendes zu spät:** C1–C9 (Prüfungsstoff, aber nicht design-relevant) sowie Achse A/B kommen in den Wochen 8–13 — Gateway- und Eval-Harness-Grundlagen (V Woche 12) genau dann, wenn Gateway/Agenten gebaut werden (Ü Woche 12) und der Eval-Harness ins CI geht (Ü Woche 13). Teil IV erhält drei Vorlesungen (Wochen 7, 10, 11) — als Herz des Moduls und Prüfungsschwerpunkt; Teil V kompakt in zwei (Wochen 12–13).
|
||||
5. **Belastung:** Ab Woche 8 nur 3 L Vorlesung (nie 4); die 1-L-Übung ist Standup/Coaching, die Implementierung läuft im Selbststudium (78 h-Budget).
|
||||
|
||||
## Notwendige Anpassungen an project_exercise.tex
|
||||
## Meilensteine (Stand 7. September 2026 — identisch mit project_exercise.tex und den Folien 1–14)
|
||||
|
||||
Die Meilenstein-Wochen im Aufgabenblatt stammen aus dem alten Ablauf und müssen auf die neue Phasenlogik gezogen werden:
|
||||
|
||||
| Meilenstein | Bisher | Neu |
|
||||
| Meilenstein | Wochen | Abgabe / Check |
|
||||
|---|---|---|
|
||||
| M1 Domain, Ontologie, Architektur | Wochen 1–5 | **Wochen 1–7** (inkl. Requirements-Dossier W3, Architektur-Dossier + Design-Gate W7) |
|
||||
| M2 Walking Skeleton | Woche 6 | **Wochen 8–9** |
|
||||
| M3 Resilienz + deterministischer Kern | Wochen 7–9 | **Wochen 10–11** |
|
||||
| M4 Multi-Agent + Evaluation | Wochen 8–10 | **Wochen 12–13** |
|
||||
| M5 Operate/Scale/Harden | Wochen 11–13 | **Woche 13** (Kür parallel) |
|
||||
| M6 Präsentation | Woche 14 | Woche 14 (unverändert) |
|
||||
| M1 Requirements und Ontologie | 1–3 | **A1 Requirements-Dossier** (Ende Woche 3) |
|
||||
| M2 Architekturentscheid und Lösungsdesign | 4–7 | **A2 Architektur-Dossier + Design-Review-Gate** (Ende Woche 7) |
|
||||
| M3 Walking Skeleton | 8–9 | läuft End-to-End (Check Woche 9) |
|
||||
| M4 Deterministischer Kern und Resilienz | 10–11 | Kern vollständig getestet + resilient (Ende Woche 11) |
|
||||
| M5 Multi-Agent-Orchestrierung, Evaluation, Hardening | 12–13 | Eval-Harness im CI + Guard + Kosten-Observability (Ende Woche 13) |
|
||||
| M6 Präsentation und Architektur-Verteidigung | 14 | **A3 Abschlusspräsentation**, Peer-Reviews |
|
||||
|
||||
Zudem: Titelseite sagt „Spring Semester" — das Modul läuft im Herbstsemester (5. Semester); bei der Anpassung korrigieren.
|
||||
Die Titelseite des Aufgabenblatts nennt das Herbstsemester (5. Semester); die Meilenstein-Wochen sind auf die Phasenlogik oben gezogen. Die Zeilen 9/11/13 der Tabelle oben entsprechen M3/M4/M5.
|
||||
|
||||
Binary file not shown.
@ -73,7 +73,7 @@
|
||||
\vspace{1.2cm}
|
||||
{\large Semester Group Project\par}
|
||||
\vspace{0.4cm}
|
||||
{\normalsize Language: Python 3.11+ \quad$\cdot$\quad Architecture-first \quad$\cdot$\quad Distributed services + AI agents\par}
|
||||
{\normalsize Language: Python 3.11+ \quad$\cdot$\quad Architecture-first \quad$\cdot$\quad Services with contracts + AI agents\par}
|
||||
\vfill
|
||||
{\small Analysis \& advisory only --- no trading, no real money.\par}
|
||||
\end{titlepage}
|
||||
@ -81,12 +81,15 @@
|
||||
%====================================================================
|
||||
\section{Overview and Goals}
|
||||
|
||||
The goal of this project is to design, build, and operate a \textbf{distributed,
|
||||
The goal of this project is to design, build, and operate a \textbf{modular,
|
||||
AI-augmented analysis platform for stock portfolios}. The platform ingests
|
||||
\emph{structured} external data (market prices) and \emph{unstructured} external data
|
||||
(company news and web reports), turns both into decision-relevant information, computes
|
||||
quantitative risk, performance, and optimisation figures, and exposes the results through
|
||||
a small set of cooperating services driven by an orchestrated multi-agent layer.
|
||||
a small set of cooperating services driven by an orchestrated multi-agent layer. Whether those
|
||||
services ship as one modular monolith or as several deployables is not prescribed here: it is
|
||||
the architecture decision you take in week~6 with the lecture's three-stage match and defend
|
||||
in week~14.
|
||||
|
||||
This is a \textbf{Software Engineering II} project: the centre of gravity is
|
||||
\emph{architecture}---how you structure a system so that it meets its quality attributes
|
||||
@ -210,6 +213,10 @@ are guidance, not a prescription.
|
||||
insights), \texttt{RiskAgent} (calls \texttt{RiskService}, interprets), and
|
||||
\texttt{OptimizationAgent} (calls \texttt{OptimizationService}, explains).
|
||||
\item Agents \emph{interpret and explain} numbers; services \emph{compute} them.
|
||||
\item The lecture script's worked example (Part~V, Section~44.1) describes the same
|
||||
workflow with the role names \emph{document analyst} ($\approx$ \texttt{ResearchAgent}),
|
||||
\emph{portfolio quant} ($\approx$ \texttt{RiskAgent} / \texttt{OptimizationAgent}) and
|
||||
\emph{compliance checker} ($\approx$ the ontology guard) -- one design, two vocabularies.
|
||||
\end{itemize}
|
||||
\end{taskbox}
|
||||
|
||||
@ -237,7 +244,9 @@ observability, hardening.
|
||||
\end{conceptbox}
|
||||
|
||||
\subsection{Reference architecture}
|
||||
The system follows a service-oriented split. Deterministic services form a trustworthy core;
|
||||
The system follows a service-oriented split -- a \emph{logical} split into services with
|
||||
contracts; the deployment cut (one modular monolith, several deployables, or a hybrid) is the
|
||||
outcome of your match. Deterministic services form a trustworthy core;
|
||||
the non-deterministic AI layer sits on top and may only \emph{read} the core through its
|
||||
contracts.
|
||||
|
||||
@ -252,6 +261,8 @@ contracts.
|
||||
| |-- ResearchAgent (news -> Insight) |
|
||||
| |-- RiskAgent (calls RiskService) |
|
||||
| +-- OptimizationAgent (calls OptimizationService) |
|
||||
| LLM Gateway: ONE port for every model call -- |
|
||||
| cost/latency per request, fallback chain (ADR-011) |
|
||||
+-----------------------------------------------------------+
|
||||
| DETERMINISTIC SERVICES (NO LLM INSIDE) |
|
||||
| MarketDataService (live API + cache fallback) |
|
||||
@ -285,6 +296,13 @@ contracts.
|
||||
\item Use structured output / schema validation; reject or repair non-conforming responses.
|
||||
\item Keep system prompts concise; separate persona, task, and data (use the prompting
|
||||
techniques from AISE501).
|
||||
\item \textbf{One LLM gateway}: every model call -- the \texttt{ResearchAgent}'s extraction as
|
||||
well as the advisor's and sub-agents' calls -- goes through a single gateway behind a
|
||||
port owned by the domain (the lecture's ADR-011). Domain code never imports a provider
|
||||
SDK; the gateway records tokens, cost, and latency per request and holds the fallback
|
||||
chain.
|
||||
\item Every \texttt{Insight} carries a provenance reference to the stored news item, so that
|
||||
the advisor's answers can cite their sources.
|
||||
\end{itemize}
|
||||
\end{taskbox}
|
||||
|
||||
@ -310,6 +328,22 @@ contracts.
|
||||
\end{itemize}
|
||||
\end{taskbox}
|
||||
|
||||
\subsection{Building with AI tools (Axis A)}
|
||||
\begin{taskbox}[Requirements]
|
||||
\begin{itemize}[nosep]
|
||||
\item The repository carries an agent instruction file (\texttt{AGENTS.md} / \texttt{CLAUDE.md})
|
||||
that states the architecture rules an agent must respect; keep it as current as code.
|
||||
\item Every architecture decision is an ADR: agents may draft it, a named team member signs it.
|
||||
\item Agent-generated changes reach the main branch only through the CI gate: module-boundary
|
||||
checks, the test suite, and -- for anything touching prompts or the gateway -- the eval
|
||||
harness.
|
||||
\item Your project handbook contains a one-page AI policy: permitted tools, provenance
|
||||
labelling of generated code, review rules.
|
||||
\end{itemize}
|
||||
The graded artefact is not the generated code but the control system around it (Lecture~12);
|
||||
the M6 reflection on where AI helped and hurt \emph{in building} draws on exactly this.
|
||||
\end{taskbox}
|
||||
|
||||
\begin{warningbox}
|
||||
\begin{itemize}[nosep]
|
||||
\item \textbf{Never} let an agent compute or invent a numeric figure --- always route
|
||||
@ -326,6 +360,7 @@ contracts.
|
||||
|
||||
%====================================================================
|
||||
\section{Mandatory vs.\ Distinction (Pflicht / K\"ur)}
|
||||
\label{sec:distinction}
|
||||
|
||||
\begin{conceptbox}[Mandatory --- required to pass]
|
||||
\begin{itemize}[nosep]
|
||||
@ -334,7 +369,10 @@ contracts.
|
||||
\item Orchestrated advisor with \textbf{2--3 specialised sub-agents} cooperating only via
|
||||
service contracts.
|
||||
\item Ontology guard on all AI insights; resilience against external-data outages.
|
||||
\item Evaluation harness for the AI component; basic observability (cost and latency).
|
||||
\item Evaluation harness for the AI component; basic observability (cost and latency)
|
||||
through the single LLM gateway.
|
||||
\item The measurement contract of deliverable A2 (module-boundary check, eval threshold,
|
||||
token-cost budget) wired into CI.
|
||||
\end{itemize}
|
||||
\end{conceptbox}
|
||||
|
||||
@ -383,7 +421,7 @@ measurement contract.
|
||||
11 & Part IV: measurement contract in depth (fitness functions, DORA) & Resilience complete; \textbf{core fully tested} \\
|
||||
12 & Part V: Axis A; Axis B (gateway, eval basics) & Advisor + sub-agents behind the gateway \\
|
||||
13 & Part V: Axis B (security, orchestration economics) & \textbf{Eval harness in CI}; hardening; distinction work \\
|
||||
14 & Synthesis & \textbf{Presentations and architecture defence} \\
|
||||
14 & Synthesis (1 lesson) & \textbf{Presentations, architecture defence, peer reviews} (A3) \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\end{center}
|
||||
@ -391,7 +429,9 @@ measurement contract.
|
||||
\begin{taskbox}[M1 --- Requirements and Ontology (Weeks 1--3) --- Deliverable A1]
|
||||
\begin{itemize}[nosep]
|
||||
\item Domain model and ontology of the investment domain.
|
||||
\item Quality attribute scenarios with response measures; utility tree; the requirements
|
||||
\item Quality attribute scenarios with response measures (at least eight scenarios, at
|
||||
least three of them for the AI components: answer correctness, token cost per
|
||||
request, provider migration -- Lecture~2); utility tree; the requirements
|
||||
profile $R(\mathrm{platform})$ with weights, workload shape, and hard constraints.
|
||||
\item Project and tooling setup (repository, environment, agentic dev tools).
|
||||
\item \textbf{Deliverable A1 (end of week 3): requirements dossier.}
|
||||
@ -408,6 +448,10 @@ measurement contract.
|
||||
\item Bounded contexts $\rightarrow$ service decomposition and contracts; measurement
|
||||
contract (token budget, eval threshold, module-boundary checks); walking-skeleton
|
||||
plan.
|
||||
\item Set numbers in the contract. Reference values from the lecture's worked ADRs
|
||||
(Lectures~3 and~11): eval pass rate $\geq 95\,\%$ on a versioned golden set, p95
|
||||
latency $\leq 20$\,s for the advisory scenario, a token-cost budget per request at
|
||||
p95, zero module-boundary violations.
|
||||
\item \textbf{Deliverable A2 (end of week 7): architecture dossier + design-review
|
||||
gate.} Production code starts only after the gate (exploratory spikes are allowed).
|
||||
\end{itemize}
|
||||
@ -431,10 +475,12 @@ measurement contract.
|
||||
|
||||
\begin{taskbox}[M5 --- Multi-Agent Orchestration, Evaluation, and Hardening (Weeks 12--13)]
|
||||
\begin{itemize}[nosep]
|
||||
\item Week 12: advisor orchestrates 2--3 sub-agents through contracts (mandatory);
|
||||
ontology guard active on all insights.
|
||||
\item Week 12: advisor orchestrates 2--3 sub-agents through contracts, every LLM call
|
||||
through the gateway (mandatory); ontology guard active on all insights.
|
||||
\item Week 13: evaluation harness as a CI gate; report accuracy and failure modes;
|
||||
observability of token cost and latency per request.
|
||||
observability of token cost and latency per request; an ADR that justifies the
|
||||
chosen orchestration topology (task-signature table, Lecture~13) with its token
|
||||
budget and eval threshold.
|
||||
\item Threat model including prompt injection via news; basic hardening;
|
||||
scaling/optimisation (caching, batching).
|
||||
\item Optional Distinction work: autonomy, self-repair, model routing, CI/CD, tracing.
|
||||
@ -445,6 +491,9 @@ measurement contract.
|
||||
\begin{itemize}[nosep]
|
||||
\item Present the system and \textbf{defend your architectural trade-offs}.
|
||||
\item Reflect on where AI helped and where it hurt --- in building (A) and in the system (B).
|
||||
\item Peer reviews: each team reviews the other teams' presentations and defences.
|
||||
\item Be prepared to show one measurement-contract violation being caught by CI
|
||||
(Lecture~11): a contract that has never failed has never been tested.
|
||||
\end{itemize}
|
||||
\end{taskbox}
|
||||
|
||||
@ -478,7 +527,7 @@ Evaluation of the project emphasises:
|
||||
\item \textbf{Quality} --- tests for deterministic services, eval harness for AI.
|
||||
\item \textbf{AI integration} --- correct anti-corruption layering and ontology guarding.
|
||||
\item \textbf{Operation} --- observability of cost/latency.
|
||||
\item Distinction criteria for top marks (see Section~6).
|
||||
\item Distinction criteria for top marks (see Section~\ref{sec:distinction}).
|
||||
\end{itemize}
|
||||
|
||||
\vfill
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user