Ai ReflectionEngine Added

This commit is contained in:
Thomas Hollenstein 2026-05-29 18:15:42 +02:00
parent 4760dbb3a3
commit e7be1c94fd
5 changed files with 754 additions and 16 deletions

315
README.me
View File

@ -1 +1,314 @@
AISE AI Code Editor
# AI Code Assistant
Ein modularer KI-gestützter Coding-Assistent mit Tool-Ausführung, Datei-Kontextverwaltung und optionaler selbstreflektierender Agentenlogik.
---
# Übersicht
Dieses Projekt ist ein erweiterbarer AI-Code-Assistent, der entwickelt wurde, um:
* Code zu verstehen und zu erklären
* Dateien zu bearbeiten und zu verwalten
* Code auszuführen und Fehler zu analysieren
* Dynamisch Tools zu verwenden
* Antworten iterativ zu verbessern
Die Anwendung kombiniert einen agentischen Workflow mit einer Reflexions-Engine, um qualitativ hochwertige Antworten und Code-Verbesserungen zu erzeugen.
---
# Projektstruktur
```text
main.py
src/
├── backend/
│ ├── chat_manager.py
│ ├── reflection_engine.py
│ ├── execution_engine.py
│ ├── file_manager.py
│ ├── llm_client.py
│ ├── system_prompter.py
│ ├── tools.py
│ └── ...weitere Backend-Module
└── ui/
└── ...Frontend- und UI-Dateien
```
---
# Architektur
## `main.py`
Der Einstiegspunkt der Anwendung.
Verantwortlich für:
* Initialisierung der Anwendung
* Starten der Streamlit-Oberfläche
* Verbinden aller Backend-Komponenten
* Laden der UI
---
# Backend (`src/backend`)
Enthält die gesamte Kernlogik der Anwendung.
---
## `chat_manager.py`
Zentrale Steuerungseinheit des Assistenten.
Aufgaben:
* Verwaltung der Konversation
* Agentischer Tool-Loop
* Datei-Kontextverwaltung
* Integration der Reflection Engine
* Tool-Ausführung
* Kommunikation zwischen UI und Backend
Der `ChatManager` bildet das Herzstück der Anwendung.
---
## `reflection_engine.py`
Implementiert den selbstreflektierenden Agenten-Workflow.
Funktionen:
* Bewertung von Antworten
* Analyse von Schwächen
* Iterative Verbesserung von Antworten
* Qualitätsbewertung über Score-System
* Automatisches Stoppen bei erreichter Qualität
* Unterstützung von Tool-Aufrufen während der Verbesserung
---
## `execution_engine.py`
Verantwortlich für:
* Ausführen von Dateien
* Ausführen von Code-Snippets
* Erfassen von `stdout` und `stderr`
* Rückgabe von Debug-Logs
---
## `file_manager.py`
Verwaltet:
* Lesen von Dateien
* Schreiben von Dateien
* Aktive Datei-Kontexte
---
## `tools.py`
Definiert verfügbare Tools und das Tool-Registry-System.
Beispiele:
* `read_file`
* `edit_file`
* `execute_file`
* `execute_code`
* `search`
---
## `llm_client.py`
Kommunikation mit dem verwendeten LLM/API.
---
## `system_prompter.py`
Erstellt dynamische Systemprompts inklusive:
* Datei-Kontext
* Tool-Beschreibungen
* Laufzeitinformationen
---
# UI (`src/ui`)
Enthält sämtliche Frontend- und Benutzeroberflächen-Komponenten.
Mögliche Funktionen:
* Chat-Oberfläche
* Dateiansicht
* Tool-Aktivitätsanzeige
* Reflection-Status
* Ausgabe von Logs und Fehlern
---
# Reflection-System
Der Assistent kann Antworten automatisch analysieren und verbessern.
Ablauf:
1. Initiale Antwort erzeugen
2. Antwort bewerten
3. Schwächen identifizieren
4. Antwort verbessern
5. Wiederholen bis Qualitätsziel erreicht ist
---
## Reflection Workflow
```text
Benutzeranfrage
Erste Antwort
Kritik & Analyse
Qualitätsbewertung
Verbesserung
Optimierte Antwort
```
---
## Reflection-Konfiguration
Im `ChatManager`:
```python
self.reflection_enabled = True
self.reflection_max_rounds = 3
self.reflection_threshold = 8.5
```
---
## Reflection aktivieren/deaktivieren
```python
chat_manager.set_reflection(
enabled=True,
max_rounds=3,
threshold=8.5
)
```
---
# Features
* Tool-basierter KI-Agent
* Iterative Antwortverbesserung
* Selbstreflektierende Analyse
* Datei-Kontextintegration
* Code-Ausführung
* Modulare Architektur
* Streamlit-UI
* Dynamische Tool-Registry
* Debug- und Fehleranalyse
* JSON-basierte Agent-Kommunikation
---
# Installation
## 1. Repository klonen
```bash
git clone <repository-url>
cd <projektname>
```
---
## 2. Virtuelle Umgebung erstellen
### Windows
```bash
python -m venv venv
venv\Scripts\activate
```
### Linux / macOS
```bash
python3 -m venv venv
source venv/bin/activate
```
---
## 3. Abhängigkeiten installieren
```bash
pip install -r requirements.txt
```
---
# Anwendung starten
Die Anwendung wird über Streamlit gestartet:
```bash
streamlit run main.py
```
---
# Beispielanfragen
## Code erklären
```text
Was macht diese Funktion?
```
---
## Datei bearbeiten
```text
Füge Kommentare zu dieser Datei hinzu
```
---
## Datei ausführen
```text
Führe die aktuelle Datei aus
```
---
## Fehler analysieren
```text
Warum tritt dieser Fehler auf?
```
---

View File

@ -27,6 +27,7 @@ MarkupSafe==3.0.3
matplotlib-inline==0.2.2
narwhals==2.21.2
numpy==2.4.6
openai
packaging==26.2
pandas==3.0.3
parso==0.8.7
@ -57,6 +58,8 @@ soupsieve==2.8
stack-data==0.6.3
starlette==1.0.0
streamlit==1.57.0
streamlit-ace
streamlit_antd_components
streamlit-tree-select==0.0.5
tenacity==9.1.4
toml==0.10.2

View File

@ -5,6 +5,7 @@ from .system_prompter import SystemPrompter
from .file_manager import FileManager
from .tools import TOOL_REGISTRY, build_tool_description
from .execution_engine import ExecutionEngine
from .reflection_engine import ReflectionEngine
import backend.tools as tools_module
@ -19,12 +20,35 @@ class ChatManager:
self.active_file_content = None
self.on_file_written = None
self.on_tool_used = None
# ---- Reflection settings ----
self.reflection_enabled = True
self.reflection_max_rounds = 3
self.reflection_threshold = 8.5
tools_module.on_tool_used = self._tool_used_callback
# ------------------------------------------------------------------ #
# Callbacks #
# ------------------------------------------------------------------ #
def _tool_used_callback(self, tool_name: str, summary: str):
if self.on_tool_used:
self.on_tool_used(tool_name, summary)
def _reflection_progress_callback(self, round_num: int, step):
if self.on_tool_used:
icon = "" if step.stop_reason in ("threshold_reached",) else "🔁"
self.on_tool_used(
"reflect",
f"{icon} Reflexion Runde {round_num} | Score {step.score:.1f}/10"
+ (f" | {step.stop_reason}" if step.stop_reason not in ("continued", "") else ""),
)
# ------------------------------------------------------------------ #
# File context #
# ------------------------------------------------------------------ #
def set_active_file(self, file_path: str, content: str = None):
self.active_file_path = file_path
self.active_file_content = content if content else None
@ -77,7 +101,55 @@ To give a final answer (NO tool needed):
- Never use execute_code just to print an explanation answer directly instead
"""
# ------------------------------------------------------------------ #
# Core generation loop #
# ------------------------------------------------------------------ #
def generate_response(self, prompt: str, execution_results: dict = None) -> str:
"""
1. Agentic loop initial answer (hat Tool-Zugriff)
2. Wenn Reflection aktiviert: ReflectionEngine verbessert die Antwort.
Die Engine bekommt jetzt agent_loop + debug_logs übergeben,
damit Refinement-Runden ebenfalls Tools aufrufen können.
"""
initial_answer = self._run_agent_loop(prompt, execution_results)
if not self.reflection_enabled:
return initial_answer
# Sehr kurze oder Fehler-Antworten nicht reflektieren
if len(initial_answer.strip()) < 30 or initial_answer.strip().upper().startswith("ERROR"):
return initial_answer
engine = ReflectionEngine(
client=self.client,
max_rounds=self.reflection_max_rounds,
threshold=self.reflection_threshold,
model=MODEL,
on_progress=self._reflection_progress_callback,
# NEU: agent_loop-Callback damit Refinement Tools nutzen kann
agent_loop=self._run_agent_loop,
)
context = self.active_file_content or ""
result = engine.reflect(
user_question=prompt,
initial_answer=initial_answer,
context=context,
# NEU: Debug-Logs werden durchgereicht
debug_logs=execution_results,
)
if self.on_tool_used and result.improved:
self.on_tool_used("reflect", f"{result.summary}")
return result.final_answer
# ------------------------------------------------------------------ #
# Agentic tool-use loop #
# ------------------------------------------------------------------ #
def _run_agent_loop(self, prompt: str, execution_results: dict = None) -> str:
system_prompt = self._build_system_prompt()
user_message = f"User request: {prompt}"
@ -90,6 +162,10 @@ To give a final answer (NO tool needed):
user_message += "\n[Execution Feedback — errors found]:\n"
for log in execution_results.get("logs", []):
user_message += f" - [{log['level']}] {log['message']}\n"
if execution_results.get("stderr", "").strip():
user_message += f"[stderr]:\n{execution_results['stderr'].strip()}\n"
if execution_results.get("stdout", "").strip():
user_message += f"[stdout]:\n{execution_results['stdout'].strip()}\n"
messages = [
{"role": "system", "content": system_prompt},
@ -160,9 +236,18 @@ To give a final answer (NO tool needed):
"content": "Your response was not valid JSON. Reply with a single JSON object only."
})
# Hard fallback — plain chat without JSON constraint
# Hard fallback
return chat(self.client, messages, model=MODEL, temperature=0.7)
# ------------------------------------------------------------------ #
# Session management #
# ------------------------------------------------------------------ #
def set_reflection(self, enabled: bool, max_rounds: int = 3, threshold: float = 8.5):
self.reflection_enabled = enabled
self.reflection_max_rounds = max(1, min(max_rounds, 5))
self.reflection_threshold = max(1.0, min(threshold, 10.0))
def get_history(self) -> list:
return self.conversation_history

View File

@ -0,0 +1,280 @@
import json
from dataclasses import dataclass, field
from typing import Optional, Callable
from .llm_client import chat_json, MODEL
# ------------------------------------------------------------------ #
# Datastructures #
# ------------------------------------------------------------------ #
@dataclass
class ReflectionStep:
round: int
critique: str
score: float
improved_answer: str
stop_reason: str = "" # "threshold_reached" | "no_improvement" | "max_rounds" | "continued"
@dataclass
class ReflectionResult:
original_answer: str
final_answer: str
rounds_taken: int
steps: list[ReflectionStep] = field(default_factory=list)
improved: bool = False
@property
def summary(self) -> str:
if not self.steps:
return "Keine Reflexion durchgeführt."
last = self.steps[-1]
return (
f"🔁 {self.rounds_taken} Reflexionsrunde(n) | "
f"Score: {last.score:.1f}/10 | "
f"Stop: {last.stop_reason}"
)
# ------------------------------------------------------------------ #
# Engine #
# ------------------------------------------------------------------ #
class ReflectionEngine:
CRITIC_SYSTEM = """You are a strict code-assistant quality reviewer.
You evaluate an AI assistant's answer to a coding/debugging question.
You receive:
- The user question
- The full file content (if available)
- Debug logs / execution output (if available)
- The assistant's answer
Use ALL of this to spot errors: wrong line numbers, fixes that don't match the
actual code, ignored error messages, missing edge cases, etc.
Respond ONLY with a JSON object:
{
"score": <float 0-10>,
"critique": "<concise bullet list of weaknesses, max 5 points>",
"needs_improvement": <true|false>
}
Scoring guide:
9-10 : Nearly perfect correct, clear, complete, matches actual code
7-8 : Good but minor issues (verbosity, one missed case, slight imprecision)
5-6 : Mediocre partly correct, ignores evidence from logs or code
3-4 : Poor significant errors or contradicts the actual file/logs
0-2 : Wrong or harmful
Set needs_improvement=false when score >= 8.5."""
# Refinement-System-Prompt für den Fallback (kein agent_loop verfügbar)
REFINE_FALLBACK_SYSTEM = """You are a code assistant improving your own previous answer.
You receive: the original user question, file content, debug logs, your previous answer,
and a critique of it.
Produce a strictly better answer that addresses ALL critique points and is consistent
with the actual file content and debug logs shown.
Do NOT mention the reflection process or that this is a revised answer.
Respond ONLY with a JSON object:
{
"answer": "<your improved answer in Markdown>"
}"""
def __init__(
self,
client,
max_rounds: int = 3,
threshold: float = 8.5,
model: str = MODEL,
on_progress: Optional[Callable] = None,
agent_loop: Optional[Callable] = None,
):
self.client = client
self.max_rounds = max_rounds
self.threshold = threshold
self.model = model
self.on_progress = on_progress
# agent_loop(prompt: str, execution_results: dict | None) -> str
self.agent_loop = agent_loop
# -------------------------------------------------------------- #
def reflect(
self,
user_question: str,
initial_answer: str,
context: str = "",
debug_logs: Optional[dict] = None,
) -> ReflectionResult:
result = ReflectionResult(
original_answer=initial_answer,
final_answer=initial_answer,
rounds_taken=0,
)
current_answer = initial_answer
prev_score: Optional[float] = None
for round_num in range(1, self.max_rounds + 1):
# ---- 1. Critique ----
critique_data = self._critique(
user_question, current_answer, context, debug_logs
)
score = critique_data.get("score", 5.0)
critique_text = critique_data.get("critique", "")
needs_improvement = critique_data.get("needs_improvement", True)
# ---- 2. Stop conditions ----
stop_reason = ""
if score >= self.threshold or not needs_improvement:
stop_reason = "threshold_reached"
elif prev_score is not None and score <= prev_score:
stop_reason = "no_improvement"
if stop_reason:
step = ReflectionStep(
round=round_num,
critique=critique_text,
score=score,
improved_answer=current_answer,
stop_reason=stop_reason,
)
result.steps.append(step)
result.rounds_taken = round_num
if self.on_progress:
self.on_progress(round_num, step)
break
# ---- 3. Refine ----
improved = self._refine(
user_question, current_answer, critique_text, context, debug_logs
)
step = ReflectionStep(
round=round_num,
critique=critique_text,
score=score,
improved_answer=improved,
stop_reason="continued",
)
result.steps.append(step)
result.rounds_taken = round_num
if self.on_progress:
self.on_progress(round_num, step)
prev_score = score
current_answer = improved
else:
if result.steps:
result.steps[-1].stop_reason = "max_rounds"
result.final_answer = current_answer
result.improved = current_answer != initial_answer
return result
# -------------------------------------------------------------- #
# Internal helpers #
# -------------------------------------------------------------- #
def _build_debug_block(self, debug_logs: Optional[dict]) -> str:
"""Wandelt execution_results in einen lesbaren Text-Block um."""
if not debug_logs:
return ""
parts = []
if debug_logs.get("stdout", "").strip():
parts.append(f"stdout:\n{debug_logs['stdout'].strip()}")
if debug_logs.get("stderr", "").strip():
parts.append(f"stderr:\n{debug_logs['stderr'].strip()}")
for log in debug_logs.get("logs", []):
parts.append(f"[{log.get('level','?')}] {log.get('message','')}")
return "\n\nDebug / execution output:\n" + "\n".join(parts) if parts else ""
def _critique(
self,
question: str,
answer: str,
context: str,
debug_logs: Optional[dict],
) -> dict:
ctx_block = f"\n\nFull file content:\n{context[:3000]}" if context else ""
dbg_block = self._build_debug_block(debug_logs)
messages = [
{"role": "system", "content": self.CRITIC_SYSTEM},
{
"role": "user",
"content": (
f"User question:\n{question}"
f"{ctx_block}"
f"{dbg_block}\n\n"
f"Assistant answer to evaluate:\n{answer}"
),
},
]
try:
raw = chat_json(self.client, messages, model=self.model, temperature=0.3)
return json.loads(raw)
except Exception:
return {"score": 5.0, "critique": "Evaluation failed.", "needs_improvement": True}
def _refine(
self,
question: str,
prev_answer: str,
critique: str,
context: str,
debug_logs: Optional[dict],
) -> str:
ctx_block = f"\n\nFull file content:\n{context[:3000]}" if context else ""
dbg_block = self._build_debug_block(debug_logs)
if self.agent_loop is not None:
refine_prompt = (
f"[REFLECTION REFINEMENT]\n"
f"The following critique was raised about your last answer:\n"
f"{critique}\n\n"
f"Original user question: {question}"
f"{ctx_block}"
f"{dbg_block}\n\n"
f"Your previous answer:\n{prev_answer}\n\n"
f"Please fix all issues from the critique. "
f"You may use tools (read_file, edit_file, search, execute_code) "
f"if needed to produce a correct and complete answer. "
f"Do NOT mention this refinement process in your answer."
)
try:
return self.agent_loop(refine_prompt, None)
except Exception:
pass # Fallback to chat_json
# ---- Fallback ohne Tool-Zugriff ----
messages = [
{"role": "system", "content": self.REFINE_FALLBACK_SYSTEM},
{
"role": "user",
"content": (
f"Original user question:\n{question}"
f"{ctx_block}"
f"{dbg_block}\n\n"
f"Your previous answer:\n{prev_answer}\n\n"
f"Critique:\n{critique}\n\n"
"Now write your improved answer."
),
},
]
try:
raw = chat_json(self.client, messages, model=self.model, temperature=0.4)
data = json.loads(raw)
return data.get("answer", prev_answer)
except Exception:
return prev_answer

View File

@ -22,7 +22,8 @@ st.markdown("""
body.chat-open .main .block-container { margin-right: 400px; }
iframe[height="1"] { display: none !important; }
[data-testid="stSidebar"] { z-index: 99 !important; }
/* Tool-use badge styling */
/* Tool-use & reflection badge styling */
.tool-badge {
display: inline-block;
background: #1e3a5f;
@ -34,6 +35,18 @@ st.markdown("""
margin: 2px 0 6px 0;
font-family: monospace;
}
.reflect-badge {
display: inline-block;
background: #1e3a1e;
color: #7ef77e;
border: 1px solid #2d9e2d;
border-radius: 6px;
padding: 3px 10px;
font-size: 0.82em;
margin: 2px 0 6px 0;
font-family: monospace;
}
/* Pin chat input to bottom of its column */
[data-testid="stChatInput"] {
position: sticky !important;
@ -45,13 +58,20 @@ st.markdown("""
</style>
""", unsafe_allow_html=True)
def _badge_html(event: str) -> str:
"""Return the appropriate badge HTML for a tool event."""
cls = "reflect-badge" if event.startswith(("🔁", "", "")) else "tool-badge"
return f'<div class="{cls}">{event}</div>'
def main():
# --- Initialize Backend Components ---
file_manager = FileManager()
navigation = FileNavigation()
chat_manager = ChatManager(file_manager=file_manager)
file_manager = FileManager()
navigation = FileNavigation()
chat_manager = ChatManager(file_manager=file_manager)
execution_engine = ExecutionEngine()
output_display = OutputDisplay()
output_display = OutputDisplay()
# --- UI Components ---
chat = ChatInterface(chat_manager=chat_manager)
@ -69,6 +89,11 @@ def main():
if "ai_wrote_file" not in st.session_state: st.session_state.ai_wrote_file = False
if "tool_events" not in st.session_state: st.session_state.tool_events = []
# Reflection state defaults
if "reflect_enabled" not in st.session_state: st.session_state.reflect_enabled = True
if "reflect_rounds" not in st.session_state: st.session_state.reflect_rounds = 3
if "reflect_threshold" not in st.session_state: st.session_state.reflect_threshold = 8.5
# --- Restore active file context on every rerender ---
if st.session_state.get("active_file") and st.session_state.get("editor_content"):
chat_manager.set_active_file(
@ -78,6 +103,13 @@ def main():
else:
chat_manager.set_active_file(None, "")
# Apply current reflection settings
chat_manager.set_reflection(
enabled = st.session_state.reflect_enabled,
max_rounds = st.session_state.reflect_rounds,
threshold = st.session_state.reflect_threshold,
)
# --- Wire up the AI file-write callback ---
def on_file_written(path: str, new_content: str):
if str(path) == str(st.session_state.get("active_file")):
@ -87,7 +119,7 @@ def main():
chat_manager.on_file_written = on_file_written
# --- Wire up tool-use notification ---
# --- Wire up tool-use / reflection notification ---
def on_tool_used(tool_name: str, summary: str):
st.session_state.tool_events.append(summary)
@ -95,13 +127,41 @@ def main():
# ---------------- SIDEBAR ----------------
st.sidebar.markdown("---")
selected_file = navigation.render_sidebar()
if selected_file:
st.session_state.selected_file = selected_file
st.sidebar.markdown("---")
current_file = st.session_state.selected_file
# ---- Reflection controls in sidebar ----
with st.sidebar.expander("🧠 Selbstreflexion", expanded=False):
st.session_state.reflect_enabled = st.toggle(
"Aktiviert",
value=st.session_state.reflect_enabled,
help="Das Modell bewertet und verbessert jede Antwort selbstständig.",
)
st.session_state.reflect_rounds = st.slider(
"Max. Runden",
min_value=1, max_value=5,
value=st.session_state.reflect_rounds,
disabled=not st.session_state.reflect_enabled,
help="Wie oft darf das Modell seine Antwort überarbeiten?",
)
st.session_state.reflect_threshold = st.slider(
"Qualitäts-Schwelle",
min_value=5.0, max_value=10.0, step=0.5,
value=st.session_state.reflect_threshold,
disabled=not st.session_state.reflect_enabled,
help="Score (010) ab dem die Reflexion stoppt.",
)
if st.session_state.reflect_enabled:
st.caption(
f"Modell überarbeitet bis Score ≥ {st.session_state.reflect_threshold:.1f} "
f"oder {st.session_state.reflect_rounds} Runden erreicht."
)
else:
st.caption("Antworten werden ohne Selbstkritik ausgegeben.")
# ---------------- FILE SWITCH ----------------
if current_file != st.session_state.active_file:
st.session_state.active_file = current_file
@ -155,8 +215,6 @@ def main():
if not st.session_state.chat_open:
st.subheader("💬 AI Assistant")
# Scrollable messages area — Streamlit's container(height=) gives it
# a fixed viewport that scrolls, so the input below is always visible.
messages_container = st.container(height=600, border=False)
with messages_container:
messages = st.session_state.chat_messages
@ -170,10 +228,9 @@ def main():
and message.get("tool_events")
):
for event in message["tool_events"]:
st.markdown(f'<div class="tool-badge">{event}</div>', unsafe_allow_html=True)
st.markdown(_badge_html(event), unsafe_allow_html=True)
st.markdown(message["content"])
# Input lives OUTSIDE the scrollable container — always at the bottom
if prompt := st.chat_input("Ask about your code..."):
st.session_state.tool_events = []
st.session_state.chat_messages.append({"role": "user", "content": prompt})
@ -183,12 +240,12 @@ def main():
st.markdown(prompt)
with st.chat_message("assistant"):
with st.spinner("AI is thinking..."):
with st.spinner("AI is thinking"):
response = chat_manager.generate_response(prompt, st.session_state.run_result)
tool_events = list(st.session_state.tool_events)
for event in tool_events:
st.markdown(f'<div class="tool-badge">{event}</div>', unsafe_allow_html=True)
st.markdown(_badge_html(event), unsafe_allow_html=True)
st.markdown(response)
@ -208,6 +265,6 @@ def main():
chat_manager.clear_history()
st.rerun()
if __name__ == "__main__":
main()
# End of main.py