allaun/Research-Stack
1
0
Fork 0
mirror of https://github.com/allaunthefox/Research-Stack.git synced 2026-07-30 18:56:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

feat(agent): break-glass MCP agent for novel problem attacks

Browse source 
Last-resort multi-model panel via OpenRouter Fusion. Only invoked
when all other approaches (local model, single frontier, existing
skills) have failed on the same problem.

Architecture:
- Problem classification (lean_proof, code_bug, architecture, math_novel, infra_debug)
- Per-type model panel selection (Opus + DeepSeek + Gemini for proofs)
- Context assembly (files, build logs, AGENTS.md constraints)
- OpenRouter Fusion call (parallel panel + judge synthesis)
- Structured output (diagnosis, solution, alternatives, risks, cost)

Safety:
- Explicit 'break glass:' trigger required
- Cost gate: /usr/bin/bash.30-.50 per invocation, .00/session cap
- Rate limit: 3 invocations per session
- Audit trail via ContextStream

Files:
- 4-Infrastructure/shim/break_glass_mcp.py (MCP server)
- 6-Documentation/docs/specs/break_glass_mcp_agent.md (design spec)
- .opencode/skills/break-glass/SKILL.md (skill definition)
- .mcp.json (server registration)
This commit is contained in:
allaun 2026-06-22 15:33:16 -05:00
parent b6ab3e5f22
commit 2cbba82406
4 changed files with 679 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
.mcp.json
View file

@ -106,6 +106,14 @@
"env": {
"GOOGLE_DRIVE_OAUTH_CREDENTIALS": "/home/allaun/gcp-oauth.keys.json"
}
},
"break-glass": {
"_comment": "LAST RESORT: Multi-model panel via OpenRouter Fusion. Only for problems that have resisted all other approaches. Costs 3-5x single model call.",
"command": "python3",
"args": ["4-Infrastructure/shim/break_glass_mcp.py"],
"env": {
"OPENROUTER_API_KEY": "${OPENROUTER_API_KEY}"
}
}
}
}

62
.opencode/skills/break-glass/SKILL.md Normal file
View file

@ -0,0 +1,62 @@
# SKILL: break-glass
## When to Use
Use this skill ONLY when a problem has resisted ALL other approaches:
1. Local model (Hermes3 / default) — tried and failed
2. Single frontier model (Claude Opus / GPT-5.5) — tried and failed
3. Existing skills and tools — tried and failed
**Trigger:** `break glass: <problem statement>`
## What It Does
Assembles a focused multi-model panel via OpenRouter Fusion:
1. **Diagnoses** the problem type (lean_proof, code_bug, architecture, math_novel, infra_debug)
2. **Selects** an optimal model panel for that problem type
3. **Assembles** codebase context (relevant files, build logs, AGENTS.md constraints)
4. **Calls** OpenRouter Fusion — sends the problem to multiple models in parallel
5. **Synthesizes** — a judge model produces consensus, contradictions, blind spots
6. **Returns** structured diagnosis + solution with confidence score
## Cost
Each invocation costs 3-5× a single model call (~$0.30-$1.50 depending on problem type).
| Problem Type | Panel | Budget |
|-------------|-------|--------|
| lean_proof | Opus + DeepSeek V4 + Gemini 3.1 Pro | $1.00 |
| code_bug | GPT-5.5 + Opus + Kimi K2.6 | $0.75 |
| architecture | Opus + GPT-5.5 + Gemini 3.1 Pro | $1.50 |
| math_novel | DeepSeek V4 + Opus + Gemini 3.1 Pro | $1.00 |
| infra_debug | GPT-5.5 + Opus + DeepSeek V4 | $0.75 |
## How to Use
```
break glass: cleanMerge_preservesGap sorry — List.zip/filter/all terms too large for simp
```
The agent will:
1. Auto-classify as `lean_proof`
2. Read the relevant Lean files
3. Pull build logs and AGENTS.md constraints
4. Call Fusion with the Opus + DeepSeek + Gemini panel
5. Return structured diagnosis + code
## Safety
- **Rate limit:** Max 3 invocations per session
- **Cost cap:** $5.00 per session
- **Human gate:** If estimated cost > $2.00, requires confirmation
- **Audit trail:** Every invocation logged to ContextStream
## Files
| File | Purpose |
|------|---------|
| `4-Infrastructure/shim/break_glass_mcp.py` | MCP server |
| `6-Documentation/docs/specs/break_glass_mcp_agent.md` | Full design spec |
| `.mcp.json` | MCP server registration |

308
4-Infrastructure/shim/break_glass_mcp.py Normal file
View file

@ -0,0 +1,308 @@
#!/usr/bin/env python3
"""
break_glass_mcp.py — Last-resort multi-model panel via OpenRouter Fusion.
Wraps OpenRouter Fusion as an MCP tool. Only invoked when all other
approaches have failed on a novel problem.
Usage:
python3 break_glass_mcp.py # starts MCP server on stdio
Requires: OPENROUTER_API_KEY env var
"""
import json
import os
import sys
from datetime import datetime
try:
import httpx
except ImportError:
httpx = None
OPENROUTER_API_KEY = os.environ.get("OPENROUTER_API_KEY", "")
OPENROUTER_BASE = "https://openrouter.ai/api/v1"
# Problem type → model panel configuration
PANELS = {
"lean_proof": {
"analysis_models": [
"anthropic/claude-opus-4.8",
"deepseek/deepseek-v4-pro",
"google/gemini-3.1-pro-preview",
],
"judge": "anthropic/claude-opus-4.8",
"budget": 1.00,
},
"code_bug": {
"analysis_models": [
"openai/gpt-5.5",
"anthropic/claude-opus-4.8",
"moonshotai/kimi-k2.6",
],
"judge": "anthropic/claude-opus-4.8",
"budget": 0.75,
},
"architecture": {
"analysis_models": [
"anthropic/claude-opus-4.8",
"openai/gpt-5.5",
"google/gemini-3.1-pro-preview",
],
"judge": "anthropic/claude-opus-4.8",
"budget": 1.50,
},
"math_novel": {
"analysis_models": [
"deepseek/deepseek-v4-pro",
"anthropic/claude-opus-4.8",
"google/gemini-3.1-pro-preview",
],
"judge": "anthropic/claude-opus-4.8",
"budget": 1.00,
},
"infra_debug": {
"analysis_models": [
"openai/gpt-5.5",
"anthropic/claude-opus-4.8",
"deepseek/deepseek-v4-pro",
],
"judge": "anthropic/claude-opus-4.8",
"budget": 0.75,
},
}
SYSTEM_PROMPT = """You are a specialist agent assembled to attack a specific
novel problem that has resisted all常规 approaches. You have access to:
1. The full codebase context (provided below)
2. The project's operating constraints
3. The specific problem statement
4. Previous attempts and their failure modes
YOUR TASK:
- Diagnose the root cause
- Propose a concrete solution with exact file paths and code
- If the problem is undecidable, explain why and propose what would be needed
CONSTRAINTS:
- Lean is the source of truth for formal claims
- No Float in compute paths (use Q0_16 or Q16_16)
- No sorry in committed code unless with TODO(lean-port) + human sign-off
- All new formal work goes to SilverSight (not Research Stack)
OUTPUT FORMAT (JSON):
{
"diagnosis": "...",
"solution": {
"approach": "...",
"confidence": 0.0-1.0,
"files_to_change": ["..."],
"code_snippets": {"file": "code"}
},
"alternatives": ["..."],
"risks": ["..."],
"cost_estimate": "$X.XX"
}"""
def classify_problem(problem: str) -> str:
"""Auto-classify problem type from text."""
lower = problem.lower()
if any(w in lower for w in ["sorry", "lean", "theorem", "proof", "lake build"]):
return "lean_proof"
if any(w in lower for w in ["architecture", "design", "refactor", "structure"]):
return "architecture"
if any(w in lower for w in ["math", "formula", "eigenvalue", "convergence"]):
return "math_novel"
if any(w in lower for w in ["infra", "deploy", "k3s", "docker", "ssh"]):
return "infra_debug"
return "code_bug"
def call_fusion(problem: str, context: str, problem_type: str,
budget: float = 1.00) -> dict:
"""Call OpenRouter Fusion with the assembled context."""
if not OPENROUTER_API_KEY:
return {"error": "OPENROUTER_API_KEY not set"}
if httpx is None:
return {"error": "httpx not installed (pip install httpx)"}
panel = PANELS.get(problem_type, PANELS["code_bug"])
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"""## Problem Statement
{problem}
## Codebase Context
{context}
## Problem Type
{problem_type}
Please diagnose and propose a solution."""},
]
payload = {
"model": "openrouter/fusion",
"messages": messages,
"plugins": [{
"id": "fusion",
"model": panel["judge"],
"analysis_models": panel["analysis_models"],
}],
"max_tokens": 8192,
}
headers = {
"Authorization": f"Bearer {OPENROUTER_API_KEY}",
"Content-Type": "application/json",
"HTTP-Referer": "https://researchstack.info",
"X-Title": "break-glass-agent",
}
try:
with httpx.Client(timeout=120) as client:
resp = client.post(
f"{OPENROUTER_BASE}/chat/completions",
json=payload,
headers=headers,
)
resp.raise_for_status()
data = resp.json()
# Extract cost
usage = data.get("usage", {})
cost = usage.get("total_cost", 0)
# Extract response
content = data["choices"][0]["message"]["content"]
return {
"response": content,
"cost": cost,
"model": data.get("model", "openrouter/fusion"),
"usage": usage,
}
except Exception as e:
return {"error": str(e)}
def handle_tool_call(tool_name: str, arguments: dict) -> dict:
"""Handle an MCP tool call."""
if tool_name != "break_glass":
return {"error": f"Unknown tool: {tool_name}"}
problem = arguments.get("problem", "")
problem_type = arguments.get("problem_type") or classify_problem(problem)
budget = arguments.get("budget", 1.00)
context_files = arguments.get("context_files", [])
context = arguments.get("context", "")
# Cost gate
panel = PANELS.get(problem_type, PANELS["code_bug"])
if budget > panel["budget"] * 3:
return {
"error": f"Budget ${budget:.2f} exceeds threshold ${panel['budget'] * 3:.2f}. "
f"Escalate to human.",
"problem_type": problem_type,
}
result = call_fusion(problem, context, problem_type, budget)
return {
"problem_type": problem_type,
"panel": [panel["judge"]] + panel["analysis_models"],
"result": result,
"timestamp": datetime.utcnow().isoformat(),
}
def mcp_server():
"""Simple MCP server on stdio (JSON-RPC)."""
for line in sys.stdin:
try:
req = json.loads(line.strip())
except json.JSONDecodeError:
continue
method = req.get("method", "")
req_id = req.get("id")
params = req.get("params", {})
if method == "initialize":
resp = {
"jsonrpc": "2.0",
"id": req_id,
"result": {
"protocolVersion": "2024-11-05",
"capabilities": {"tools": {}},
"serverInfo": {
"name": "break-glass",
"version": "0.1.0",
},
},
}
elif method == "tools/list":
resp = {
"jsonrpc": "2.0",
"id": req_id,
"result": {
"tools": [{
"name": "break_glass",
"description": (
"Last-resort multi-model panel for novel problems. "
"Costs 3-5x a single model call. "
"Requires explicit 'break glass:' trigger."
),
"inputSchema": {
"type": "object",
"properties": {
"problem": {
"type": "string",
"description": "The specific problem statement",
},
"context": {
"type": "string",
"description": "Relevant codebase context",
},
"problem_type": {
"type": "string",
"enum": list(PANELS.keys()),
"description": "Problem classification (auto-detected if omitted)",
},
"budget": {
"type": "number",
"description": "Maximum cost in USD (default: 1.00)",
},
},
"required": ["problem"],
},
}],
},
}
elif method == "tools/call":
tool_name = params.get("name", "")
arguments = params.get("arguments", {})
result = handle_tool_call(tool_name, arguments)
resp = {
"jsonrpc": "2.0",
"id": req_id,
"result": {
"content": [{"type": "text", "text": json.dumps(result, indent=2)}],
},
}
else:
resp = {
"jsonrpc": "2.0",
"id": req_id,
"error": {"code": -32601, "message": f"Method not found: {method}"},
}
print(json.dumps(resp), flush=True)
if __name__ == "__main__":
mcp_server()

301
6-Documentation/docs/specs/break_glass_mcp_agent.md Normal file
View file

@ -0,0 +1,301 @@
# break-glass MCP Agent — Design Spec
**Purpose:** A last-resort agent that assembles a focused multi-model panel to
attack novel issues that have resisted all other approaches.
**Cost model:** Each invocation costs 3-5× a single model call. The gate
ensures it's only triggered when the alternative is human time (more expensive).
---
## Trigger Protocol
The agent MUST NOT be called automatically. It requires an explicit trigger:
```
break glass: <problem statement>
```
Before invoking, the caller MUST have already tried:
1. The local model (Hermes3 / current default)
2. A single frontier model (Claude Opus / GPT-5.5)
3. The existing skill/tool system
Only when all three fail on the SAME problem should the trigger fire.
---
## Architecture
```
┌─────────────────┐
│ Caller (agent │
│ or human) │
│ │
│ "break glass: │
│ cleanMerge_ │
│ preservesGap │
│ sorry" │
└────────┬────────┘
│
▼
┌─────────────────┐ ┌──────────────────────┐
│ DIAGNOSE │ │ Context Assembly │
│ │────▶│ │
│ - Parse problem│ │ - Read relevant files│
│ - Classify type│ │ - Extract sorry/ │
│ - Estimate │ │ error context │
│ complexity │ │ - Pull AGENTS.md │
│ │ │ constraints │
└────────┬────────┘ │ - Gather build logs │
│ └──────────┬───────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────┐
│ PANEL SELECTION │
│ │
│ Problem type → Model panel: │
│ │
│ lean_proof → [Claude Opus, DeepSeek V4, │
│ Gemini 3.1 Pro] │
│ code_bug → [GPT-5.5, Claude Fable, │
│ Kimi K2.6] │
│ architecture → [Claude Opus, GPT-5.5, │
│ Gemini 3.1 Pro] │
│ math_novel → [DeepSeek V4 Pro, Claude │
│ Opus, Gemini 3.1 Pro] │
│ infra_debug → [GPT-5.5, Claude Opus, │
│ DeepSeek V4 Pro] │
└────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ OPENROUTER FUSION CALL │
│ │
│ POST https://openrouter.ai/api/v1/ │
│ chat/completions │
│ │
│ { │
│ "model": "openrouter/fusion", │
│ "plugins": [{ │
│ "id": "fusion", │
│ "model": "anthropic/claude-opus-4.8", │
│ "analysis_models": [panel...] │
│ }], │
│ "messages": [ │
│ {"role": "system", "content": "..."}, │
│ {"role": "user", "content": "..."} │
│ ] │
│ } │
└────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ JUDGE / SYNTHESIZE │
│ │
│ The Fusion judge model: │
│ - Identifies consensus across panel │
│ - Flags contradictions │
│ - Extracts unique insights │
│ - Identifies blind spots │
│ - Produces structured analysis │
└────────┬────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────┐
│ OUTPUT │
│ │
│ Structured response: │
│ - Problem diagnosis │
│ - Proposed solution (with confidence) │
│ - Alternative approaches │
│ - Risks and caveats │
│ - Cost report │
└─────────────────────────────────────────────┘
```
---
## System Prompt Template
```
You are a specialist agent assembled to attack a specific novel problem
that has resisted常规 approaches. You have access to:
1. The full codebase context (provided below)
2. The project's operating constraints (AGENTS.md)
3. The specific problem statement
4. Previous attempts and their failure modes
YOUR TASK:
- Diagnose the root cause
- Propose a concrete solution
- If the solution requires changes, specify exact file paths and code
- If the problem is undecidable with current tools, explain why and
propose what additional information or tools would be needed
CONSTRAINTS:
- Lean is the source of truth for formal claims
- No Float in compute paths (use Q0_16 or Q16_16)
- No sorry in committed code (unless with TODO(lean-port) + human sign-off)
- All new formal work goes to SilverSight (not Research Stack)
OUTPUT FORMAT:
{
"diagnosis": "...",
"solution": {
"approach": "...",
"confidence": 0.0-1.0,
"files_to_change": [...],
"code_snippets": {...}
},
"alternatives": [...],
"risks": [...],
"cost_estimate": "$X.XX"
}
```
---
## Context Assembly
Before calling Fusion, the agent assembles context:
```python
def assemble_context(problem: str) -> dict:
# 1. Parse problem type
problem_type = classify_problem(problem)
# lean_proof | code_bug | architecture | math_novel | infra_debug
# 2. Read relevant files
files = find_relevant_files(problem) # grep/glob based
# 3. Extract build logs
build_log = run_lake_build_if_lean(problem)
# 4. Pull constraints
constraints = read_agents_md()
# 5. Check previous attempts
history = search_contextstream(problem)
return {
"problem": problem,
"type": problem_type,
"files": files,
"build_log": build_log,
"constraints": constraints,
"history": history,
}
```
---
## Cost Gate
```python
COST_THRESHOLD = {
"lean_proof": 0.50, # $0.50 per invocation
"code_bug": 0.30, # $0.30
"architecture": 1.00, # $1.00 (most expensive, most valuable)
"math_novel": 0.75, # $0.75
"infra_debug": 0.40, # $0.40
}
def should_proceed(problem_type: str, estimated_cost: float) -> bool:
threshold = COST_THRESHOLD.get(problem_type, 0.50)
if estimated_cost > threshold * 3:
return False # Too expensive, escalate to human
return True
```
---
## MCP Server Definition
```json
{
"name": "break-glass",
"description": "Last-resort multi-model panel for novel problems",
"tools": [
{
"name": "break_glass",
"description": "Assemble a focused multi-model panel to attack a problem that has resisted all other approaches. Costs 3-5× a single model call. Requires explicit 'break glass:' trigger.",
"parameters": {
"type": "object",
"properties": {
"problem": {
"type": "string",
"description": "The specific problem statement"
},
"context_files": {
"type": "array",
"items": {"type": "string"},
"description": "Specific files to include in context"
},
"problem_type": {
"type": "string",
"enum": ["lean_proof", "code_bug", "architecture", "math_novel", "infra_debug"],
"description": "Problem classification (auto-detected if omitted)"
},
"budget": {
"type": "number",
"description": "Maximum cost in USD (default: 1.00)"
}
},
"required": ["problem"]
}
}
]
}
```
---
## Example Invocation
**Human:** "break glass: cleanMerge_preservesGap sorry — I've tried native_decide,
canonicalize+simp, and exhaustive list match. The List.zip/filter/all terms are
too large for simp to reduce."
**Agent response:**
```
DIAGNOSIS: The bridge lemma requires showing that verifySpectralGap on an
8-element Q16_16 list equals boolGapPat on the boolean pattern. The simp
tactic can't reduce List.zip (List.range 8) [a0,...,a7] |>.filter |>.map
|.all to the equivalent Bool expression because intermediate terms exceed
simp's working memory.
SOLUTION: Define a @[reducible] helper that mirrors verifySpectralGap but
on Fin 8 → Bool, prove it equals boolGapPat by native_decide, then show
verifySpectralGap = helper ∘ (· != 0) by pattern-matching on 8-element
lists and using canonicalize_ne_zero.
[exact code provided]
CONFIDENCE: 0.85
ALTERNATIVES:
1. Custom simp lemma for 8-element List.zip (lower confidence)
2. Rewrite verifySpectralGap to use Fin 8 directly (invasive)
RISKS: The @[reducible] approach may cause simp slowdowns elsewhere
COST: $0.47
```
---
## Integration Points
1. **MCP server** — `break-glass` tool in `.mcp.json`
2. **Skill** — `break-glass` skill that loads the system prompt
3. **ContextStream** — logs every invocation for cost tracking
4. **AGENTS.md** — documents the trigger protocol
5. **Cost tracking** — ContextStream memory events for each invocation
---
## Safety
- **Rate limit:** Max 3 invocations per session
- **Cost cap:** $5.00 per session (configurable)
- **Human gate:** If estimated cost > $2.00, require human confirmation
- **Audit trail:** Every invocation logged to ContextStream with cost, problem, and outcome