mirror of
https://github.com/allaunthefox/Research-Stack.git
synced 2026-07-31 03:05:21 +00:00
11 KiB
11 KiB
Python → Lean Migration Plan
Policy (AGENTS.md §0): Lean is the source of truth. Everything else is a shim. All Python code must move to Lean unless there is an explicit, documented reason it cannot.
Generated: 2026-04-22 Auditor: Cascade (via static review + grep pass)
§0 AGENTS.md Compliance Gates
Every migration must clear all of the following — per-task reviewer checklist:
| Gate | Rule | Source |
|---|---|---|
| G1 | No new dependencies (crates / pip / lake) | §1.1 |
| G2 | No Float in new core logic — use Q16_16 |
§1.4 |
| G3 | No open string matching in decisions — use Fin n / inductives |
§1.5 |
| G4 | No sorry in committed code |
§1.6 |
| G5 | No helper/utility files — every file is a domain module | §1.8 |
| G6 | PascalCase.lean file, camelCase functions, PascalCase types |
§2 |
| G7 | Every cost/invariant def has an #eval or theorem witness |
§4 |
| G8 | Logic expressible as a bind instance (one of 5 allowed classes) |
§4 (bind) |
| G9 | lake build passes with zero warnings |
§5.3 |
| G10 | Deletions require explicit human sign-off | §7.1, §8 |
Violations are reverted. If a migration cannot satisfy G1–G9, stop and ask before proceeding (§1.7).
Summary
| Bucket | File Count | Disposition |
|---|---|---|
| Keep in Python (external-boundary) | ~40 | Shim layer only |
| Delete — already has identical Lean counterpart | 4 | Remove immediately |
| Migrate to Lean — pure logic | ~55 (scripts) + ~10 (infra) | Swarm work |
| Migrate to Lean — mixed, needs split | ~20 | Swarm with human review |
| Scratch / sandbox | 344 (scratch/) | Out of scope, keep or gitignore |
Core Python kept: ~40 files (HTTP servers, DB drivers, OAuth clients, subprocess wrappers, hardware bridges).
§1 Explicit "Keep in Python" Criteria
A file stays in Python only if all of the following are true:
- Hard external boundary — talks to an OS/network/hardware interface Lean cannot reach directly (HTTP server, SQLite, rclone subprocess, OAuth, POSIX signals, GPU runtime, etc.).
- No business logic — logic must already live in Lean; Python is a thin transport/marshaling shim.
- Documented reason — file header names the boundary (e.g.
# BOUNDARY: FastAPI HTTP listener; logic in Semantics.SwarmQueryAPI).
If the file has logic and IO, split it: logic → Lean, IO → Python shim calling the Lean binary via lean --run or FFI.
§2 Immediate Deletions (identical Lean already exists)
These Python files have complete, registered Lean counterparts:
| Python File | Lean Replacement | Lines Removed | Status |
|---|---|---|---|
infra/omnidirectional_interface.py |
Semantics.OmnidirectionalInterface |
752 | Delete |
infra/gpu_duty_assignment.py |
Semantics.GpuDutyAssignment |
439 | Delete |
infra/rclone_integration.py |
Semantics.RcloneIntegration |
537 (keep thin IO shim) | Split |
infra/domain_model_integration.py |
Semantics.DomainModelIntegration |
442 (keep thin IO shim) | Split |
tools/swarm_api.py |
Semantics.SwarmQueryAPI + FastAPI shim |
335 (keep ~60 line shim) | Split |
Action (requires human sign-off per §7.1/§8):
- Swarm posts a deletion request listing each file + grep proof that nothing in-tree imports it.
- On approval, files are deleted and replaced with a ≤ 80-line FastAPI/subprocess shim that only marshals JSON into the Lean binary.
- Shim bodies contain no cost functions, invariants, or branching on semantic content (§6.1).
§3 Swarm Work Queue (Prioritized)
Each task is tagged with a lead role and estimated size.
§3.1 Wave 1 — Pure-Logic Migration (no IO)
These 18 scripts produce JSON reports from hand-authored cost models / designs. They have no IO dependencies beyond json.dump and deserve Lean modules with theorems.
| # | Python Source | Target Lean Module | Lead Role |
|---|---|---|---|
| 1 | scripts/swarm_design_nextgen_agents.py |
Semantics.NextGenAgentDesign |
Type System Architect |
| 2 | scripts/swarm_genetic_groundup_redesign.py |
(done: Semantics.GeneticGroundUp) |
— |
| 3 | scripts/swarm_review_genetic_groundup.py |
Semantics.SwarmCodeReview |
Formal Verification |
| 4 | scripts/swarm_gene_bytecode_jit.py |
Semantics.GeneBytecodeJIT |
Compiler Theory |
| 5 | scripts/swarm_math_debate.py |
Semantics.MathDebate |
Logic |
| 6 | scripts/swarm_competition.py |
Semantics.SwarmCompetition |
Game Theory |
| 7 | scripts/swarm_topology_integration.py |
Semantics.SwarmTopology |
Topology |
| 8 | scripts/swarm_topology_optimizer.py |
Semantics.TopologyOptimizer |
Topology |
| 9 | scripts/swarm_nonstandard_web_interfaces.py |
Semantics.NonStandardInterfaces |
Interface |
| 10 | scripts/tsm_swarm_efficiency_optimization.py |
Semantics.TSMEfficiency |
Perf Analysis |
| 11 | scripts/virtual_gpu_real_benchmark_fast.py |
Semantics.VirtualGPUBenchmark |
Perf Analysis (honest toy-cost framing) |
| 12 | scripts/virtual_gpu_workload_testbench.py |
Semantics.WorkloadTestbench |
Perf Analysis |
| 13 | scripts/virtual_gpu_testbench.py |
Semantics.VirtualGPUTestbench |
Perf Analysis |
| 14 | scripts/virtual_gpu_topology_loader.py |
Semantics.VirtualGPUTopology |
Topology |
| 15 | scripts/combined_resource_layers.py |
Semantics.ResourceLayers |
Resource |
| 16 | scripts/swarm_network_capacity.py |
Semantics.NetworkCapacity |
Resource |
| 17 | scripts/q_factor.py |
(likely subset of Semantics.QFactor) |
Verify + delete |
| 18 | scripts/efficiency_analysis.py |
Semantics.EfficiencyAnalysis |
Perf |
Rule for Wave 1: No theorem may overclaim. All cost/speedup claims must be framed as toyXxxCostRatio per the GeneticGroundUpBenchmark review.
§3.2 Wave 2 — ask_swarm_* → SwarmQueryAPI calls
All scripts/ask_swarm_*.py files are query routers that should go through the just-built Semantics.SwarmQueryAPI.
| # | Python Source | Migration |
|---|---|---|
| 1 | ask_swarm_tsm.py |
Build SwarmQueryRequest for subject "tsm" |
| 2 | ask_swarm_gossip_sync.py |
subject "gossip" |
| 3 | ask_swarm_gpu_translation_surface.py |
subject "gpu" → routes to Subsystem.gpuDuty |
| 4 | ask_swarm_kimi_optimization.py |
subject "kimi" |
| 5 | ask_swarm_self_solving_space.py |
subject "self-solving" |
| 6 | ask_swarm_virtual_gpu_limits.py |
subject "virtual-gpu" |
| 7 | ask_swarm_web_interaction_surface.py |
subject "web" |
| 8 | ask_swarm_hotload_kimi.py |
subject "hotload" |
Deliverable: Replace all 8 scripts with a single scripts/ask_swarm.py (≤ 30 lines) that calls the Lean query binary. Lead: Formal Verification.
§3.3 Wave 3 — Infra split (logic vs IO)
Each of these infra modules has substantial logic mixed with IO. Split them.
| Python File | Logic → Lean | IO → Python shim (≤ 100 lines) |
|---|---|---|
infra/ene_api.py |
Semantics.ENEApi (security, key derivation, envelope format) |
HTTP routes |
infra/ene_distributed_node.py |
Semantics.ENEDistributedNode (gossip protocol, consensus) |
Transport (asyncio) |
infra/ene_cloud_credential_manager.py |
Semantics.ENECredentialEnvelope (encrypt/decrypt spec) |
Disk IO + rclone call |
infra/hyperbolic_encoding.py |
Semantics.HyperbolicEncoding (math) |
None (pure — migrate fully) |
infra/moe_ene_cache.py |
Semantics.MoECache (eviction policy) |
LRU container |
infra/swarm_ene_middleware.py |
Semantics.SwarmENEMiddleware (routing rules) |
HTTP bridge |
infra/web_interaction_surface.py |
Semantics.WebInteractionSurface (protocol spec) |
HTTP client |
infra/ascii_art_competition.py |
Semantics.ASCIIArtCompetition (scoring) |
File IO |
infra/ascii_art_store.py |
Semantics.ASCIIArtStore (layout/diff) |
Disk write |
infra/gemma_4_integration.py |
Semantics.GemmaIntegration (prompt routing) |
LLM HTTP client |
§3.4 Keep in Python (documented shims only)
| Python File | Reason Kept |
|---|---|
infra/lean_shim.py |
Required — marshals Python ↔ Lean. Cannot self-replace. |
tools/swarm_api.py → reduce to ~60-line shim |
FastAPI listener only |
scripts/deploy_ene_full_mesh.py |
SSH/subprocess mesh deploy |
scripts/ingest_*.py (3 files) |
File system watch + HTTP fetch |
scripts/test_*.py (9 files) |
Pytest integration tests against live components |
Data-import scripts (extract_math_from_dumps.py, materials_data_ingestion.py, etc.) |
Parse binary/JSON dumps |
§4 Deliverables per Swarm Agent
Each Wave 1 task deliverable must include:
- New Lean module at
0-Core-Formalism/lean/Semantics/Semantics/<Name>.lean - Registration in
Semantics.leanimports - ≥ 2 theorems about module invariants (not restatements of hypotheses)
- No
sorryin proofs (or explicit TODO with rationale) - Honest naming — any speedup/cost claim prefixed
toyif not empirically backed - Deletion of the source Python script (or reduction to a ≤ 30-line shim)
- Update
docs/PYTHON_TO_LEAN_MIGRATION_PLAN.mdstatus column
§5 Review Gate
All migrations pass through Semantics.SwarmCodeReview (see Wave 1 task 3) before the Python file is deleted:
- Builder (ADD clock): Lean module compiles, theorems type-check
- Warden (SUBTRACT clock): No
sorry, no hypothesis-restatement, claims match content - Judge (PAUSE clock): Assigns accept/reject
§6 Progress Tracker
| Wave | Total | Done | In Progress | Queued |
|---|---|---|---|---|
| Wave 0 — duplicates | 5 | 5 (unified shim: lean_unified_shim.py replaces omnidirectional_interface.py, gpu_duty_assignment.py, rclone_integration.py, domain_model_integration.py, swarm_api.py) | 0 | 0 |
| Wave 1 — pure logic | 18 | 18 (GeneticGroundUp, SwarmCodeReview, NextGenAgentDesign, GeneBytecodeJIT, MathDebate, SwarmCompetition, SwarmTopology, TopologyOptimization, NonStandardInterfaces, TSMEfficiency, VirtualGPUBenchmark, WorkloadTestbench, VirtualGPUTestbench, VirtualGPUTopology, ResourceLayers, NetworkCapacity, QFactor (already exists as Lean shim), EfficiencyAnalysis) | 0 | 0 |
| Wave 2 — ask_swarm_* | 8 | 8 (unified_swarm_query.py consolidates 8 ask_swarm_*.py scripts) | 0 | 0 |
| Wave 3 — infra split | 10 | 10 (ENEApi.lean, MoECache.lean, ASCIIArtCompetition.lean, ASCIIArtStore.lean, SwarmENEMiddleware.lean, HyperbolicEncoding.lean, WebInteractionSurface.lean, GemmaIntegration.lean, ENECredentialEnvelope.lean, ENEDistributedNode.lean) | 0 | 0 |
| Total migrations | 41 | 41 | 0 | 0 |
Scratch files under scratch/ (344 py) are excluded; they are not production code.