Research-Stack/6-Documentation/docs/protocols/TM_MCP_SPECIFICATION.md
Brandon Schneider 6679908f7b fix(arch): enforce Lean-first hierarchy across project — remove all secondary-Lean language
Project-wide sweep to find and fix every place Lean was treated as secondary,
optional, or subordinate to Python/Rust. The invariant: Lean is the source of
truth. Python and Rust are extraction targets only; they contain no logic, no
invariant checks, no decisions.

Changes:

1-Distributed-Systems/ene/src/lib.rs
  - CRITICAL: Remove 'Rust is the canonical implementation language for
    operational components' — replace with correct extraction-target framing

1-Distributed-Systems/agents/claw/README.md
  - Remove 'canonical implementation lives in rust/' and 'source of truth is
    ultraworkers/claw-code' blanket claims — scope to CLI binary only;
    add Research Stack domain-logic note (Lean is source of truth per AGENTS.md)
  - 'canonical Rust workspace' → 'Rust workspace (I/O extraction target)'

1-Distributed-Systems/agents/claw/src/Tool.py
  - 'Python-first porting summary' → 'Lean-to-Python extraction summary'

1-Distributed-Systems/agents/claw/src/projectOnboardingState.py
  - python_first: bool = True → lean_first: bool = True (Lean always leads)

6-Documentation/docs/specs/ENE_MEMORY_ATLAS_SPEC.md
  - CRITICAL: 'Python first (reference) ... Lean-formal next' → correct order:
    Lean specification first, Python extraction shim, Verilog hardware extraction

6-Documentation/docs/recovered/geocognition_equation_types_map.mmd
  - 'Lean owns logic; Rust owns boundary' → 'Lean owns all logic and decisions;
    Rust is boundary shim only'

6-Documentation/docs/semantics/HYPER_DIMENSIONAL_PHYSICS_INTRO.md
  - 'Python implementation ... Lean formalization' → 'Lean specification (source
    of truth) ... Python extraction shim'

6-Documentation/docs/geometry/GEOMETRY_TAXONOMY_FOR_NLOCAL_ADAPTATION.md
  - 'reference specification for the Python implementation' → 'source of truth;
    Python is an extraction shim against this spec'

6-Documentation/docs/protocols/TM_MCP_SPECIFICATION.md
  - 'Python reference implementation' → 'Python extraction shim' (×2)

5-Applications/scripts/snn/README.md
  - 'deterministic Python reference' → 'Python extraction shim / golden-vector
    harness'; add TODO(lean-port) note; RTL must match 'Python shim (pending
    Lean golden vector)' not 'Python reference'

6-Documentation/docs/METAPROBE_APPROACH.md
  - 'DeltaGCLCompression.lean — Lean implementation / scripts/delta_gcl_encoder.py
    — Python reference implementation' → Lean is source of truth / Python is
    extraction shim

Generated with Devin (https://cli.devin.ai/docs)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-26 22:40:03 -05:00

1144 lines
48 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# TotalMath Multimodal Compression Protocol (TM-MCP)
**Protocol Version:** 1.0.0-alpha
**Date:** 2026-05-02
**Status:** Specification Draft
**Compiler Task:** Extract shared encoding invariants from heterogeneous mathematical/biological/neural/geometric/symbolic channels
---
## Executive Summary
TM-MCP is a protocol compiler that transforms heterogeneous information channels—neural spikes, geometric embeddings, biological concentrations, symbolic expressions, electrical potentials, temporal sequences—into a unified, invariant-preserving compression and transport grammar.
**Core Design Rule:**
```
source_state → canonical_atom → delta → compressed_packet → verified_reconstruction
```
**Core Invariant:**
```
∀x ∈ ChannelData: decode(encode(x)) preserves Invariants(x) within ErrorBudget(x)
```
**Key Innovation:** Channel-agnostic intermediate representation (IR) that carries modality-specific invariants through the compression pipeline without loss of semantic or topological fidelity.
---
## 1. Architecture Diagram (Text)
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ TM-MCP ARCHITECTURE │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ INPUT LAYER CANONICAL LAYER TRANSPORT │
│ ┌──────────────┐ ┌──────────────┐ ┌────────┐ │
│ │ Neural │──spike_train──▶│ Canonical │──atom_stream──▶│ Packet │ │
│ │ Channel │ │ Atom IR │ │ Encoder│ │
│ └──────────────┘ └──────────────┘ └───┬────┘ │
│ ┌──────────────┐ │ │ │
│ │ Geometric │──manifold──────────┤ │ │
│ │ Channel │ ▼ ▼ │
│ └──────────────┘ ┌──────────────┐ ┌──────────┐│
│ ┌──────────────┐ │ Delta │ │ Transport││
│ │ Symbolic │──expression───▶│ Extractor │──delta_seq────▶│ Layer ││
│ │ Channel │ └──────────────┘ │ (MNN) ││
│ └──────────────┘ │ └────┬─────┘│
│ ┌──────────────┐ ▼ │ │
│ │ Biological │──concentration──▶│ Compression │ │ │
│ │ Channel │ │ (DeltaGCL) │◄────receipt──────┘ │
│ └──────────────┘ └──────────────┘ │
│ ┌──────────────┐ │ │
│ │ Temporal │──timestamp──────▶│ Verification│◄────invariant_check │
│ │ Channel │ │ Layer │ │
│ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────────────────────────────────────────────────────────────┐ │
│ │ ROUTING LAYER (MNN) │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌────────────┐ │ │
│ │ │ LOCAL │ │ ATLAS/ │ │ REJECT │ │ RECOVER │ │ │
│ │ │ PROCESS │ │ GLOBAL │ │ (buffer) │ │ (retry) │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ └────────────┘ │ │
│ │ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ ATTEST │ │ DEFER │ Goal: health/compress/route/recover│ │
│ │ │ (validate) │ │ (queue) │ Constraints: mem/bw/latency/trust │ │
│ │ └─────────────┘ └─────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## 2. Canonical Type System
### 2.1 Base Types (Fixed-Point)
```rust
// Fixed-point scalar types
pub type Q0_8 = u8; // Pure fraction: [0.0, 0.99609375], precision = 1/256
pub type Q0_16 = u16; // Pure fraction: [0.0, 0.99998474], precision = 1/65536
pub type Q16_16 = u32; // Mixed: [-32768.0, 32767.99998], precision = 1/65536
pub type Q0_32 = u32; // High-precision fraction
pub type Q0_64 = u64; // Ultra-precision fraction
// Temporal types
pub type Timestamp = u64; // Nanoseconds since epoch (Q64.0)
pub type Duration = u64; // Nanosecond duration
pub type Phase = Q0_16; // [0.0, 1.0) cyclic phase
// Topological types
pub type ManifoldCoord = [Q16_16; 3]; // 3D position in manifold space
pub type BraidIndex = u16; // Braid group generator index
pub type Quaternion = [Q0_16; 4]; // Unit quaternion (normalized)
```
### 2.2 Canonical Atom IR
```rust
/// The universal intermediate representation for all modalities
pub enum CanonicalAtom {
// Neural modalities
SpikeEvent {
timestamp: Timestamp,
channel_id: u32,
amplitude: Q0_16, // Normalized spike amplitude
width: Q0_16, // Spike duration
polarity: SpikePolarity,
phase: Phase, // Oscillatory phase at spike time
},
// Geometric modalities
ManifoldPoint {
coord: ManifoldCoord,
mass: Q0_16, // Weight/importance
torsion: Q0_16, // Torsional stress (FAMM)
shell_index: u32, // PIST shell coordinate
},
BraidCrossing {
index: BraidIndex,
sign: i8, // +1 or -1 (Artin generator)
timestamp: Timestamp,
},
QuaternionState {
q: Quaternion,
angular_velocity: [Q0_16; 3],
layer: u8, // Matroska brane layer
},
// Symbolic modalities
SymbolicTerm {
hash: [u8; 32], // SHA-256 of normalized expression
complexity: Q0_16, // Normalized complexity score
dependencies: Vec<u32>, // Referenced atom IDs
},
// Biological modalities
ConcentrationDelta {
species_id: u16, // Chemical species identifier
delta: Q16_16, // Signed concentration change
compartment: Compartment,
diffusion_coeff: Q0_16,
},
MembranePotential {
voltage: Q16_16, // Millivolts in Q16_16
ion_channel_state: u16, // Bitfield of channel states
timestamp: Timestamp,
},
// Temporal modalities
TemporalWindow {
start: Timestamp,
duration: Duration,
admissibility: Q0_16, // TVI-style admissibility score
},
// Routing metadata
RoutingIntent {
goal: OperationGoal, // health, compress, route, recover, attest
priority: Q0_16,
constraints: RoutingConstraints,
},
}
pub enum SpikePolarity { Positive, Negative, Biphasic }
pub enum Compartment { Cytoplasm, Nucleus, Mitochondrion, ER, Extracellular }
pub enum OperationGoal { Health, Compress, Route, Recover, Attest }
pub struct RoutingConstraints {
pub max_latency_ms: u32,
pub min_trust_score: Q0_16,
pub max_energy_cost: Q0_16,
pub encryption_required: bool,
pub recovery_mode: bool,
}
```
### 2.3 Channel Registry
```rust
/// Registered channel types with invariant profiles
pub enum ChannelType {
// MVP Channels (Phase 1)
SymbolicText, // Delta-encoded UTF-8, normalized expressions
SpikeEvent, // Neural spike trains, temporal coding
GeometricShape, // Manifold embeddings, quaternions, braids
// Phase 2 Channels
BiologicalConc, // Chemical concentration gradients
ElectricalPot, // Membrane voltages, EOD, electroreception
MagneticField, // Magnetoreception data
ThermalGradient, // Infrared, temperature fields
// Phase 3 Channels
Vibrational, // Substrate-borne signals
VisualPattern, // Chromatophore states, bioluminescence
GeneticSequence, // DNA/RNA encodings, BioBrick parts
// Meta channels
RoutingControl, // MNN routing decisions
Verification, // Receipts, attestations, proofs
}
/// Per-channel invariant specifications
pub struct ChannelSpec {
pub channel: ChannelType,
pub precision_tier: PrecisionTier, // Q0_8, Q0_16, Q0_32, Q0_64
pub timing_constraint: TimingConstraint,
pub topology_preservation: TopologyPreservation,
pub compression_ratio_target: Q0_16, // e.g., 0.1 = 10x compression
pub max_reconstruction_error: Q0_16,
}
pub enum PrecisionTier { Q0_8, Q0_16, Q0_32, Q0_64 }
pub enum TimingConstraint { HardRealtime, SoftRealtime, BestEffort }
pub enum TopologyPreservation { Exact, Homotopy, Homeomorphism, None }
```
---
## 3. Packet Format
### 3.1 Transport Packet Structure
```
┌─────────────────────────────────────────────────────────────────┐
│ TM-MCP PACKET │
├─────────────────────────────────────────────────────────────────┤
│ HEADER (fixed 32 bytes) │
│ ┌──────────────┬──────────────┬──────────────────────────────┐│
│ │ Version │ Type │ Flags ││
│ │ u8 = 0x01 │ u8 │ u16 ││
│ ├──────────────┼──────────────┼──────────────────────────────┤│
│ │ Channel ID │ Sequence Num │ Timestamp (ns) ││
│ │ u16 │ u32 │ u64 ││
│ ├──────────────┴──────────────┴──────────────────────────────┤│
│ │ Source Node ID (8 bytes) ││
│ │ Destination Node ID (8 bytes) ││
│ └──────────────────────────────────────────────────────────────┘│
│ │
│ ROUTING META (16 bytes) │
│ ┌──────────────┬──────────────┬──────────────┬──────────────┐│
│ │ Goal │ Trust Score │ Mem Budget │ Bandwidth ││
│ │ u8 │ Q0_16 │ u16 (KB) │ u16 (KB/s) ││
│ ├──────────────┼──────────────┼──────────────┼──────────────┤│
│ │ Latency Ms │ Hop Count │ Reserved │ Checksum ││
│ │ u16 │ u8 │ u8 │ u16 ││
│ └──────────────┴──────────────┴──────────────┴──────────────┘│
│ │
│ PAYLOAD (variable, max 65,536 bytes) │
│ ┌────────────────────────────────────────────────────────────┐│
│ │ Atom Count (u16) ││
│ │ Compressed Delta Atoms... ││
│ │ [CanonicalAtom::delta_encode()] ││
│ └────────────────────────────────────────────────────────────┘│
│ │
│ VERIFICATION TRAILER (32 bytes) │
│ ┌────────────────────────────────────────────────────────────┐│
│ │ Compression Receipt: ratio, error, invariant checksum ││
│ │ Topology Receipt: barcode hash, persistence diagram ││
│ │ Timing Receipt: admissibility window, phase alignment ││
│ │ Routing Receipt: path taken, hop latencies ││
│ └────────────────────────────────────────────────────────────┘│
│ │
│ FOOTER (8 bytes) │
│ ┌──────────────────────────────┬──────────────────────────────┐│
│ │ Payload CRC32 │ Total Packet CRC32 ││
│ │ u32 │ u32 ││
│ └──────────────────────────────┴──────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
```
### 3.2 Delta Encoding for Canonical Atoms
```rust
/// Delta-encoded atom for compression
pub struct DeltaAtom {
pub atom_type: u8, // CanonicalAtom discriminant
pub delta_flags: u16, // Which fields are delta-encoded
pub base_reference: u32, // Reference to previous atom (0 = absolute)
// Delta fields (variable presence based on delta_flags)
pub timestamp_delta: Option<i64>, // Nanoseconds from base
pub coord_delta: Option<[i32; 3]>, // Q16_16 deltas
pub amplitude_delta: Option<i16>, // Q0_16 delta
pub id_delta: Option<i32>, // Channel/atom ID delta
}
/// Delta GCL-style rule-based compression
pub struct DeltaGCLPayload {
pub base_atoms: Vec<CanonicalAtom>, // Keyframe atoms (absolute)
pub delta_sequence: Vec<DeltaAtom>, // Delta-encoded stream
pub rule_applied: DeltaRule, // Which compression rule used
}
pub enum DeltaRule {
Identity, // No compression
TemporalDelta, // Time-series delta encoding
SpatialDelta, // Coordinate delta encoding
TopologicalDelta, // Manifold coordinate delta
SymbolicDelta, // Expression tree diff
HybridDelta, // Multi-modal combined
}
```
---
## 4. Compression Pipeline (6 Stages)
### Stage 1: Normalize Source Modality
```rust
/// Convert any channel input to canonical atoms
fn normalize_channel(
channel: ChannelType,
raw_data: &[u8],
metadata: &ChannelMetadata
) -> Result<Vec<CanonicalAtom>, NormalizationError> {
match channel {
ChannelType::SymbolicText => {
// Parse text → normalized AST → symbolic atoms
let ast = parse_expression(raw_data)?;
let normalized = normalize_ast(ast);
Ok(vec![CanonicalAtom::SymbolicTerm {
hash: sha256(&normalized),
complexity: compute_complexity(&normalized),
dependencies: extract_dependencies(&normalized),
}])
}
ChannelType::SpikeEvent => {
// Parse spike train → temporal atoms
parse_spike_train(raw_data).map(|spikes| {
spikes.into_iter().map(|s| CanonicalAtom::SpikeEvent {
timestamp: s.time_ns,
channel_id: s.electrode_id,
amplitude: Q0_16::from_f32(s.amplitude),
width: Q0_16::from_f32(s.duration_ms / 10.0),
polarity: s.polarity.into(),
phase: compute_phase(s.time_ns, metadata.reference_frequency),
}).collect()
})
}
ChannelType::GeometricShape => {
// Parse manifold embedding → geometric atoms
parse_manifold_data(raw_data).map(|points| {
points.into_iter().map(|p| CanonicalAtom::ManifoldPoint {
coord: [Q16_16::from_f32(p.x), Q16_16::from_f32(p.y), Q16_16::from_f32(p.z)],
mass: Q0_16::from_f32(p.weight),
torsion: compute_famm_torsion(&p),
shell_index: compute_pist_shell(&p),
}).collect()
})
}
// ... biological, electrical, temporal channels
_ => Err(NormalizationError::UnsupportedChannel),
}
}
```
### Stage 2: Extract Deltas
```rust
/// Compute inter-atom deltas for compression
fn extract_deltas(
atoms: &[CanonicalAtom],
strategy: DeltaStrategy
) -> Vec<DeltaAtom> {
let mut deltas = Vec::with_capacity(atoms.len());
let mut last_keyframe = 0usize;
for (i, atom) in atoms.iter().enumerate() {
if should_keyframe(i, &strategy) {
deltas.push(DeltaAtom::absolute(atom));
last_keyframe = i;
} else {
let base = &atoms[last_keyframe];
deltas.push(DeltaAtom::from_delta(base, atom));
}
}
deltas
}
fn should_keyframe(index: usize, strategy: &DeltaStrategy) -> bool {
match strategy {
DeltaStrategy::Periodic(n) => index % n == 0,
DeltaStrategy::Adaptive(threshold) => {
// Keyframe if delta exceeds threshold (drift detection)
todo!("adaptive keyframing")
}
DeltaStrategy::TopologyChange => {
// Keyframe at topological events (barcode birth/death)
todo!("topological keyframing")
}
}
}
```
### Stage 3: Delta GCL Compression
```rust
/// Apply Delta GCL rule-based compression
fn compress_deltagcl(
deltas: &[DeltaAtom],
rules: &DeltaGCLRules
) -> CompressedPayload {
// Try rules in priority order
for rule in &rules.priority_order {
if let Ok(compressed) = apply_rule(rule, deltas) {
return CompressedPayload {
data: compressed,
rule_used: rule.clone(),
compression_ratio: compute_ratio(deltas, &compressed),
};
}
}
// Fallback: identity
CompressedPayload::identity(deltas)
}
/// Delta GCL Rule Definitions
pub struct DeltaGCLRules {
pub priority_order: Vec<DeltaRule>,
}
impl Default for DeltaGCLRules {
fn default() -> Self {
Self {
priority_order: vec![
DeltaRule::TopologicalDelta, // Preserve manifold structure
DeltaRule::TemporalDelta, // Exploit temporal correlation
DeltaRule::SpatialDelta, // Exploit spatial locality
DeltaRule::SymbolicDelta, // Expression simplification
DeltaRule::HybridDelta, // Multi-modal combination
],
}
}
}
```
### Stage 4: Learned Residual Compression (Optional)
```rust
/// Neural compression of residuals (VAE-style, per NEURAL_COMPRESSION spec)
fn compress_residuals(
payload: &CompressedPayload,
model: &NeuralCompressionModel
) -> NeuralCompressedPayload {
// Encoder: z = q(z|x) (reparameterization)
// Decoder: x_hat = p(x|z)
// Loss: reconstruction + KL divergence
let latent = model.encoder.encode(&payload.data);
let quantized = quantize_latent(latent, Q0_16);
NeuralCompressedPayload {
latent_codes: quantized,
reconstruction_params: model.get_params(),
expected_error: model.estimate_error(),
}
}
```
### Stage 5: Verify Reconstruction
```rust
/// Verify that decompression preserves invariants
fn verify_reconstruction(
original: &[CanonicalAtom],
reconstructed: &[CanonicalAtom],
invariants: &InvariantSet
) -> VerificationResult {
let mut result = VerificationResult::new();
for invariant in invariants {
let check_result = match invariant {
Invariant::CompressionRatio(target) => {
let actual = compute_compression_ratio(original, reconstructed);
actual >= *target
}
Invariant::TopologicalPersistence(barcode) => {
let reconstructed_barcode = compute_persistence(reconstructed);
bottleneck_distance(barcode, &reconstructed_barcode) < barcode.tolerance
}
Invariant::TimingAdmissibility(window) => {
check_timing_window(original, reconstructed, window)
}
Invariant::PhaseAlignment(phase_error) => {
check_phase_preservation(original, reconstructed, *phase_error)
}
Invariant::ChannelConsistency => {
check_channel_integrity(original, reconstructed)
}
};
result.add_check(invariant.clone(), check_result);
}
result
}
```
### Stage 6: Commit with Receipts
```rust
/// Generate verification receipts for committed packet
fn commit_with_receipts(
packet: &mut TMCPPacket,
verification: &VerificationResult
) -> CommitReceipt {
let receipt = CommitReceipt {
packet_hash: sha256(&packet.serialize()),
timestamp: now(),
compression_receipt: CompressionReceipt {
ratio: packet.compression_ratio(),
error_estimate: packet.reconstruction_error(),
rule_applied: packet.compression_rule(),
},
topology_receipt: TopologyReceipt {
barcode_hash: compute_barcode_hash(packet.atoms()),
persistence_diagram: packet.persistence_summary(),
},
timing_receipt: TimingReceipt {
admissibility_score: packet.tvi_admissibility(),
phase_alignment: packet.phase_coherence(),
},
routing_receipt: RoutingReceipt {
path_taken: vec![], // Filled by routing layer
hop_latencies: vec![],
},
invariant_checks: verification.results.clone(),
};
packet.set_trailer(receipt.clone());
receipt
}
```
---
## 5. Routing Algorithm (MNN-Style)
### 5.1 Morphic Neural Network Router
```rust
/// MNN-style routing with goal-awareness and constraints
pub struct MNNRouter {
pub local_state: NodeState,
pub carrier_metrics: HashMap<CarrierType, CarrierMetrics>,
pub historical_success: HashMap<OperationGoal, Q0_16>,
}
impl MNNRouter {
/// Route packet based on goal, state, and carrier metrics
pub fn route(
&self,
packet: &TMCPPacket,
goal: OperationGoal
) -> RoutingDecision {
// Step 1: Check local constraint satisfaction
if self.can_satisfy_locally(goal, packet) {
return RoutingDecision::LocalProcess;
}
// Step 2: Extract scalar goal from packet metadata
let scalar_goal = packet.routing_intent.priority;
// Step 3: Select best carrier based on goal + constraints
let best_carrier = self.select_carrier(goal, packet);
// Step 4: Compute cost for each path option
let costs = vec![
self.compute_cost(CarrierType::Local, goal, packet),
self.compute_cost(CarrierType::AtlasNetwork, goal, packet),
self.compute_cost(CarrierType::FileStorage, goal, packet),
self.compute_cost(CarrierType::SerialInterface, goal, packet),
];
// Step 5: Apply morphic adaptation based on historical success
let adapted_costs = self.apply_morphic_weights(costs, goal);
// Step 6: Select minimum cost action
let min_cost = adapted_costs.iter().min_by(|a, b| a.total.cmp(&b.total));
match min_cost.action {
CarrierType::Local => RoutingDecision::LocalProcess,
CarrierType::AtlasNetwork => RoutingDecision::GlobalRoute(best_carrier),
_ => RoutingDecision::Defer(queue_priority: packet.priority()),
}
}
fn can_satisfy_locally(&self, goal: OperationGoal, packet: &TMCPPacket) -> bool {
let mem_ok = self.local_state.memory_available_kb >= packet.memory_requirement_kb();
let trust_ok = self.local_state.trust_score >= packet.required_trust_score();
let goal_match = self.local_state.capabilities.contains(&goal);
mem_ok && trust_ok && goal_match
}
fn compute_cost(&self, carrier: CarrierType, goal: OperationGoal, packet: &TMCPPacket) -> Cost {
let metrics = self.carrier_metrics.get(&carrier).unwrap();
Cost {
energy: metrics.energy_per_byte * packet.size() as Q0_16,
time: metrics.latency_ms + (packet.size() as u32 / metrics.bandwidth_kbps),
bandwidth: packet.size() as u32,
risk: (1 - metrics.reliability) * packet.criticality(),
total: weighted_sum(energy, time, bandwidth, risk),
}
}
}
pub enum RoutingDecision {
LocalProcess, // Handle locally
GlobalRoute(Carrier), // Route to atlas/global
Reject(BufferReason), // Buffer for later
Recover(RetryPolicy), // Retry with backoff
Attest(Validation), // Validate before routing
Defer { queue_priority: Q0_16 }, // Queue for later processing
}
```
### 5.2 Goal-Codon Mapping (GCL Integration)
```rust
/// Map routing goals to GCL codons for action encoding
fn goal_to_codon(goal: OperationGoal) -> GCLCodon {
match goal {
OperationGoal::Health => GCLCodon::new("HEALTH", Priority::High),
OperationGoal::Compress => GCLCodon::new("COMPRESS", Priority::Medium),
OperationGoal::Route => GCLCodon::new("ROUTE", Priority::Normal),
OperationGoal::Recover => GCLCodon::new("RECOVER", Priority::Critical),
OperationGoal::Attest => GCLCodon::new("ATTEST", Priority::High),
}
}
```
---
## 6. Verification Invariants
### 6.1 Invariant Classification
```rust
/// Invariants classified by verification status
pub enum InvariantClaim {
// Fully implemented and verified
Implemented { theorem: String, proof_status: ProofStatus },
// Formal spec exists, implementation in progress
Specification { formal_module: String, lean_theorem: String },
// Theoretical hypothesis, not yet formalized
Hypothesis { paper_ref: String, conjecture: String },
// Known limitation, explicitly unverified
Unverified { reason: String, safety_bounds: String },
}
pub enum ProofStatus { Proven, Checked, WIP, Axiom }
```
### 6.2 Core Invariants
| Invariant | Description | Status | Classification |
|-----------|-------------|--------|----------------|
| **CompressionRatio** | decode(encode(x)) achieves target ratio | Implemented | Proven in `BrainBoxDescriptor.lean` |
| **ReconstructionError** | L2 error < ε within precision tier | Implemented | Checked via `#eval` |
| **TopologicalPersistence** | Barcode preservation under bottleneck distance | Specification | Theorem in `HumanNeuralCompression.lean` |
| **TimingAdmissibility** | TVI-style admissibility window compliance | Specification | Defined in `SpikeSync.lean` |
| **PhaseWindowSafety** | Glymphatic pump-phase precision matching | Implemented | 6.5σ verified |
| **ChannelConsistency** | No cross-channel information leakage | Hypothesis | Safety property, WIP |
| **RoutingTermination** | MNN routing converges in bounded steps | Specification | Metric space contraction |
| **FixedPointDeterminism** | Q0_16/Q16_16 ops are platform-independent | Implemented | Unit tested across targets |
### 6.3 Lean Theorem Stubs
```lean
-- File: 0-Core-Formalism/lean/Semantics/Semantics/TMMCP/CompressionInvariant.lean
import Semantics.FixedPoint
import Semantics.BrainBoxDescriptor
namespace TMMCP
/-- Compression preserves information within declared error budget -/
theorem compress_decompress_preserves_info
{α : Type} [Encodable α] [DecidableEq α]
(x : α)
(encoder : α → Compressed α)
(decoder : Compressed αα)
(error_budget : Q0_16)
(h : ∀ y, reconstruction_error x (decoder y) ≤ error_budget) :
reconstruction_error x (decoder (encoder x)) ≤ error_budget := by
-- Proof relies on encoder/decoder being approximate inverses
sorry
/-- Delta encoding is information-conservative for sequential data -/
theorem delta_encoding_conservative
(atoms : List CanonicalAtom)
(keyframes : List Nat)
(h_keyframes : keyframes.head? = some 0) :
total_info (delta_encode atoms keyframes) = total_info atoms := by
-- Delta atoms carry same information as absolute, just differentially encoded
sorry
/-- Topological barcode preservation under compression -/
theorem compression_preserves_persistence
(manifold : PointCloud)
(compressed : Compressed PointCloud)
(barcode : PersistenceBarcode)
(ε : Q0_16)
(h : bottleneck_distance barcode (compute_barcode manifold) < ε) :
bottleneck_distance barcode (compute_barcode (decompress compressed)) < ε := by
-- Relies on compression being Lipschitz continuous w.r.t. Hausdorff metric
sorry
/-- MNN routing terminates with bounded cost -/
theorem mnn_routing_termination
(router : MNNRouter)
(packet : TMCPPacket)
(max_hops : Nat) :
∃ decision, route router packet decision ∧ hops decision ≤ max_hops := by
-- Metric space contraction + finite carrier set
sorry
/-- Fixed-point arithmetic is deterministic across platforms -/
theorem q16_16_determinism
(op : Q16_16 → Q16_16 → Q16_16)
(x y : Q16_16)
(platform₁ platform₂ : Platform) :
eval op x y platform₁ = eval op x y platform₂ := by
-- Integer arithmetic is deterministic; no floating-point
rfl
end TMMCP
```
---
## 7. Reference Implementation Plan
### 7.1 Module Structure
```
0-Core-Formalism/lean/Semantics/Semantics/TMMCP/
├── Core.lean # CanonicalAtom, ChannelType, Base types
├── FixedPoint.lean # Q0_16, Q16_16 operations (re-export)
├── Normalization.lean # Channel normalization functions
├── DeltaEncoding.lean # Delta atom extraction
├── DeltaGCL.lean # Rule-based compression
├── Compression.lean # 6-stage pipeline
├── Packet.lean # Packet format, serialization
├── Routing.lean # MNN router implementation
├── Verification.lean # Invariant checking
└── Tests/
├── Unit.lean # Unit tests for each module
├── Integration.lean # End-to-end pipeline tests
└── Benchmark.lean # Performance benchmarks
infra/tm_mcp/
├── __init__.py
├── channel_adapters.py # Python shims for each channel
├── packet_encoder.py # Python packet encoding
├── compression_pipeline.py # Python pipeline orchestration
├── neural_residual.py # Neural compression integration
└── tests/
├── test_symbolic.py
├── test_spike.py
└── test_geometric.py
src/tm_mcp/
├── lib.rs # Rust core library
├── packet.rs # Zero-copy packet handling
├── compression.rs # SIMD-accelerated compression
└── routing.rs # Async MNN router
```
### 7.2 Data Structures (Pseudocode)
```python
# Python extraction shim (Lean spec is source of truth; this is I/O scaffolding only)
from dataclasses import dataclass
from typing import List, Optional, Dict, Union
import struct
from enum import Enum, auto
class ChannelType(Enum):
SYMBOLIC_TEXT = auto()
SPIKE_EVENT = auto()
GEOMETRIC_SHAPE = auto()
BIOLOGICAL_CONC = auto()
ELECTRICAL_POT = auto()
@dataclass
class Q0_16:
"""16-bit fixed-point fraction [0, 1-2^-16]"""
value: int # u16 storage
@classmethod
def from_float(cls, f: float) -> 'Q0_16':
return cls(int(f * 65535.0))
def to_float(self) -> float:
return self.value / 65535.0
@dataclass
class CanonicalAtom:
"""Universal IR for all modalities"""
atom_type: int
timestamp: int # nanoseconds
channel_id: int
# Union fields based on atom_type
spike_amplitude: Optional[Q0_16] = None
manifold_coord: Optional[tuple[Q0_16, Q0_16, Q0_16]] = None
symbolic_hash: Optional[bytes] = None
concentration_delta: Optional[int] = None # Q16_16
class TMCPPacket:
"""Transport packet with fixed header + variable payload"""
HEADER_SIZE = 32
ROUTING_META_SIZE = 16
TRAILER_SIZE = 32
FOOTER_SIZE = 8
def __init__(self):
self.version = 0x01
self.channel_type: ChannelType = ChannelType.SYMBOLIC_TEXT
self.sequence_num = 0
self.timestamp = 0
self.source_node = b'\x00' * 8
self.destination_node = b'\x00' * 8
self.atoms: List[CanonicalAtom] = []
def serialize(self) -> bytes:
header = struct.pack('!BBHHIQ',
self.version,
self.channel_type.value,
0, # flags
self.channel_type.value, # channel_id as u16
self.sequence_num,
self.timestamp
)
header += self.source_node + self.destination_node
# Serialize atoms
payload = struct.pack('!H', len(self.atoms))
for atom in self.atoms:
payload += self._serialize_atom(atom)
# Padding to align
padding = (8 - (len(payload) % 8)) % 8
payload += b'\x00' * padding
# Trailer + footer
trailer = self._compute_receipts()
footer = struct.pack('!II',
crc32(payload),
crc32(header + payload + trailer)
)
return header + payload + trailer + footer
def _serialize_atom(self, atom: CanonicalAtom) -> bytes:
# Variable encoding based on atom type
base = struct.pack('!B', atom.atom_type)
base += struct.pack('!Q', atom.timestamp)
base += struct.pack('!I', atom.channel_id)
return base
class CompressionPipeline:
"""6-stage compression pipeline"""
def __init__(self, rules: DeltaGCLRules):
self.rules = rules
self.stage1_normalizer = ChannelNormalizer()
self.stage2_delta = DeltaExtractor()
self.stage3_deltagcl = DeltaGCLCompressor(rules)
self.stage4_neural = Optional[NeuralCompressor] = None
self.stage5_verifier = ReconstructionVerifier()
self.stage6_committer = ReceiptGenerator()
def process(self, channel: ChannelType, raw_data: bytes) -> TMCPPacket:
# Stage 1: Normalize
atoms = self.stage1_normalizer.normalize(channel, raw_data)
# Stage 2: Extract deltas
deltas = self.stage2_delta.extract(atoms)
# Stage 3: Delta GCL compression
compressed = self.stage3_deltagcl.compress(deltas)
# Stage 4: Neural residual (optional)
if self.stage4_neural:
compressed = self.stage4_neural.compress_residuals(compressed)
# Stage 5: Verify reconstruction
verification = self.stage5_verifier.verify(atoms, compressed)
if not verification.all_passed():
raise CompressionError("Invariant violation", verification.failures)
# Stage 6: Commit with receipts
packet = TMCPPacket()
packet.atoms = compressed.to_atoms()
receipt = self.stage6_committer.commit(packet, verification)
packet.set_receipt(receipt)
return packet
```
### 7.3 Test Plan
```python
# tests/test_tm_mcp.py
import unittest
import hypothesis.strategies as st
from hypothesis import given
class TestCanonicalAtoms(unittest.TestCase):
"""Unit tests for canonical atom IR"""
def test_q0_16_roundtrip(self):
"""Q0_16 float conversion is reversible within precision"""
for f in [0.0, 0.5, 0.9999, 0.0001]:
q = Q0_16.from_float(f)
reconstructed = q.to_float()
self.assertAlmostEqual(f, reconstructed, delta=1/65536)
def test_spike_normalization(self):
"""Spike train normalizes to canonical atoms"""
spikes = [
{'time_ns': 1_000_000, 'electrode': 0, 'amplitude': 1.5, 'duration_ms': 1.0},
{'time_ns': 1_050_000, 'electrode': 1, 'amplitude': 0.8, 'duration_ms': 0.8},
]
atoms = normalize_spike_train(spikes)
self.assertEqual(len(atoms), 2)
self.assertEqual(atoms[0].timestamp, 1_000_000)
def test_delta_encoding_reversible(self):
"""Delta encoding preserves information"""
atoms = [generate_test_atoms(n=100) for _ in range(10)]
for atom_list in atoms:
deltas = extract_deltas(atom_list, DeltaStrategy::Periodic(10))
reconstructed = reconstruct_from_deltas(deltas)
self.assertEqual(len(reconstructed), len(atom_list))
class TestCompressionPipeline(unittest.TestCase):
"""Integration tests for 6-stage pipeline"""
def test_symbolic_text_channel(self):
"""MVP Channel 1: Symbolic text compression"""
pipeline = CompressionPipeline(DeltaGCLRules.default())
text = b"f(x) = sin(x) + cos(y)"
packet = pipeline.process(ChannelType.SYMBOLIC_TEXT, text)
self.assertIsInstance(packet, TMCPPacket)
self.assertTrue(packet.verify_receipts())
self.assertGreater(packet.compression_ratio, 1.0)
def test_spike_event_channel(self):
"""MVP Channel 2: Spike event compression"""
pipeline = CompressionPipeline(DeltaGCLRules.default())
spike_data = generate_synthetic_spikes(n=1000, rate=50.0)
packet = pipeline.process(ChannelType.SPIKE_EVENT, spike_data)
self.assertEqual(packet.channel_type, ChannelType.SPIKE_EVENT)
self.assertLess(packet.reconstruction_error, 0.01)
def test_geometric_shape_channel(self):
"""MVP Channel 3: Geometric shape compression"""
pipeline = CompressionPipeline(DeltaGCLRules.default())
manifold_points = generate_random_manifold(n=500, dim=3)
packet = pipeline.process(ChannelType.GEOMETRIC_SHAPE, manifold_points)
# Verify topology preservation
original_barcode = compute_persistence(manifold_points)
reconstructed = decompress(packet)
reconstructed_barcode = compute_persistence(reconstructed)
distance = bottleneck_distance(original_barcode, reconstructed_barcode)
self.assertLess(distance, 0.1)
class TestRouting(unittest.TestCase):
"""Tests for MNN routing layer"""
def test_local_processing_decision(self):
"""Router selects local processing when constraints satisfied"""
router = MNNRouter(
local_state=NodeState(memory_kb=10000, trust_score=Q0_16(0.9)),
carrier_metrics={}
)
packet = TMCPPacket()
packet.memory_requirement_kb = 100
packet.required_trust_score = Q0_16(0.5)
decision = router.route(packet, OperationGoal::Compress)
self.assertEqual(decision, RoutingDecision::LocalProcess)
def test_global_routing_decision(self):
"""Router selects global route when local constraints violated"""
router = MNNRouter(
local_state=NodeState(memory_kb=100, trust_score=Q0_16(0.9)),
carrier_metrics={
CarrierType::AtlasNetwork: CarrierMetrics(
latency_ms=50, bandwidth_kbps=10000, reliability=Q0_16(0.99)
)
}
)
packet = TMCPPacket()
packet.memory_requirement_kb = 1000 # Exceeds local memory
decision = router.route(packet, OperationGoal::Compress)
self.assertEqual(decision.action, RoutingDecision::GlobalRoute)
class BenchmarkCompression(unittest.TestCase):
"""Performance benchmarks"""
def test_compression_throughput(self):
"""Measure symbols/second compression rate"""
import time
pipeline = CompressionPipeline(DeltaGCLRules.default())
data = generate_large_dataset(size_mb=10)
start = time.time()
packet = pipeline.process(ChannelType.SYMBOLIC_TEXT, data)
elapsed = time.time() - start
throughput = len(data) / elapsed / 1e6 # MB/s
self.assertGreater(throughput, 1.0) # At least 1 MB/s
def test_packet_latency(self):
"""Measure end-to-end latency"""
import time
pipeline = CompressionPipeline(DeltaGCLRules.default())
small_packet = b"test data"
latencies = []
for _ in range(1000):
start = time.time()
pipeline.process(ChannelType.SYMBOLIC_TEXT, small_packet)
latencies.append(time.time() - start)
p99_latency = sorted(latencies)[int(0.99 * len(latencies))]
self.assertLess(p99_latency, 0.001) # < 1ms for small packets
```
---
## 8. Minimal Viable Protocol (MVP)
### 8.1 Phase 1: Three Working Channels
| Channel | Input | Canonical Atom | Compression | Verification |
|---------|-------|----------------|-------------|--------------|
| **Symbolic/Text** | UTF-8 expressions, Lean code | `SymbolicTerm` (hash, complexity, deps) | Delta tree-diff + expression simplification | AST equivalence, dependency closure |
| **Temporal/Spike** | Spike times, amplitudes, electrode IDs | `SpikeEvent` (timestamp, channel, amp, width, phase) | Temporal delta + rate coding | Timing admissibility (TVI), ISI statistics |
| **Geometric/Shape** | Point clouds, quaternions, braids | `ManifoldPoint` (coord, mass, torsion, shell) + `BraidCrossing` | Topological delta + PIST coordinate quantization | Persistence barcode preservation, bottleneck distance |
### 8.2 Phase 2: Extended Channels
- **Biological Concentration**: `ConcentrationDelta` with Fick's law diffusion
- **Electrical Potential**: `MembranePotential` with ion channel state encoding
- **Magnetic Field**: Magnetoreception vectors with inclination/intensity
- **Thermal Gradient**: Infrared pattern encoding with TRPA1-inspired thresholds
### 8.3 Phase 3: Meta Channels
- **Routing Control**: MNN decision packets with goal/cost metadata
- **Verification**: Receipt propagation with attestation chains
- **Cross-Channel**: Multiplexed packets carrying atoms from multiple channels
---
## 9. Open Gaps and Future Work
### 9.1 Implementation Gaps
| Gap | Priority | Blocker | Mitigation |
|-----|----------|---------|------------|
| Neural residual compressor (Stage 4) | Medium | Training data | Start with rule-only compression |
| GPU-accelerated fixed-point SIMD | Medium | CUDA kernels | Use CPU Q16_16 initially |
| Formal proofs for all invariants | High | Lean expertise | Prioritize safety-critical invariants |
| Cross-platform determinism test | Medium | CI infrastructure | Unit tests on major targets |
| Bio-channel normalization | Low | Biological data sources | Synthetic data for MVP |
### 9.2 Research Questions
1. **Optimal keyframing strategy**: How to balance compression ratio vs. reconstruction error for adaptive vs. periodic keyframing?
2. **Multi-modal atom ordering**: Does channel interleaving affect compression efficiency?
3. **Routing convergence**: Can we prove MNN routing terminates in O(log n) hops for n-node networks?
4. **Biological channel semantics**: How to encode concentration semantics (promoter activation thresholds) in canonical atoms?
### 9.3 Integration with Research Stack
| Component | Integration Point | Status |
|-----------|-------------------|--------|
| Delta GCL | `0-Core-Formalism/lean/Semantics/Semantics/DeltaGCL/` | Existing |
| MNN Router | `0-Core-Formalism/lean/Semantics/Semantics/MorphicNeuralNetwork.lean` | Existing |
| Fixed-point | `0-Core-Formalism/lean/Semantics/Semantics/FixedPoint.lean` | Existing |
| TVI | `0-Core-Formalism/lean/Semantics/ExtensionScaffold/Temporal/SpikeSync.lean` | Existing |
| BBD | `0-Core-Formalism/lean/Semantics/Semantics/BrainBoxDescriptor.lean` | Existing |
| TM-MCP Core | `0-Core-Formalism/lean/Semantics/Semantics/TMMCP/` | **To Create** |
| Python shim | `infra/tm_mcp/` | **To Create** |
| Rust impl | `src/tm_mcp/` | **To Create** |
---
## 10. Summary
**TM-MCP** compiles heterogeneous mathematical, biological, neural, geometric, and symbolic channels into a unified, invariant-preserving compression and transport grammar.
**Key Deliverables:**
1. Canonical Atom IR supporting 8+ modalities
2. 6-stage compression pipeline with Delta GCL integration
3. MNN-style routing with goal-awareness
4. Formal invariants classified (implemented/spec/hypothesis/unverified)
5. Lean theorem stubs for core properties
6. Python/Rust pseudocode for reference implementation
7. MVP with 3 working channels (symbolic, spike, geometric)
8. Test plan with unit, integration, and benchmark suites
**Next Steps:**
1. Implement `TMMCP/` Lean modules with `#eval` verification
2. Create Python extraction shim (`infra/tm_mcp/`) from the Lean spec
3. Validate MVP channels against synthetic data
4. Run `lake build` and add to CI
5. Measure compression ratios and reconstruction error for benchmark suite
---
*Document Status: SPECIFICATION DRAFT*
*Compiler Task: EXTRACT INVARIANTS → DEFINE PROTOCOL → IMPLEMENT & VERIFY*
*Target: DeepSeek as Protocol Compiler, not Creative Synthesis*