Research-Stack/6-Documentation/wiki/RDS-Rust-Workspace.md

ENE RDS Rust Workspace

Overview

The ene-rds workspace replaces the Python RDS stack (ene_rds_* modules) with a unified Rust binary suite. All PostgreSQL surfaces—wiki, ephemeral nodes, chat logs, and the HTTP API—are now native Rust crates.

Location: 4-Infrastructure/infra/ene-rds/

Workspace Crates

Crate	Type	Replaces	Purpose
ene-rds-core	lib	ene_rds_* base connection logic	Shared RdsClient, DSN builder, vec_to_pgtext(), ingestion receipts
ene-rds-wiki	lib	ene_rds_wiki_layer.py	Wiki pages, revisions, links, categories, full-text search
ene-rds-ephemeral	lib	ene_rds_ephemeral_node.py	EphemeralNode thermal zones, tasks, receipts, scar events, metrics
ene-rds-chat	lib	ene_rds_chat_log.py	Chat session ingestion, keyword search, semantic similarity
ene-api	bin	ene_chat_api.py	Axum HTTP server on :3000 mounting all surfaces
ene-sync	bin	ene_claw_sync.py	Polls opencode.db SQLite → upserts into RDS chat tables

Build

cd 4-Infrastructure/infra/ene-rds
cargo build --release

Dependencies are fetched from crates.io on first build.

Run

Initialize schema only

export RDS_PASSWORD=...
./target/release/ene-sync init-schema

One-shot sync (opencode.db → RDS)

./target/release/ene-sync sync --embed

Watch mode (polls every 60s, incremental)

./target/release/ene-sync watch --interval 60 --embed

API server

./target/release/ene-api

Endpoints:

• GET /health — service status
• GET /sessions?limit=10 — list chat sessions
• GET /sessions/:id — get full session with messages
• GET /search?q=query&limit=10 — keyword search across messages
• GET /wiki/search?q=query — wiki full-text search
• GET /wiki/:slug — get wiki page by slug
• GET /ephemeral/nodes?zone=hot — list nodes by thermal zone
• GET /ephemeral/nodes/:id — get single node

Environment Variables

Variable	Default	Purpose
RDS_HOST	database-1.cluster-c9i0w8eu8fnv.us-east-2.rds.amazonaws.com	PostgreSQL host
RDS_PORT	5432	PostgreSQL port
RDS_USER	postgres	PostgreSQL user
RDS_PASSWORD	—	Password or IAM token
RDS_DB	postgres	Database name
OLLAMA_HOST	http://localhost:11434	Ollama embedding endpoint
OLLAMA_EMBED_MODEL	nomic-embed-text	Embedding model

Architecture Changes from Python

Concern	Python (old)	Rust (new)
PostgreSQL driver	psycopg2	tokio-postgres with with-serde_json-1
JSONB handling	Manual string serialization	Native serde_json::Value ↔ JSONB
pgvector vectors	Python list → string	vec_to_pgtext() — '[0.1,0.2,...]'::vector
HTTP server	FastAPI	Axum
SQLite source	—	rusqlite with bundled bindings
Ollama embeddings	requests	reqwest
Runtime	Python 3.11 + venv	Single static binary

Database Schema

The Rust surfaces create the following tables on init_tables():

ene.chat_sessions

• session_id TEXT PRIMARY KEY
• workspace_fingerprint TEXT, workspace_root TEXT
• fork_parent_session_id TEXT (self-referential for session forks)
• compaction_count INTEGER, compaction_summary TEXT
• message_count INTEGER, token_input_total BIGINT, token_output_total BIGINT
• created_at_ms BIGINT, updated_at_ms BIGINT
• first_message_at_ms BIGINT, last_message_at_ms BIGINT
• embedding vector(768) — session-level semantic vector
• meta JSONB — slug, agent, model, project_id, cost, cache tokens
• receipt TEXT, updated_at TIMESTAMPTZ

ene.chat_messages

• id BIGSERIAL PRIMARY KEY
• session_id TEXT REFERENCES chat_sessions
• message_index INTEGER (ordered within session)
• role TEXT — user, assistant, tool, system
• blocks JSONB — typed content blocks (text, reasoning, tool_use, tool_result)
• text_content TEXT — concatenated text for full-text search
• token_input BIGINT, token_output BIGINT, token_cache_creation BIGINT, token_cache_read BIGINT
• tool_calls JSONB — extracted tool calls for structured search
• embedding vector(768) — per-message semantic vector
• receipt_hash TEXT — links to ingestion receipt
• created_at_ms BIGINT, created_at TIMESTAMPTZ

ene.ephemeral_nodes

• node_id TEXT PRIMARY KEY
• thermal_zone TEXT — hot, warm, cold
• reliability_raw BIGINT — Q0_32 fixed-point
• latency_p95_ms BIGINT
• scar_count INTEGER
• last_seen_ms BIGINT
• reputation_raw BIGINT — Q16_16 fixed-point
• quarantine_until_ms BIGINT
• meta JSONB
• created_at_ms BIGINT, updated_at_ms BIGINT

ene.ephemeral_tasks

• task_id TEXT PRIMARY KEY
• session_id TEXT, node_id TEXT REFERENCES ephemeral_nodes
• task_state TEXT — pending, dispatched, partial, complete, expired, abandoned
• priority_raw BIGINT, ttl_ms BIGINT
• dispatched_at_ms BIGINT, completed_at_ms BIGINT
• result_hash TEXT, meta JSONB

ene.ephemeral_receipts

• receipt_id TEXT PRIMARY KEY
• task_id TEXT, node_id TEXT
• cross_matrix JSONB — braid crossing matrix
• sidon_slack INTEGER, step_count INTEGER
• residual_series BIGINT[] — Q0_32 residual series
• write_timing_ms BIGINT
• scar_absent BOOLEAN

ene.ephemeral_scar_events

• scar_id TEXT PRIMARY KEY
• node_id TEXT, task_id TEXT
• scar_pressure BIGINT, failure_mode TEXT, coarsening_agent TEXT

ene.ephemeral_metrics

• metric_id TEXT PRIMARY KEY
• node_id TEXT, metric_name TEXT
• metric_value_raw BIGINT, metric_scale INTEGER
• recorded_at_ms BIGINT

Sync Daemon (ene-sync)

The sync binary reads from the OpenCode SQLite database at ~/.local/share/opencode/opencode.db:

1. Sessions — metadata, tokens, costs, workspace fingerprint
2. Messages — conversation turns with role and timing
3. Parts — actual content (text, reasoning, tool calls, tool results)
4. Session messages — lifecycle events (agent switches, model switches)

Content is normalized into ChatSession + ChatMessage structs, embeddings generated via Ollama, then upserted into RDS.

Incremental sync

Watch mode tracks MAX(time_updated) from the session table in a local state file (~/.cache/ene-sync/state.json). Only sessions updated since the checkpoint are re-ingested.

Python Bridge (Legacy Support)

For calling unported Python modules, bridge_wrapper.py auto-discovers classes with handle_request() methods and dispatches JSON requests. The Rust surfaces do not depend on this bridge.

Future Work

• Add postgres-native-tls or rustls for RDS TLS instead of NoTls
• Wire semantic search in ene-api (currently returns placeholder error)
• Add pgvector binary format support for faster embedding inserts
• Batch embedding requests if Ollama adds native batching

Related

• 4-Infrastructure/infra/ene-rds/README.md — crate-level README with quickstart
• 4-Infrastructure/infra/ene_rds_wiki_layer.py — legacy Python wiki surface
• 4-Infrastructure/infra/ene_rds_fractal_fold.py — legacy Python fractal surface
• 0-Core-Formalism/lean/Semantics/EphemeralNode/ — Lean thermal zone formalism