Research-Stack/4-Infrastructure/infra/ene-session-sync/README.md

ene-session-sync

Rust daemon that syncs OpenCode chat sessions to the ENE RDS PostgreSQL cluster.

Architecture

opencode.db (SQLite)   ──► OpenCodeSource   ──►┐
.claw/sessions/*.jsonl ──► ClawSource        ──►├── RdsSink ──► ene.chat_sessions
                                                │              ene.chat_messages
                                                │              ene.ingestion_receipts
                            Ollama embed        ──► vector(768) columns (optional)

                            PythonBridge        ──► ene_rds_wiki_layer / fractal_fold
                                                    (existing Python surfaces via subprocess)

Rust surfaces (this crate)

Module	Role
source.rs	OpenCodeSource (SQLite/rusqlite) + ClawSource (JSONL)
sink.rs	RdsSink — tokio-postgres, idempotent upserts, DDL init
embed.rs	Embedder — Ollama /api/embeddings via reqwest
normalize.rs	OpenCode row → ChatSession/ChatMessage normalization
bridge.rs	PythonBridge — subprocess delegation to Python infra modules
models.rs	Shared data types
main.rs	CLI (clap) + orchestration

Python adaptation layer (bridge.rs + bridge_wrapper.py)

The Rust binary can delegate to any Python infra surface via:

Rust PythonBridge::call("ene_rds_wiki_layer", payload)
  └─► python3 bridge_wrapper.py <infra_dir>
        ├── reads JSON request from stdin
        ├── imports the named module
        ├── calls instance.handle_request(payload)
        └── writes JSON response to stdout

Modules currently bridgeable: ene_rds_wiki_layer, ene_rds_fractal_fold, ene_rds_ephemeral_node, ene_embedding, and any module that exposes handle_request(payload: dict) -> dict.

RDS Schema

Tables are created/migrated automatically on first connect:

ene.chat_sessions   -- one row per session
ene.chat_messages   -- one row per message (UNIQUE on session_id, message_index)
ene.ingestion_receipts -- audit trail for each sync run

Full DDL lives in sink.rs::init_tables().

CLI

ene-session-sync [OPTIONS] <SUBCOMMAND>

Global options:
  --db <PATH>          opencode.db path (default: ~/.local/share/opencode/opencode.db)
  --dsn <DSN>          PostgreSQL DSN (default: RDS_* env vars)
  --infra-dir <DIR>    Python infra directory for bridge calls
  --embed              Enable Ollama embedding generation

Subcommands:
  sync [--since <MS>]           One-shot sync of opencode.db
  watch [--interval <SECS>]     Continuous poll (default 60 s)
  claw-sync --sessions-dir <D>  Sync .claw/sessions/ JSONL files
  search <QUERY> [--limit N] [--semantic]
  list [--limit N]
  get <SESSION_ID>
  bridge-test [MODULE]          Test Python bridge for a module
  init-schema                   Initialize RDS schema only

Configuration

Credentials come from environment variables only — never hard-code them.

Variable	Purpose
RDS_HOST	PostgreSQL host
RDS_PORT	PostgreSQL port (default 5432)
RDS_USER	PostgreSQL user
RDS_PASSWORD / RDS_IAM_TOKEN	Password or RDS IAM auth token
RDS_DB	Database name (default postgres)
RDS_DSN	Full libpq DSN (overrides all above)
OLLAMA_HOST	Ollama base URL (default http://localhost:11434)
OLLAMA_EMBED_MODEL	Embedding model (default nomic-embed-text)
PYTHON_CMD	Python executable for bridge (default python3)

Store these in /etc/ene-session-sync/env or ~/.config/ene-session-sync/env (sourced by the systemd unit).

Build

# From this directory:
cargo build --release
# Binary: target/release/ene-session-sync

# Or install to ~/.cargo/bin:
cargo install --path .

Note: rusqlite is compiled with the bundled feature so no system libsqlite3 is required.

Note on TLS: tokio-postgres is linked with NoTls in this build. RDS requires SSL; use sslmode=disable with an SSL-terminating proxy (e.g. pg_bouncer or ssh -L) or add tokio-postgres-rustls as a dependency and swap NoTls in sink.rs::connect().

Systemd installation

# Copy binary
sudo cp target/release/ene-session-sync /usr/local/bin/

# Install units
sudo cp systemd/ene-session-sync.{service,timer} /etc/systemd/system/

# Create env file
sudo mkdir -p /etc/ene-session-sync
sudo tee /etc/ene-session-sync/env <<'EOF'
RDS_HOST=database-1.cluster-xxxx.us-east-2.rds.amazonaws.com
RDS_USER=postgres
RDS_IAM_TOKEN=<generated by aws rds generate-db-auth-token>
EOF
sudo chmod 600 /etc/ene-session-sync/env

# Enable
sudo systemctl daemon-reload
sudo systemctl enable --now ene-session-sync.timer

# Check
systemctl status ene-session-sync.timer
journalctl -u ene-session-sync -f

Python bridge setup

The bridge expects bridge_wrapper.py to be co-located with the infra modules. The --infra-dir flag (or the directory next to the running binary) must contain both bridge_wrapper.py and the Python modules.

# Test:
ene-session-sync --infra-dir /path/to/stack/4-Infrastructure/infra bridge-test ene_rds_wiki_layer