From 80ebcca11c3255b63dc14f8f7aa1cd02281f9cb5 Mon Sep 17 00:00:00 2001 From: allaun Date: Fri, 19 Jun 2026 22:44:38 -0500 Subject: [PATCH] docs(infra): remove AWS-specific credential and RDS backend details Update Credential-System.md and RDS-Rust-Workspace.md to describe a provider-neutral PostgreSQL backend instead of AWS RDS. Replace IAM auth examples with standard libpq env-var connection. Remove the ~/.aws/ file layout and AWS hostname defaults. --- 6-Documentation/wiki/Credential-System.md | 63 ++++++++++------------ 6-Documentation/wiki/RDS-Rust-Workspace.md | 2 +- 2 files changed, 29 insertions(+), 36 deletions(-) diff --git a/6-Documentation/wiki/Credential-System.md b/6-Documentation/wiki/Credential-System.md index 1ba0c6f5..547e97cb 100644 --- a/6-Documentation/wiki/Credential-System.md +++ b/6-Documentation/wiki/Credential-System.md @@ -2,7 +2,7 @@ > **Canonical source**: `4-Infrastructure/infra/ene-session-sync/src/credential.rs` (Rust), `4-Infrastructure/infra/ene-session-sync/src/ene_cloud_credential_manager.rs` (Rust), `4-Infrastructure/infra/recover_credential_server.sh` (deployment script) > **Runtime host**: `microvm-racknerd` (100.101.247.127) — Debian 13 VM on RackNerd -> **Primary backend**: AWS RDS PostgreSQL (`database-1-instance-1.cghu8yqogqwo.us-east-1.rds.amazonaws.com`) +> **Primary backend**: PostgreSQL (configured via `RDS_*` environment variables) > **Fallback chain**: RDS → remote server → local JSON → environment variables > **SOPS key**: `age1tp4vr565zkmvnyulatpyaj6z8zrz7q9mpaypz85yz8rty99crdasualxyr` @@ -26,9 +26,9 @@ │ ┌───────────────┼───────────────┐ │ │ │ │ │ │ │ ┌────────▼────────┐ ┌───▼────────┐ ┌────▼───────┐ │ -│ │ AWS RDS │ │ Remote │ │ Local │ │ -│ │ PostgreSQL │ │ Server │ │ JSON │ │ -│ │ (primary) │ │ (chain) │ │ (backup) │ │ +│ │ PostgreSQL │ │ Remote │ │ Local │ │ +│ │ (primary) │ │ Server │ │ JSON │ │ +│ │ │ │ (chain) │ │ (backup) │ │ │ └────────┬────────┘ └────────────┘ └────────────┘ │ │ │ │ │ ┌────────▼────────┐ │ @@ -45,14 +45,14 @@ The `credential_provider.py` (`v0.4`) implements a strict priority order. The fi | Tier | Function | Trigger | Data Source | |------|----------|---------|-------------| -| **1** | `_load_from_rds()` | Always runs first | AWS RDS PostgreSQL `credential_store.credentials` | +| **1** | `_load_from_rds()` | Always runs first | PostgreSQL `credential_store.credentials` | | **2** | `_load_from_remote()` | Skipped if RDS empty AND `RS_CREDENTIAL_SERVER` points to self | Another credential server URL | | **3** | `_load_from_config()` | Skipped if tiers 1–2 empty | `/etc/rs-surface/credentials.json` | | **4** | `_load_from_env()` | Last resort | Environment variables per `PROVIDER_ENV_MAP` | -### Why RDS is the primary source +### Why PostgreSQL is the primary source -RDS credentials are **encrypted at rest** (AES-256-GCM payload), **IAM-authenticated** (temporary tokens), and **auditable** (`access_log` table). The local JSON file (`/etc/rs-surface/credentials.json`) is a plaintext fallback for disaster recovery only. +PostgreSQL credentials are **encrypted at rest** (AES-256-GCM payload) and **auditable** (`access_log` table). The local JSON file (`/etc/rs-surface/credentials.json`) is a plaintext fallback for disaster recovery only. **Current status** (2026-05-21): Tier 1 (RDS) is active and serving all 11 credentials. Tier 3 (local JSON) exists as a warm standby but is not consulted because RDS succeeds. @@ -91,7 +91,6 @@ curl -s http://100.101.247.127:8444/status | jq . | Provider | Classification | pkg | Notes | |----------|---------------|-----|-------| -| aws | 3 (secret) | credentials/aws | | | bedrock | 3 | credentials/bedrock | | | deepseek | 3 | credentials/deepseek | | | linear | 3 | credentials/linear | | @@ -104,36 +103,35 @@ curl -s http://100.101.247.127:8444/status | jq . | wolfram_alpha | 2 | credentials/wolfram_alpha | | | **authentik** | 3 | credentials/authentik | LLM controller token for Authentik API | -## Authentication to RDS +## Authentication to PostgreSQL -### Method: IAM Authentication Token (preferred) +Use a standard libpq connection string built from `RDS_*` environment variables: ```python -import boto3 +import os +import psycopg2 + +host = os.environ.get('RDS_HOST', 'localhost') +port = os.environ.get('RDS_PORT', '5432') +user = os.environ.get('RDS_USER', 'postgres') +password = os.environ.get('RDS_PASSWORD', '') +dbname = os.environ.get('RDS_DB', 'postgres') -client = boto3.client('rds', region_name='us-east-1') -token = client.generate_db_auth_token( - DBHostname='database-1-instance-1.cghu8yqogqwo.us-east-1.rds.amazonaws.com', - Port=5432, - DBUsername='postgres', - Region='us-east-1' -) -# token is a signed URL valid for 15 minutes conn = psycopg2.connect( - host=RDS_HOST, port=5432, user='postgres', - password=token, dbname='postgres', + host=host, port=port, user=user, + password=password, dbname=dbname, sslmode='require', connect_timeout=10 ) ``` **Prerequisites on the client node:** -- AWS credentials with `rds-db:connect` permission (IAM role or `~/.aws/credentials`) -- `psycopg2` + `boto3` installed +- `psycopg2` installed - `libpq5` (Debian) or equivalent PostgreSQL C client library +- Network reachability to the configured `RDS_HOST` -### Fallback: RDS_PASSWORD environment variable +### Fallback: `RDS_PASSWORD` environment variable -If `boto3` is unavailable or IAM permissions fail, the provider falls back to `RDS_PASSWORD` (plain text). This is **not recommended** for production but is the historical path on nodes without AWS CLI. +If no password is configured, the provider falls back to `RDS_PASSWORD` (plain text). This is **not recommended** for production. ## MicroVM-Racknerd Deployment @@ -143,7 +141,6 @@ If `boto3` is unavailable or IAM permissions fail, the provider falls back to `R /opt/rs-surface/ ├── credential_server.py # HTTP server (v0.4) ├── credential_provider.py # 4-tier fallback logic (v0.4) -├── ene_rds_schema.py # Schema provisioning script └── __pycache__/ /etc/rs-surface/ @@ -153,10 +150,6 @@ If `boto3` is unavailable or IAM permissions fail, the provider falls back to `R /opt/credential-provider/ # STALE — v0.2 code, renamed ├── credential_provider.py.v0.2.stale └── server.py.v0.2.stale - -~/.aws/ -├── config # region = us-east-1 -└── credentials # IAM access key + secret ``` ### Systemd service @@ -221,10 +214,10 @@ journalctl -u rs-credential-server --since today --no-pager sops 4-Infrastructure/infra/secrets/credentials.json ``` -2. **Insert into RDS** (from microvm, which has IAM access): +2. **Insert into PostgreSQL** (from a node with network access): ```python - import psycopg2, boto3, json, os - # ... generate IAM token, connect ... + import psycopg2 + # ... connect using RDS_* env vars ... cur.execute(""" INSERT INTO credential_store.credentials (pkg, provider, encrypted_payload, nonce, classification, integrity_hash, is_active) @@ -260,10 +253,10 @@ sops -d 4-Infrastructure/infra/secrets/credentials.json ## Historical Note: v0.2 → v0.4 Migration -The original credential system (`/opt/credential-provider/`, v0.2) had a **hardcoded IAM token** in `credential_provider.py` with an expiration date (2026-05-17). When that token expired, `_load_from_rds()` silently returned `[]`, causing the service to fall through to environment variables. This was the "bug" that appeared to be RDS not working, but was actually an expired token in stale code. +The original credential system (`/opt/credential-provider/`, v0.2) had a **hardcoded database token** in `credential_provider.py` with an expiration date (2026-05-17). When that token expired, `_load_from_rds()` silently returned `[]`, causing the service to fall through to environment variables. This was the "bug" that appeared to be the database not working, but was actually an expired token in stale code. The fix was: -1. Deploy `credential_provider.py` v0.4 to `/opt/rs-surface/` (uses `boto3.generate_db_auth_token()` dynamically) +1. Deploy `credential_provider.py` v0.4 to `/opt/rs-surface/` (reads `RDS_PASSWORD` or `RDS_IAM_TOKEN` from environment) 2. Point systemd `ExecStart` at `/opt/rs-surface/credential_server.py` 3. Rename old v0.2 files with `.v0.2.stale` suffix diff --git a/6-Documentation/wiki/RDS-Rust-Workspace.md b/6-Documentation/wiki/RDS-Rust-Workspace.md index 72078831..9086e9d6 100644 --- a/6-Documentation/wiki/RDS-Rust-Workspace.md +++ b/6-Documentation/wiki/RDS-Rust-Workspace.md @@ -63,7 +63,7 @@ export RDS_PASSWORD=... | Variable | Default | Purpose | |----------|---------|---------| -| `RDS_HOST` | `database-1.cluster-c9i0w8eu8fnv.us-east-2.rds.amazonaws.com` | PostgreSQL host | +| `RDS_HOST` | `localhost` | PostgreSQL host | | `RDS_PORT` | `5432` | PostgreSQL port | | `RDS_USER` | `postgres` | PostgreSQL user | | `RDS_PASSWORD` | — | Password or IAM token |