mirror of
https://github.com/allaunthefox/Research-Stack.git
synced 2026-07-31 03:05:21 +00:00
docs(infra): remove AWS-specific credential and RDS backend details
Update Credential-System.md and RDS-Rust-Workspace.md to describe a provider-neutral PostgreSQL backend instead of AWS RDS. Replace IAM auth examples with standard libpq env-var connection. Remove the ~/.aws/ file layout and AWS hostname defaults.
This commit is contained in:
parent
ff0e377361
commit
80ebcca11c
2 changed files with 29 additions and 36 deletions
|
|
@ -2,7 +2,7 @@
|
|||
|
||||
> **Canonical source**: `4-Infrastructure/infra/ene-session-sync/src/credential.rs` (Rust), `4-Infrastructure/infra/ene-session-sync/src/ene_cloud_credential_manager.rs` (Rust), `4-Infrastructure/infra/recover_credential_server.sh` (deployment script)
|
||||
> **Runtime host**: `microvm-racknerd` (100.101.247.127) — Debian 13 VM on RackNerd
|
||||
> **Primary backend**: AWS RDS PostgreSQL (`database-1-instance-1.cghu8yqogqwo.us-east-1.rds.amazonaws.com`)
|
||||
> **Primary backend**: PostgreSQL (configured via `RDS_*` environment variables)
|
||||
> **Fallback chain**: RDS → remote server → local JSON → environment variables
|
||||
> **SOPS key**: `age1tp4vr565zkmvnyulatpyaj6z8zrz7q9mpaypz85yz8rty99crdasualxyr`
|
||||
|
||||
|
|
@ -26,9 +26,9 @@
|
|||
│ ┌───────────────┼───────────────┐ │
|
||||
│ │ │ │ │
|
||||
│ ┌────────▼────────┐ ┌───▼────────┐ ┌────▼───────┐ │
|
||||
│ │ AWS RDS │ │ Remote │ │ Local │ │
|
||||
│ │ PostgreSQL │ │ Server │ │ JSON │ │
|
||||
│ │ (primary) │ │ (chain) │ │ (backup) │ │
|
||||
│ │ PostgreSQL │ │ Remote │ │ Local │ │
|
||||
│ │ (primary) │ │ Server │ │ JSON │ │
|
||||
│ │ │ │ (chain) │ │ (backup) │ │
|
||||
│ └────────┬────────┘ └────────────┘ └────────────┘ │
|
||||
│ │ │
|
||||
│ ┌────────▼────────┐ │
|
||||
|
|
@ -45,14 +45,14 @@ The `credential_provider.py` (`v0.4`) implements a strict priority order. The fi
|
|||
|
||||
| Tier | Function | Trigger | Data Source |
|
||||
|------|----------|---------|-------------|
|
||||
| **1** | `_load_from_rds()` | Always runs first | AWS RDS PostgreSQL `credential_store.credentials` |
|
||||
| **1** | `_load_from_rds()` | Always runs first | PostgreSQL `credential_store.credentials` |
|
||||
| **2** | `_load_from_remote()` | Skipped if RDS empty AND `RS_CREDENTIAL_SERVER` points to self | Another credential server URL |
|
||||
| **3** | `_load_from_config()` | Skipped if tiers 1–2 empty | `/etc/rs-surface/credentials.json` |
|
||||
| **4** | `_load_from_env()` | Last resort | Environment variables per `PROVIDER_ENV_MAP` |
|
||||
|
||||
### Why RDS is the primary source
|
||||
### Why PostgreSQL is the primary source
|
||||
|
||||
RDS credentials are **encrypted at rest** (AES-256-GCM payload), **IAM-authenticated** (temporary tokens), and **auditable** (`access_log` table). The local JSON file (`/etc/rs-surface/credentials.json`) is a plaintext fallback for disaster recovery only.
|
||||
PostgreSQL credentials are **encrypted at rest** (AES-256-GCM payload) and **auditable** (`access_log` table). The local JSON file (`/etc/rs-surface/credentials.json`) is a plaintext fallback for disaster recovery only.
|
||||
|
||||
**Current status** (2026-05-21): Tier 1 (RDS) is active and serving all 11 credentials. Tier 3 (local JSON) exists as a warm standby but is not consulted because RDS succeeds.
|
||||
|
||||
|
|
@ -91,7 +91,6 @@ curl -s http://100.101.247.127:8444/status | jq .
|
|||
|
||||
| Provider | Classification | pkg | Notes |
|
||||
|----------|---------------|-----|-------|
|
||||
| aws | 3 (secret) | credentials/aws | |
|
||||
| bedrock | 3 | credentials/bedrock | |
|
||||
| deepseek | 3 | credentials/deepseek | |
|
||||
| linear | 3 | credentials/linear | |
|
||||
|
|
@ -104,36 +103,35 @@ curl -s http://100.101.247.127:8444/status | jq .
|
|||
| wolfram_alpha | 2 | credentials/wolfram_alpha | |
|
||||
| **authentik** | 3 | credentials/authentik | LLM controller token for Authentik API |
|
||||
|
||||
## Authentication to RDS
|
||||
## Authentication to PostgreSQL
|
||||
|
||||
### Method: IAM Authentication Token (preferred)
|
||||
Use a standard libpq connection string built from `RDS_*` environment variables:
|
||||
|
||||
```python
|
||||
import boto3
|
||||
import os
|
||||
import psycopg2
|
||||
|
||||
host = os.environ.get('RDS_HOST', 'localhost')
|
||||
port = os.environ.get('RDS_PORT', '5432')
|
||||
user = os.environ.get('RDS_USER', 'postgres')
|
||||
password = os.environ.get('RDS_PASSWORD', '')
|
||||
dbname = os.environ.get('RDS_DB', 'postgres')
|
||||
|
||||
client = boto3.client('rds', region_name='us-east-1')
|
||||
token = client.generate_db_auth_token(
|
||||
DBHostname='database-1-instance-1.cghu8yqogqwo.us-east-1.rds.amazonaws.com',
|
||||
Port=5432,
|
||||
DBUsername='postgres',
|
||||
Region='us-east-1'
|
||||
)
|
||||
# token is a signed URL valid for 15 minutes
|
||||
conn = psycopg2.connect(
|
||||
host=RDS_HOST, port=5432, user='postgres',
|
||||
password=token, dbname='postgres',
|
||||
host=host, port=port, user=user,
|
||||
password=password, dbname=dbname,
|
||||
sslmode='require', connect_timeout=10
|
||||
)
|
||||
```
|
||||
|
||||
**Prerequisites on the client node:**
|
||||
- AWS credentials with `rds-db:connect` permission (IAM role or `~/.aws/credentials`)
|
||||
- `psycopg2` + `boto3` installed
|
||||
- `psycopg2` installed
|
||||
- `libpq5` (Debian) or equivalent PostgreSQL C client library
|
||||
- Network reachability to the configured `RDS_HOST`
|
||||
|
||||
### Fallback: RDS_PASSWORD environment variable
|
||||
### Fallback: `RDS_PASSWORD` environment variable
|
||||
|
||||
If `boto3` is unavailable or IAM permissions fail, the provider falls back to `RDS_PASSWORD` (plain text). This is **not recommended** for production but is the historical path on nodes without AWS CLI.
|
||||
If no password is configured, the provider falls back to `RDS_PASSWORD` (plain text). This is **not recommended** for production.
|
||||
|
||||
## MicroVM-Racknerd Deployment
|
||||
|
||||
|
|
@ -143,7 +141,6 @@ If `boto3` is unavailable or IAM permissions fail, the provider falls back to `R
|
|||
/opt/rs-surface/
|
||||
├── credential_server.py # HTTP server (v0.4)
|
||||
├── credential_provider.py # 4-tier fallback logic (v0.4)
|
||||
├── ene_rds_schema.py # Schema provisioning script
|
||||
└── __pycache__/
|
||||
|
||||
/etc/rs-surface/
|
||||
|
|
@ -153,10 +150,6 @@ If `boto3` is unavailable or IAM permissions fail, the provider falls back to `R
|
|||
/opt/credential-provider/ # STALE — v0.2 code, renamed
|
||||
├── credential_provider.py.v0.2.stale
|
||||
└── server.py.v0.2.stale
|
||||
|
||||
~/.aws/
|
||||
├── config # region = us-east-1
|
||||
└── credentials # IAM access key + secret
|
||||
```
|
||||
|
||||
### Systemd service
|
||||
|
|
@ -221,10 +214,10 @@ journalctl -u rs-credential-server --since today --no-pager
|
|||
sops 4-Infrastructure/infra/secrets/credentials.json
|
||||
```
|
||||
|
||||
2. **Insert into RDS** (from microvm, which has IAM access):
|
||||
2. **Insert into PostgreSQL** (from a node with network access):
|
||||
```python
|
||||
import psycopg2, boto3, json, os
|
||||
# ... generate IAM token, connect ...
|
||||
import psycopg2
|
||||
# ... connect using RDS_* env vars ...
|
||||
cur.execute("""
|
||||
INSERT INTO credential_store.credentials
|
||||
(pkg, provider, encrypted_payload, nonce, classification, integrity_hash, is_active)
|
||||
|
|
@ -260,10 +253,10 @@ sops -d 4-Infrastructure/infra/secrets/credentials.json
|
|||
|
||||
## Historical Note: v0.2 → v0.4 Migration
|
||||
|
||||
The original credential system (`/opt/credential-provider/`, v0.2) had a **hardcoded IAM token** in `credential_provider.py` with an expiration date (2026-05-17). When that token expired, `_load_from_rds()` silently returned `[]`, causing the service to fall through to environment variables. This was the "bug" that appeared to be RDS not working, but was actually an expired token in stale code.
|
||||
The original credential system (`/opt/credential-provider/`, v0.2) had a **hardcoded database token** in `credential_provider.py` with an expiration date (2026-05-17). When that token expired, `_load_from_rds()` silently returned `[]`, causing the service to fall through to environment variables. This was the "bug" that appeared to be the database not working, but was actually an expired token in stale code.
|
||||
|
||||
The fix was:
|
||||
1. Deploy `credential_provider.py` v0.4 to `/opt/rs-surface/` (uses `boto3.generate_db_auth_token()` dynamically)
|
||||
1. Deploy `credential_provider.py` v0.4 to `/opt/rs-surface/` (reads `RDS_PASSWORD` or `RDS_IAM_TOKEN` from environment)
|
||||
2. Point systemd `ExecStart` at `/opt/rs-surface/credential_server.py`
|
||||
3. Rename old v0.2 files with `.v0.2.stale` suffix
|
||||
|
||||
|
|
|
|||
|
|
@ -63,7 +63,7 @@ export RDS_PASSWORD=...
|
|||
|
||||
| Variable | Default | Purpose |
|
||||
|----------|---------|---------|
|
||||
| `RDS_HOST` | `database-1.cluster-c9i0w8eu8fnv.us-east-2.rds.amazonaws.com` | PostgreSQL host |
|
||||
| `RDS_HOST` | `localhost` | PostgreSQL host |
|
||||
| `RDS_PORT` | `5432` | PostgreSQL port |
|
||||
| `RDS_USER` | `postgres` | PostgreSQL user |
|
||||
| `RDS_PASSWORD` | — | Password or IAM token |
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue