mirror of
https://github.com/allaunthefox/Research-Stack.git
synced 2026-07-31 03:05:21 +00:00
ac4e23dc9b
- .devcontainer/Dockerfile: add PostgreSQL client libs, OpenSSL/libffi headers, gfortran/BLAS for scipy, rclone; install full Python dependency set (boto3, psycopg2-binary, fastapi, uvicorn, notion-client, httpx, pytest, numpy, scipy, etc.) in uv-managed venv; add rclone S3 gateway init script as ENTRYPOINT - .devcontainer/devcontainer.json: switch from build to pre-built image (localhost/research
54 lines
3 KiB
Bash
54 lines
3 KiB
Bash
# restic.env — source this before any restic or backup operation
|
|
# Never commit this file (listed in .gitignore via /4-Infrastructure/storage/restic/*.env)
|
|
#
|
|
# ROLE OF EACH TOOL
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# restic — deduplicated, encrypted, content-addressed SNAPSHOTS of source trees
|
|
# and database dumps. Gives you point-in-time restore, dedup across
|
|
# backups, and cryptographic integrity verification.
|
|
# Backends: Garage (primary), rclone (secondary = gdrive or nodes)
|
|
#
|
|
# Garage — the S3-compatible object store that restic and rclone talk to.
|
|
# Provides the distributed, self-hosted storage layer across nodes.
|
|
# Buckets: research-stack (restic repo), db-scratch, rds-overflow,
|
|
# gdrive-mirror, snap-zone
|
|
#
|
|
# rclone — moves/syncs raw objects between remotes:
|
|
# gdrive ↔ Garage (gdrive-mirror bucket)
|
|
# Garage ↔ Garage nodes (cross-node sync before rf=3)
|
|
# Garage → gdrive as a cold second copy of restic repo chunks
|
|
# Does NOT deduplicate or encrypt — Garage + restic handle that.
|
|
#
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
# DATA FLOW
|
|
#
|
|
# Hot write path (every git commit, via post-commit hook):
|
|
# SQLite scratch DBs → restic backup → Garage:research-stack
|
|
# RDS overflow dumps → restic backup → Garage:rds-overflow
|
|
#
|
|
# Cold copy path (daily, via systemd timer):
|
|
# Garage:research-stack (restic chunks) ──rclone copy──▶ gdrive:restic-mirror/
|
|
# gdrive:research-stack ──rclone sync──▶ Garage:gdrive-mirror
|
|
#
|
|
# Restore path:
|
|
# restic restore <snapshot> -r s3:http://localhost:3900/research-stack
|
|
# OR if Garage is down:
|
|
# restic restore <snapshot> -r rclone:gdrive:restic-mirror/
|
|
#
|
|
# ─────────────────────────────────────────────────────────────────────────────
|
|
|
|
# Garage S3 credentials
|
|
export AWS_ACCESS_KEY_ID="GK55105d55675994caea2c2d3d"
|
|
export AWS_SECRET_ACCESS_KEY="3e8f080bc708f78b312681f6651ca6ae50cbe44de799ffb5cc65219c29143914"
|
|
export AWS_DEFAULT_REGION="garage"
|
|
|
|
# Primary restic repo: Garage
|
|
export RESTIC_REPOSITORY="s3:http://localhost:3900/research-stack"
|
|
export RESTIC_PASSWORD_FILE="/etc/garage/restic-password"
|
|
|
|
# Secondary restic repo: gdrive (via rclone backend) — cold copy
|
|
# Use with: RESTIC_REPOSITORY="$RESTIC_REPO_COLD" restic snapshots
|
|
export RESTIC_REPO_COLD="rclone:gdrive:restic-mirror"
|
|
|
|
# Garage endpoint (for scripts that use aws cli directly)
|
|
export AWS_ENDPOINT_URL="http://localhost:3900"
|