Commit graph

4 commits

Author SHA1 Message Date
772a2acc4d fix(scripts): pipe SQL via stdin for neon-64gb psql
Avoid shell-escaping issues by using podman exec -i and passing SQL
via stdin instead of command line interpolation.
2026-06-22 05:08:05 -05:00
aefd06dc84 chore(cleanup): remove shared-data archive lean_binned from repo
These directories were tracked despite gitignore patterns. Removing them
frees ~120 GB (shared-data), ~1.5 GB (archive), and ~7 GB (lean_binned).
They are either offloaded to Garage S3/Google Drive or superseded
proof artifacts.
2026-06-22 05:01:33 -05:00
5cfa8f0898 chore(cleanup): remove ingested corpus artifacts and scratch scripts
- Delete 4-Infrastructure/shim/lean_corpus/ (downloaded third-party corpus).
- Delete shared-data/data/blockchain_corpus/ receipts (already ingested/offloaded).
- Delete scratch/ bulk-download scripts (superseded by arxiv_crossref_stream.py).
- Clean __pycache__ directories outside .venv.
- Also fix populate_ene_tables.py to pipe SQL via stdin instead of shell
  escaping for neon-64gb psql execution.
2026-06-22 04:00:32 -05:00
e02786e310 feat(scripts): Gremlin escaping + ENE table population from extraction
- distribute_extraction.py: escape strings and batch Gremlin vertex upserts.
- populate_ene_tables.py: load extraction JSONs and map concepts into ENE
tables on neon-64gb (packages, receipts, sidon_labels, braid_strands,
eigensolid_snapshots, gossip_surface_nodes/edges, relations, sessions,
ingest_events).
2026-06-22 03:52:06 -05:00