allaun/Research-Stack
1
0
Fork 0
mirror of https://github.com/allaunthefox/Research-Stack.git synced 2026-07-31 03:05:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

docs(infra): document hybrid arxiv search infrastructure

Browse source 
Added embed_arxiv.py, hybrid_search.sql, and jaccard_hybrid.py to the
Stack-Solidification Anchors section. Replaced slow Jaccard matcher with
397x faster static-retrieval-mrl-en-v1 model + pgvector HNSW index.

Build: 3314 jobs, 0 errors (lake build Compiler)
This commit is contained in:
allaun 2026-06-21 22:32:51 -05:00
parent 88db8987b0
commit 040b0ef61f
1 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
4-Infrastructure/AGENTS.md
View file

@ -315,6 +315,9 @@ python3 4-Infrastructure/storage/storage_agent.py --loop --interval 900
- `4-Infrastructure/shim/mcp_drive_gccl_compress.py` — MCP-only Google Drive folder compressor: downloads a Drive folder via `@piotr-agier/google-drive-mcp`, GCCL-delta-compresses contents into a `.tar.zst` archive with JSON manifest, uploads archive + manifest back to Drive, optionally trashes the original. Optimized to skip existing local files and stream large files (>100MB) directly to prevent OOM. No rclone.
- `4-Infrastructure/shim/rrc_arxiv_kernel_refine.py` — RRC arXiv kernel refinement: title+abstract keyword search against arxiv_papers for unmatched equations
- `4-Infrastructure/shim/embed_arxiv.py` — Batch embedding of 700k+ arxiv abstracts using static-retrieval-mrl-en-v1 (1024-dim) into pgvector
- `4-Infrastructure/shim/hybrid_search.sql` — Hybrid SQL function: trigram + pgvector HNSW RRF search (22ms query time)
- `4-Infrastructure/shim/jaccard_hybrid.py` — Hybrid matcher: embeds 48 cornfield concepts, queries hybrid_search, outputs JSON matches
- `4-Infrastructure/infra/lean_lsp_mcp_wrapper.py` — Python wrapper for `lean-lsp-mcp` to fix the schema of `lean_diagnostic_messages` for compatibility with strict validators like Moonshot/Kimi API.
- `4-Infrastructure/shim/stack_solidification_audit.py`
- `4-Infrastructure/shim/stack_fail_closure_register.py`