Research-Stack/6-Documentation/docs/CompressionBenchmarkPlan.md

Compression Benchmark Plan

Purpose: Require empirical comparisons between compression methods

Status: Draft for Review
Date: 2026-04-29

---

Benchmark Requirements

1. Baseline Comparisons

Required Baselines:

• zstd: Industry-standard lossless compression
• ndzip: Domain-specific compression (if applicable)
• Delta GCL: Our custom compression
• gzip/zlib: Legacy compression for reference

Metric Requirements:

• Original size (bytes)
• Compressed size (bytes)
• Compression ratio (original/compressed)
• Compression time (seconds)
• Decompression time (seconds)
• Memory usage (MB)

2. Corpus Requirements

Test Corpora:

• Toy lattice data: Small SU(2)/SU(3) lattice configurations (L = 4-16)
• Metadata: Similar to Lean metadata (for comparison with existing achievements)
• Structured data: Swarm component metadata (for comparison with existing achievements)
• Field data: Actual gauge field configurations (for realistic assessment)

Corpus Provenance:

• Document source of each test corpus
• Document generation method (if synthetic)
• Document size and characteristics

3. Experimental Protocol

Compression Test:

1. Load original data
2. Measure original size
3. Compress with each method
4. Measure compressed size, time, memory
5. Decompress with each method
6. Verify exact reconstruction (for lossless)
7. Measure decompression time, memory

Statistical Requirements:

• Minimum 10 runs per method
• Report mean ± standard deviation
• Use SI standard compression ratio (original/compressed)

4. Acceptance Criteria

Delta GCL Requirements:

• Must outperform zstd on metadata by ≥ 10%
• Must outperform gzip on structured data by ≥ 20%
• Must achieve ≥ 2× compression on field data (conservative)
• Must have reasonable compression time (< 2× baseline)

Claims Without Benchmarks:

• 60-135× compression on field data: REQUIRES BENCHMARK
• 99.9% compression on field data: REQUIRES BENCHMARK
• "Massive" compression without comparison: REQUIRES BENCHMARK

5. Reporting Format

Required Fields:

Method: <name>
Corpus: <name>
Original Size: <bytes>
Compressed Size: <bytes>
Compression Ratio: <ratio>
Compression Time: <seconds>
Decompression Time: <seconds>
Memory Usage: <MB>

Comparison Table:

Method	Corpus	Original	Compressed	Ratio	Time	Memory
zstd	toy lattice	1MB	800KB	1.25×	0.1s	10MB
Delta GCL	toy lattice	1MB	500KB	2.0×	0.2s	15MB

---

Implementation Plan

Phase 1: Corpus Generation (1-2 weeks)

• Generate toy lattice data (L = 4-16)
• Collect metadata samples
• Collect structured data samples

Phase 2: Baseline Implementation (1 week)

• Implement zstd compression
• Implement gzip compression
• Implement ndzip (if applicable)

Phase 3: Delta GCL Integration (1 week)

• Integrate Delta GCL with benchmark framework
• Ensure fair comparison

Phase 4: Benchmark Execution (1 week)

• Run all benchmarks
• Collect metrics
• Generate comparison tables

Phase 5: Analysis (1 week)

• Analyze results
• Document findings
• Update claims based on evidence

---

Success Criteria

Benchmark Success:

• All baselines implemented and tested
• Delta GCL compared against industry standards
• Claims backed by empirical evidence
• SI standard compression ratios used

Failure Criteria:

• Claims made without benchmarks
• Only metadata used as evidence
• No comparison to industry standards
• Non-SI compression ratios reported