The Secret is Parquet: The Columnar Substrate→
You cannot build modern intelligence on a 20-year-old row database. The file itself is the database.
$duckdb -c "SELECT custodian, count(*) AS docs, round(sum(size_bytes)/1024/1024, 1) AS mb FROM 'evidence/showcase_manifest.parquet' GROUP BY custodian ORDER BY docs DESC LIMIT 5;"
Inspect In-Process DuckDB Pushdown Kernel ▾
# Zero-copy projection: reads only requested columns, skipping unreferenced text blobs
import duckdb
con = duckdb.connect()
top_custodians = con.execute("""
SELECT custodian, count(*) as docs, round(sum(size_bytes)/1024/1024, 1) as mb
FROM read_parquet('evidence/showcase_manifest.parquet')
GROUP BY custodian ORDER BY docs DESC LIMIT 5
""").fetchall()
Co-Located Embedded Engines: Eliminating Cloud SaaS Sprawl→
Stop scattering your data across five hosted SaaS tools. Run embedded directly over the memory bus.
Inspect Zero-Copy PyArrow C-Buffer Kernel ▾
# Bypasses .tolist() heap unpacking: contiguous C-buffer into Arrow FixedSizeListArray
contiguous = np.ascontiguousarray(embeddings_matrix, dtype=np.float32)
flat_buf = pa.py_buffer(contiguous.ravel())
arrow_vectors = pa.FixedSizeListArray.from_arrays(
pa.Array.from_buffers(pa.float32(), contiguous.size, [None, flat_buf]), dim=256
)
Workstation Physics: Apple Silicon as a Personal Supercomputer→
A single Mac Studio has 800 GB/s unified memory bandwidth. Distributed cloud clusters are billing theater.
Sovereign & Air-Gapped: Local Execution Without Compromise→
Local is not a budget compromise. It is the most legally defensible architecture for privileged data.
Real Predictive Coding: Beyond the Linear Hyperplane→
TAR 2.0 with a linear hyperplane is a coin flip. Real predictive coding requires non-linear decision trees on chunks.