Performance
| Operation | magmalake | Reference |
|---|---|---|
| Parquet read, 1M rows — 1 core int64, double and two dictionary columns, uncompressed, four row groups | 3.77 ms p50 / 3.83 p90 — 265M rows/s | pyarrow, one thread: 8.0 ms — we are 2.1× faster |
| Parquet read, 1M rows — 4 workers the same file, ParquetReader.num_workers = 4 | 1.96 ms p50 / 2.07 p90 — 510M rows/s | pyarrow, threaded across all 10 CPUs: 2.57 ms — we are 1.3× faster while using four threads to its ten |
| Parquet read, worker scaling the same file at 1 / 2 / 4 / 8 / 10 workers | 3.77 / 2.42 / 1.96 / 1.90 / 1.90 ms | it bends at four, which is how many performance cores this M4 has. Past four the p50 buys about 3% and the p90 gets worse — 2.07 ms at four workers, 2.50 ms at eight. |
| Parquet read, 100k rows, nested and mixed int64, double, string, bool and a list<int32>, snappy — 1 core / 4 workers | 2.63 ms / 0.98 ms — 0.78 ms at eight workers | pyarrow 2.24 ms one thread, 0.66 ms threaded — 1.17× and 1.18× faster than us. Still the file we lose, but by a fifth rather than by half. |
| Parquet write, 1M rows | 32.8 ms | pyarrow 31.7 ms — parity, inside the run-to-run spread |
| Parquet footer 1,000 columns × 50 row groups = 50,000 column chunks | 56.6 ms read / 2.6 ms write | — |
scroll the table sideways →
Apple M4 — ten cores, four of them performance — macOS, stable Mojo 1.0.0, with nothing else building on the machine. Measured on 5 September 2026; the Iceberg rows on 15 September, the boundary rows on 19 September. How these were measured.
Every merge also runs its benchmarks on a GitHub runner: the benchmark history. Read it for trends; a shared runner is slower and noisier than the table.