magmalake objectstore

Tins · Storage & databases

objectstore

Storage and HTTP for Iceberg tables: a FileIO abstraction over local files, HTTP(S) range reads and S3, with the pooled HTTP transport the rest of the stack was missing.

Install 
pixi shelf add objectstore-mojo
Version

0.4.1 on mojoshelf

Source

GitHub README

What it does

Works today

  • S3 with SigV4, vended credentials, presigned URLs and multipart upload
  • Pooled libcurl transport, so range reads reuse a connection
  • Pure-Mojo SHA-256 and HMAC on hardware crypto paths
  • GCS, Azure and plain HTTP range reads alongside local files

Correctness

Checked against AWS SigV4 suite 37/37, and S3 verified end-to-end against MinIO in CI

37 of 37 cases of the official AWS SigV4 test suite, every stage.

S3 is verified end-to-end against MinIO in CI, not only on a developer's machine.

Also in Storage & databases