Tins · brotli

API reference

The brotli package, from brotli 0.1.4, as published on mojoshelf (commit dcbf840d22e3), by mojo doc from Mojo 1.2.0. Source.

brotli — Mojo bindings to libbrotlienc/libbrotlidec (RFC 7932) via a thin C shim (libbrotlimojo.{dylib,so}), loaded through an OwnedDLHandle.

Brotli is a Parquet page codec (CompressionCodec.BROTLI) and the last one magmalake could not read. A Brotli stream records no uncompressed size anywhere — not in a header, not in a trailer — which shapes this API:

  • decompress(data, uncompressed_size) is the Parquet path. A page header carries uncompressed_page_size, so the output buffer can be sized exactly and the stream decoded in a single pass.
  • decompress_unsized(data) is for everything else (a bare .br file, a blob from somewhere that did not record a length). It guesses a capacity and doubles it until the stream fits, re-decoding from the start each time — Brotli's one-shot decoder cannot resume, so a short buffer costs a full retry. Use the sized form whenever the size is known.

Mirrors zstd.mojo's and lz4.mojo's FFI pattern: a single-call C wrapper (shim/brotli_wrapper.c, built to $CONDA_PREFIX/lib/libbrotlimojo.{dylib,so} by the brotli-shim pixi package) so Mojo never reads back internal library state after a foreign call. The handle is opened once per process and never closed (_LIB below) — a dlopen/dlclose cycle costs hundreds of microseconds, far more than decompressing a Parquet page, so opening one per call made the binding's fixed cost dwarf the work it wrapped. It is passed as an imm borrow to each worker function, so Mojo's ASAP destruction cannot dlclose the library between get_function and the call it returned.

Functions

compress

def compress(data: Span[UInt8], quality: Int = Int(11), lgwin: Int = Int(22), mode: Int = Int(0)) -> List[UInt8]

Compress data into a single Brotli stream.

Zero-length data is a real (if degenerate) case: Brotli emits a valid one-byte stream that decodes back to nothing.

Args
data: Bytes to compress.
quality: 0..11; higher is slower and smaller. 11 is libbrotli's own default and is very slow on large inputs — 4 or 5 is the usual choice for Parquet pages.
lgwin: Sliding-window size in bits, 10..24 (22 = 4 MiB).
mode: MODE_GENERIC, MODE_TEXT, or MODE_FONT — a hint that selects the encoder's context model.

decompress_into

def decompress_into(data: Span[UInt8], dst: Span[UInt8]) -> Int

Decompress a Brotli stream into a caller-supplied buffer, returning the number of bytes written.

Raises if dst is too small — Brotli records no uncompressed size, so this cannot grow the buffer for you. Use decompress_unsized when the size is genuinely unknown.

decompress

def decompress(data: Span[UInt8], uncompressed_size: Int) -> List[UInt8]

Decompress a Brotli stream whose decoded length is already known — the Parquet page path, where uncompressed_page_size comes from the page header. uncompressed_size must be exact; a stream that decodes to a different length is treated as corrupt.

decompress_unsized

def decompress_unsized(data: Span[UInt8]) -> List[UInt8]

Decompress a Brotli stream of unknown decoded length.

Starts from a guess and doubles until the stream fits. Each retry decodes from the beginning — Brotli's decoder cannot resume a one-shot call into a bigger buffer — so prefer decompress whenever the size is known out of band, which for Parquet it always is.

encoder_version

def encoder_version() -> String

The libbrotlienc version this process is linked against, e.g. "1.1.0".

decoder_version

def decoder_version() -> String

The libbrotlidec version this process is linked against, e.g. "1.1.0".

Aliases

comptime MODE_GENERIC = 0

No assumption about the input (BROTLI_MODE_GENERIC). The Parquet default.

comptime MODE_TEXT = 1

UTF-8 text (BROTLI_MODE_TEXT).

comptime MODE_FONT = 2

WOFF 2.0 font data (BROTLI_MODE_FONT).

comptime MIN_QUALITY = 0
comptime MAX_QUALITY = 11
comptime DEFAULT_QUALITY = 11

BROTLI_DEFAULT_QUALITY. Slow and small; 4–5 is the usual Parquet trade.

comptime MIN_WINDOW_BITS = 10
comptime MAX_WINDOW_BITS = 24
comptime DEFAULT_WINDOW_BITS = 22

BROTLI_DEFAULT_WINDOW — a 4 MiB sliding window.