# Docs

> How to use snappy, a magmalake tin.

[magmalake](https://magmalake.dev/) [snappy](https://magmalake.dev/s/snappy)

## What it’s for

snappy is Snappy written in Mojo: the raw block format Parquet and Avro compress with, and the `sNaPpY` framing format with a CRC-32C per chunk. [parquet](https://magmalake.dev/s/parquet) reads and writes `SNAPPY` pages with it and [avro](https://magmalake.dev/s/avro) uses it for the `snappy` codec. Unlike the other codecs it has no C library and no shim, so there is nothing to build or load.

## Install

A new project needs Modular’s channel, pixi’s build preview and the compiler before the tin:

```
pixi init app -c https://conda.modular.com/max -c conda-forge && cd app
sed -i.bak 's/^\[workspace\]/[workspace]\npreview = ["pixi-build"]/' pixi.toml
pixi add "mojo-compiler==1.2.0"
pixi shelf add snappy-mojo     # no shelf dependencies, no C library
```

## Quick start

Compress a page as a raw block and as a framed stream, and decompress both:

```
from snappy import (
    compress, compress_framed, crc32c, decompress, decompress_framed,
    uncompressed_length,
)

def main() raises:
    var page = List[UInt8]()
    for i in range(100_000):
        page.append(UInt8(i % 7))

    # Raw block, as in Parquet and Avro: the length leads the block.
    var packed = compress(Span(page))
    print("raw:", len(packed), "bytes for", uncompressed_length(Span(packed)))
    print(decompress(Span(packed)) == page)

    # Framing format: sNaPpY stream, CRC-32C checked per 64 KiB chunk.
    var stream = compress_framed(Span(page))
    print("framed:", len(stream), "bytes,", decompress_framed(Span(stream)) == page)
    print(hex(crc32c("123456789".as_bytes())))   # 0xe3069283
```

## API

Everything is exported from `snappy`.

**Raw blocks**, which begin with the decoded length as a varint:

- `compress(data: Span[UInt8, _]) -> List[UInt8]`: one block; it never fails.
- `decompress(data: Span[UInt8, _]) raises -> List[UInt8]`: one block, sized from its own header.
- `decompress_into[origin: Origin[mut=True], //](data: Span[UInt8, _], dst: Span[UInt8, origin]) raises -> Int`: into a buffer of at least `uncompressed_length(data)` bytes; returns the bytes written.
- `uncompressed_length(data: Span[UInt8, _]) raises -> Int`: the decoded length, read from the header without decompressing.
- `max_compressed_length(n: Int) -> Int`: the worst case, `32 + n + n / 6`, as in the C++ reference.
- `validate(data: Span[UInt8, _]) -> Bool`: whether `data` decompresses without error; it decodes the whole block to find out.

**Framing format**:

- `compress_framed(data: Span[UInt8, _]) -> List[UInt8]` and `decompress_framed(data: Span[UInt8, _]) raises -> List[UInt8]`: a whole stream, in 64 KiB chunks, every checksum verified.
- `FramedWriter()`: `write(mut self, data: Span[UInt8, _])` as often as you like, then `finish(deinit self) -> List[UInt8]`.
- `FramedReader(data: Span[UInt8, _])`: `read_chunk(mut self) raises -> Optional[List[UInt8]]` returns each chunk’s bytes, then None.

**CRC-32C**: `crc32c(data: Span[UInt8, _], seed: UInt32 = 0) -> UInt32`, the incremental `Crc32c` with `update` and `finish`, and the framing format’s `mask_crc32c` and `unmask_crc32c`.

## Errors and limits

- Corrupt input raises `snappy: <what>`: `truncated varint`, `invalid copy offset`, `length mismatch`, `destination buffer too small`. The framing reader adds `snappy: crc32c mismatch`, `bad stream identifier` and `unsupported unskippable chunk type`; padding and skippable chunks are passed over.
- The compressor is a single-pass greedy matcher with no lazy matching. Its element encoding follows the C++ reference, but its output isn’t guaranteed to match libsnappy’s byte for byte.
- `FramedReader` copies its whole input when you construct it; it is incremental on the output side only.

## Tests

`pixi run test` checks the CRC-32C check value, the spec’s edge cases (long literals, overlapping copies), 64 KiB boundaries, 1 MiB round trips, corrupt input and bad checksums, and decodes a block produced by `python-snappy` and baked in as a constant. CI runs it on Linux and macOS with Mojo 1.2.0 and nightly.

The full reference is [the README on GitHub](https://github.com/magmalake/snappy.mojo#readme).

[A magmalake project](https://magmalake.dev/)

Apache 2.0 · © 2026 Marius Seritan