# iceberg-rs API reference > The `iceberg_rs` package, from iceberg-rs 0.1.3, as published on mojoshelf (commit 096434e40a0d), by `mojo doc` from Mojo 1.2.0. `iceberg_rs` — Apache Iceberg for Mojo, over a Rust cdylib wrapping iceberg-rust. Mirrors lancedb.mojo's FFI pattern: one relocatable library (`ffi/src/*.rs` -> `$CONDA_PREFIX/lib/libicebergrsmojo.{dylib,so}`, built by the `iceberg-shim` pixi package) loaded through an `OwnedDLHandle`. The handle is passed as a BORROWED `read` param to every worker so Mojo's ASAP destruction can't `dlclose` the library mid-call (the flare gotcha) — even though each struct also owns a handle as a field, the borrow keeps it pinned across the C call. Nothing but integers, raw buffers and UTF-8 bytes crosses FFI. Iceberg handles (catalog, table, scan, batch) are opaque `Int` pointers; results that don't fit in a register come back as heap C strings which `_take_string` copies out through `ib_string_len` / `ib_string_copy` and frees — so no Mojo code ever dereferences a foreign pointer. from iceberg_rs import Catalog var cat = Catalog.sql("sqlite:/tmp/cat.db?mode=rwc", "file:///tmp/warehouse") cat.create_namespace("sales") var t = cat.create_table("sales", "orders", SCHEMA_JSON, SPEC_JSON) var b = t.builder() b.int_col("id", [Int64(1), Int64(2)]) b.str_col("region", ["eu", "us"]) t.append(b.build()) var snapshot = t.commit() var scan = t.scan(columns="id,region", filter='["=","region","eu"]') for batch in scan.batches(): print(batch.int_col("id")) ## Structs ### `Batch` ```mojo struct Batch ``` One Arrow record batch owned by the Rust side. Columns are copied out on demand and cast on the way: `int_col` reads any integer, date, time, timestamp or boolean column as `Int64` (timestamps in microseconds since the epoch, dates in days), `float_col` reads any numeric column as `Float64`, `str_col` reads any column as UTF-8. Implements: `Movable` Fields: - `lib: OwnedDLHandle` - `ptr: Int` #### `__init__` ```mojo def __init__(out self, var lib: OwnedDLHandle, ptr: Int) ``` #### `__deinit__` ```mojo def __deinit__(deinit self) ``` #### `num_rows` ```mojo def num_rows(self) -> Int ``` #### `num_columns` ```mojo def num_columns(self) -> Int ``` #### `schema_json` ```mojo def schema_json(self) -> String ``` JSON array of `{"name", "type", "nullable"}` for the batch's columns. #### `int_col` ```mojo def int_col(self, name: String) -> List[Int64] ``` Column `name` as `Int64`. Nulls read back as 0 — use `validity` to tell them apart. #### `float_col` ```mojo def float_col(self, name: String) -> List[Float64] ``` Column `name` as `Float64`. Nulls read back as 0.0. #### `bool_col` ```mojo def bool_col(self, name: String) -> List[Bool] ``` Column `name` as `Bool`. Nulls read back as False. #### `str_col` ```mojo def str_col(self, name: String) -> List[String] ``` Column `name` as UTF-8 strings. Nulls read back as empty strings. #### `validity` ```mojo def validity(self, name: String) -> List[Bool] ``` One flag per row of column `name`: True when the value is present. #### `export_c_data` ```mojo def export_c_data(self) -> ArrowCData ``` Export this batch through the Arrow C Data Interface as a struct array (one child per column) — the shape marrow's `c_data.mojo` imports. ### `ArrowCData` ```mojo struct ArrowCData ``` Storage for one Arrow C Data Interface struct pair. The two structs live in `Int64`-backed Mojo buffers (guaranteed 8-byte aligned, sized from the shim's own `sizeof`), so a consumer such as marrow can read them straight from `array_ptr()` / `schema_ptr()`. Releasing is automatic: the destructor calls the structs' `release` callbacks through `ib_arrow_release`. Implements: `Movable` Fields: - `lib: OwnedDLHandle` - `array_buf: List[Int64]` - `schema_buf: List[Int64]` - `live: Bool` #### `__init__` ```mojo def __init__(out self, var lib: OwnedDLHandle, sizer: OwnedDLHandle) ``` #### `__deinit__` ```mojo def __deinit__(deinit self) ``` #### `array_ptr` ```mojo def array_ptr(self) -> Int ``` Address of the `ArrowArray` struct. #### `schema_ptr` ```mojo def schema_ptr(self) -> Int ``` Address of the `ArrowSchema` struct. ### `Scan` ```mojo struct Scan ``` A planned table scan. Pull batches with `next()` until it returns an empty `Optional`, or inspect the plan with `plan_files()` without reading any data. Implements: `Movable` Fields: - `lib: OwnedDLHandle` - `ptr: Int` #### `__init__` ```mojo def __init__(out self, var lib: OwnedDLHandle, ptr: Int) ``` #### `__deinit__` ```mojo def __deinit__(deinit self) ``` #### `plan_files` ```mojo def plan_files(self) -> String ``` JSON array of the file scan tasks: data file path, format, record count, byte range, and any delete files that apply to each. #### `next` ```mojo def next(mut self) -> Optional[Batch] ``` The next record batch, or `None` at the end of the stream. #### `next_c_data` ```mojo def next_c_data(mut self) -> Optional[ArrowCData] ``` The next record batch straight through the Arrow C Data Interface, skipping the opaque handle entirely. #### `batches` ```mojo def batches(mut self) -> List[Batch] ``` Drain the scan into a list of batches. Convenient for the small results a parity check produces; stream with `next()` for large tables. ### `BatchBuilder` ```mojo struct BatchBuilder ``` Builds one record batch column by column against a table's schema. Each `*_col` call names a column of the table; the values are cast to the column's declared type on the Rust side, so `int_col` feeds `int`, `long`, `date` (days) and `timestamp` (microseconds) columns alike. Columns you leave out must be optional in the schema; they are filled with nulls. Implements: `Movable` Fields: - `lib: OwnedDLHandle` - `ptr: Int` #### `__init__` ```mojo def __init__(out self, var lib: OwnedDLHandle, ptr: Int) ``` #### `__deinit__` ```mojo def __deinit__(deinit self) ``` #### `int_col` ```mojo def int_col(mut self, name: String, values: List[Int64]) ``` ```mojo def int_col(mut self, name: String, values: List[Int64], valid: List[Bool]) ``` #### `float_col` ```mojo def float_col(mut self, name: String, values: List[Float64]) ``` ```mojo def float_col(mut self, name: String, values: List[Float64], valid: List[Bool]) ``` #### `bool_col` ```mojo def bool_col(mut self, name: String, values: List[Bool]) ``` ```mojo def bool_col(mut self, name: String, values: List[Bool], valid: List[Bool]) ``` #### `str_col` ```mojo def str_col(mut self, name: String, values: List[String]) ``` ```mojo def str_col(mut self, name: String, values: List[String], valid: List[Bool]) ``` #### `build` ```mojo def build(self) -> Batch ``` Assemble the staged columns into a `Batch`. ### `Table` ```mojo struct Table ``` A loaded Iceberg table: metadata, scans, and append/commit. iceberg-rust 0.10 is append-only. `append` writes Parquet data files right away and stages them; `commit` turns everything staged into exactly one new snapshot and returns its id. Implements: `Movable` Fields: - `lib: OwnedDLHandle` - `ptr: Int` #### `__init__` ```mojo def __init__(out self, var lib: OwnedDLHandle, ptr: Int) ``` #### `__deinit__` ```mojo def __deinit__(deinit self) ``` #### `refresh` ```mojo def refresh(mut self) ``` Re-load from the catalog, picking up snapshots committed elsewhere. #### `metadata_json` ```mojo def metadata_json(self) -> String ``` The table's full current `metadata.json` text. #### `schema_json` ```mojo def schema_json(self) -> String ``` The current schema in Iceberg metadata JSON form. #### `partition_spec_json` ```mojo def partition_spec_json(self) -> String ``` The default partition spec as JSON. #### `snapshots_json` ```mojo def snapshots_json(self) -> String ``` JSON array of the table's snapshots, oldest first. #### `location` ```mojo def location(self) -> String ``` The table's base location. #### `current_snapshot_id` ```mojo def current_snapshot_id(self) -> Int64 ``` The current snapshot id, or 0 when the table has no snapshot yet. #### `builder` ```mojo def builder(self) -> BatchBuilder ``` A `BatchBuilder` bound to this table's schema. #### `append` ```mojo def append(mut self, batch: Batch) ``` Stage `batch` for append: writes its Parquet data file(s) now and remembers them until `commit`. #### `append_c_data` ```mojo def append_c_data(mut self, mut data: ArrowCData) ``` Stage an Arrow C Data Interface struct pair for append. Consumes the pair — `data` is inert afterwards. #### `pending_files` ```mojo def pending_files(self) -> Int ``` Data files staged but not yet committed. #### `commit` ```mojo def commit(mut self) -> Int64 ``` Commit every staged append as one new snapshot; returns its id. With nothing staged this is a no-op returning the current snapshot id. #### `rollback` ```mojo def rollback(mut self) ``` Drop every staged (uncommitted) append. #### `scan` ```mojo def scan(self) -> Scan ``` Scan every column of the current snapshot. ```mojo def scan(self, columns: String) -> Scan ``` Scan a comma-separated column projection of the current snapshot. ```mojo def scan(self, columns: String, filter: String) -> Scan ``` Scan a projection of the current snapshot under a filter (see the filter DSL in the README). ```mojo def scan(self, columns: String, snapshot_id: Int64, filter: String) -> Scan ``` Scan `columns` (empty for all) of `snapshot_id` (0 for current) under `filter` (empty for none). ### `Catalog` ```mojo struct Catalog ``` An Iceberg catalog: REST, or SQL over sqlite/postgres/mysql. Implements: `Movable` Fields: - `lib: OwnedDLHandle` - `ptr: Int` #### `__init__` ```mojo def __init__(out self, var lib: OwnedDLHandle, ptr: Int) ``` #### `__deinit__` ```mojo def __deinit__(deinit self) ``` #### `sql` ```mojo def sql(uri: String, warehouse: String) -> Self ``` A SQL catalog. `uri` is an sqlx connection string (`sqlite:/abs/path/catalog.db?mode=rwc`, `postgres://…`, `mysql://…`) and `warehouse` is the data root (`file:///…`, `s3://…`). The sqlite flavour needs no infrastructure at all — it is what the test-suite uses. #### `rest` ```mojo def rest(uri: String) -> Self ``` A REST catalog with no warehouse hint and no extra properties. ```mojo def rest(uri: String, warehouse: String, props_json: String) -> Self ``` A REST catalog. `props_json` is a JSON object of extra properties passed to the REST client — OAuth2 (`credential`, `oauth2-server-uri`, `scope`, `token`) and `header.*` entries go here. See `examples/rest_polaris.md`. #### `list_namespaces` ```mojo def list_namespaces(self) -> String ``` JSON array of top-level namespaces, each a dotted string. ```mojo def list_namespaces(self, parent: String) -> String ``` JSON array of the namespaces under `parent` (empty for top level). #### `create_namespace` ```mojo def create_namespace(mut self, ns: String) ``` ```mojo def create_namespace(mut self, ns: String, props_json: String) ``` #### `namespace_exists` ```mojo def namespace_exists(self, ns: String) -> Bool ``` #### `drop_namespace` ```mojo def drop_namespace(mut self, ns: String) ``` #### `list_tables` ```mojo def list_tables(self, ns: String) -> String ``` JSON array of the table names in `ns`. #### `table_exists` ```mojo def table_exists(self, ns: String, name: String) -> Bool ``` #### `load_table` ```mojo def load_table(self, ns: String, name: String) -> Table ``` #### `drop_table` ```mojo def drop_table(mut self, ns: String, name: String) ``` #### `create_table` ```mojo def create_table(mut self, ns: String, name: String, schema_json: String) -> Table ``` Create an unpartitioned table. `schema_json` is Iceberg's metadata JSON schema form. ```mojo def create_table(mut self, ns: String, name: String, schema_json: String, partition_spec_json: String) -> Table ``` Create a table with a partition spec (unbound partition spec JSON). ```mojo def create_table(mut self, ns: String, name: String, schema_json: String, partition_spec_json: String, props_json: String) -> Table ``` ## Functions ### `version` ```mojo def version() -> String ``` The version of the `iceberg` Rust crate the shim was built against.