`iceberg.catalog.rest`

Iceberg REST catalog — URLs, headers, responses, and the socket.

The socket is objectstore.mojo's HttpClient, a libcurl-backed client behind a fixed-arity C shim. It exists because no Mojo HTTP package resolves from conda on either of this repo's environments (floki, flare, lightbug-http and fire-http were all tried; all reported "no candidates were found"), and an Iceberg REST catalog is HTTPS or nothing.

Three layers, separable and separately tested:

RestCatalogConfig builds the URLs and headers for GET /v1/config, loadTable, listNamespaces and listTables, including the prefix a config response can prepend to every subsequent path, bearer authorization, and the X-Iceberg-Access-Delegation: vended-credentials header.
LoadTableResult parses a loadTable response body — the inline metadata, its metadata-location, per-table config, and storage-credentials — into a real TableMetadata.
RestCatalog puts a client behind them, maps HTTP status codes onto Iceberg error types, and hands back a Table whose FileIO is already configured with whatever the catalog vended.

Structs

`StorageCredential`

struct StorageCredential

One entry of a loadTable response's storage-credentials.

Implements Copyable, Movable

Fields: prefix: String; config: Dict[String, String]

`LoadTableResult`

struct LoadTableResult

A parsed LoadTableResult body.

Implements Copyable, Movable

Fields: metadata_location: String; has_metadata_location: Bool; metadata: TableMetadata; config: Dict[String, String]; storage_credentials: List[StorageCredential]

`init`

def __init__(out self, var metadata_location: String, has_metadata_location: Bool, var metadata: TableMetadata, var config: Dict[String, String], var storage_credentials: List[StorageCredential])

`parse`

def parse(body: String) -> Self

`RestCatalogConfig`

struct RestCatalogConfig

URL and header construction for the REST catalog API.

Implements Copyable, Movable

Fields: uri: String; prefix: String: The prefix a GET /v1/config response asks clients to insert.; token: String; has_token: Bool; warehouse: String; vend_credentials: Bool; extra_headers: List[Header]

`init`

def __init__(out self, var uri: String)

`with_token`

def with_token(mut self, var token: String)

`with_warehouse`

def with_warehouse(mut self, var warehouse: String)

`config_url`

def config_url(self) -> String

GET /v1/config — always unprefixed; it is what supplies a prefix.

`namespaces_url`

def namespaces_url(self, parent: String = "") -> String

`tables_url`

def tables_url(self, namespace: String) -> String

`commit_table_url`

def commit_table_url(self, namespace: String, table: String) -> String

`load_table_url`

def load_table_url(self, namespace: String, table: String) -> String

`headers`

def headers(self) -> List[Header]

`apply_config`

def apply_config(mut self, body: String)

Absorb a GET /v1/config response: overrides, defaults, prefix.

`parse_namespaces`

def parse_namespaces(body: String) -> List[String]

ListNamespacesResponse -> dotted namespace names.

`parse_tables`

def parse_tables(body: String) -> List[String]

ListTablesResponse -> table names.

`RestCatalog`

struct RestCatalog

An Iceberg REST catalog reachable over HTTP(S).

var cat = RestCatalog("https://catalog.example.com")
cat.config.with_token(token)
cat.config.vend_credentials = True
cat.connect()                        # GET /v1/config, absorb the prefix
var t = cat.load_table("db", "orders")

connect() is separate from construction because it does I/O, and a caller that already knows the prefix (or is talking to a server without one) can skip it.

Implements Copyable, Movable

Fields: config: RestCatalogConfig; client: HttpClient; io: FileIO: The FileIO template every loaded table starts from — rebases and storage properties set here survive into each table.

`init`

def __init__(out self, var uri: String)

def __init__(out self, var config: RestCatalogConfig, var io: FileIO)

`connect`

def connect(mut self)

GET /v1/config — absorbs overrides, defaults and prefix.

`list_namespaces`

def list_namespaces(self, parent: String = "") -> List[String]

`list_tables`

def list_tables(self, namespace: String) -> List[String]

`load_table_result`

def load_table_result(self, namespace: String, table: String) -> LoadTableResult

`table_exists`

def table_exists(self, namespace: String, table: String) -> Bool

`load_table`

def load_table(self, namespace: String, table: String) -> Table

Load a table, configuring its FileIO from what the catalog says.

The table's own config becomes storage properties (s3.endpoint, s3.region, …) and every storage-credentials entry is registered under its prefix, so a vended, per-prefix STS credential is used for exactly the objects it was issued for.

`commit_append`

def commit_append(self, namespace: String, table: String, data_files: List[DataFile], retries: Int = Int(4)) -> Table

Commit already-written data files as one append snapshot.

Optimistic concurrency, the way the REST spec defines it: the request carries assert-table-uuid and assert-ref-snapshot-id for main, so a server whose main has moved answers 409 and this reloads, rebuilds the snapshot against the new parent — the data files are still valid, only the manifest list has to be rewritten — and tries again. A 5xx is not retried: the commit may well have been applied, and the caller is told the state is unknown rather than being handed a second attempt at the same append.

Each attempt writes a fresh manifest and manifest list; the ones a refused attempt wrote are orphaned, exactly as in every other implementation, and are nobody's to read.

There is no next-row-id update in the spec's TableUpdate list. A v3 server derives it from the snapshot's first-row-id plus its added-rows, both of which add-snapshot carries.

`commit_change`

def commit_change(self, namespace: String, table: String, kind: Int, filter_dsl: String, batches: List[RecordBatch], mode: String, retries: Int) -> Table

A delete or an overwrite, committed the way an append is.

The CommitTableRequest is identical — one add-snapshot and one set-snapshot-ref behind the same two requirements — because the spec has no per-operation update action: what the snapshot is lives entirely in the manifest list it names. So the retry story is the same too, and one 409 reload re-plans the whole operation against the new parent, because a delete planned against a snapshot that has moved may now be missing rows a concurrent writer added.

`update_schema`

def update_schema(self, namespace: String, table: String, update: UpdateSchema, retries: Int = Int(4)) -> Table

Commit schema changes through the catalog.

The request asserts the table's uuid, its current schema id and its last assigned column id, and sends add-schema + set-current-schema (see iceberg.update_schema). A 409 reloads and applies the same changes to the schema that won; a 5xx is reported as unknown state, as a commit of data is. Changes that leave the schema as it is send nothing.

`delete_where`

def delete_where(self, namespace: String, table: String, filter_dsl: String, mode: String = "", retries: Int = Int(4)) -> Table

DELETE FROM t WHERE <filter>, committed through the catalog.

`overwrite`

def overwrite(self, namespace: String, table: String, batches: List[RecordBatch], filter_dsl: String = "[\22true\22]", retries: Int = Int(4)) -> Table

Delete what the filter matches and add batches, in one snapshot.

`dynamic_partition_overwrite`

def dynamic_partition_overwrite(self, namespace: String, table: String, batches: List[RecordBatch], retries: Int = Int(4)) -> Table

Replace exactly the partitions the new rows land in.

`append`

def append(self, namespace: String, table: String, batches: List[RecordBatch], retries: Int = Int(4)) -> Table

Write batches into the table's location, then commit them.

`create_table`

def create_table(mut self, namespace: String, table: String, schema: Schema, spec: PartitionSpec = PartitionSpec.unpartitioned(Int(0)), var properties: Dict[String, String] = Dict(), format_version: Int = Int(2), location: String = "") -> Table

POST .../tables — a CreateTableRequest.

The format version travels as the format-version property, which is how every REST catalog is told; there is no field for it in the request body.

Functions

`encode_namespace`

def encode_namespace(dotted: String) -> String

Multipart namespaces travel as one path segment joined by 0x1F.

`url_encode`

def url_encode(s: String) -> String

Percent-encode everything outside the unreserved set.

`rest_error`

def rest_error(what: String, resp: Response) -> Error

Turn a REST error response into an Error naming what actually happened.

The spec's ErrorModel carries {"error": {"message", "type", "code"}}; servers that do not send one still leave a status code worth reporting.

`commit_append_body`

def commit_append_body(namespace: String, table: String, current: TableMetadata, result: AppendResult) -> String

The CommitTableRequest an append produces.

Two requirements, which together are exactly what an optimistic append needs: the table must still be the one we read (assert-table-uuid), and main must still point where it did when the snapshot's parent was chosen (assert-ref-snapshot-id, with a null id when the table has no snapshot yet).

Aliases

comptime ACCESS_DELEGATION_HEADER = String("X-Iceberg-Access-Delegation")

comptime COMMIT_DELETE = 0

comptime COMMIT_OVERWRITE = 1

comptime COMMIT_DYNAMIC = 2

comptime VENDED_CREDENTIALS = String("vended-credentials")

comptime REMOTE_SIGNING = String("remote-signing")