`iceberg.maintain`

Expiring snapshots, and deleting the files that nothing points at any more.

Expiry is two decisions, in this order:

Which snapshots go. Never the current one, never one a branch or tag names, and never so many that fewer than keep_last are left. Of what remains, the ones older than older_than_ms.
Which files go with them. A file is deleted only when no retained snapshot still lists it live. That is stronger than "it was in an expired snapshot": a data file added five snapshots ago is in every snapshot since, and expiring the oldest of them must not touch it. So the retained snapshots are walked first and every manifest list, manifest and live entry they name is put in a keep set; only then is the expired snapshots' own inventory subtracted from it.

A DELETED entry does not protect a file — that is the entry that says it is gone — but an older retained snapshot that still lists it ADDED or EXISTING does, which is why the keep set is a union over all of them.

The new metadata is written before any file is deleted. A crash in between leaves orphans, which cost storage; the other order leaves a snapshot pointing at files that are not there, which costs the table. dry_run stops after the decision and returns the list, which is the only safe way to find out what an expiry would do.

Metadata files (*.metadata.json) are left alone. Java deletes them under write.metadata.delete-after-commit.enabled with its own retention count, and that is a separate policy from snapshot expiry: the metadata log is how a reader reaches a snapshot by timestamp in the first place.

Structs

`ExpireResult`

struct ExpireResult

What an expiry decided, and what it did about it.

Implements Copyable, Defaultable, Movable

Fields: expired: List[Int64]: Snapshot ids removed from the metadata.; retained: List[Int64]; deleted_files: List[String]: Data and delete files nothing retained points at any more.; deleted_manifests: List[String]: Manifests and manifest lists, likewise.; dry_run: Bool

`init`

def __init__(out self)

`total_deleted`

def total_deleted(self) -> Int

Functions

`choose_expired`

def choose_expired(metadata: TableMetadata, older_than_ms: Int64, keep_last: Int) -> List[Int64]

The snapshot ids an expiry would remove, oldest first.

The current snapshot and every snapshot a ref names are never candidates, and keep_last is a floor on how many survive in total.

`expire_snapshots`

def expire_snapshots(io: FileIO, metadata: TableMetadata, older_than_ms: Int64 = Int64(-1), keep_last: Int = Int(1), dry_run: Bool = False) -> Tuple[TableMetadata, ExpireResult]

Decide the expiry and, unless dry_run, carry out the deletions.

Returns the pruned metadata and a report. The caller commits the metadata — and must do so before the files go, which is why deletion is the last thing this does rather than the first.

`delete_expired_files`

def delete_expired_files(io: FileIO, result: ExpireResult) -> Int

Remove what an expiry decided is unreachable. Content first.

A file that is already gone is not an error: an expiry that was interrupted, or two of them racing, both end at the same place.