# orca.lineage

> A module of Orca's API.

Lineage: which snapshots a dataset's snapshot was made from, and what was made from it, from the reads and commits every attempt records.

```
orca lineage lake.report            the latest snapshot's, back and forward
orca lineage lake.report 4417...    one snapshot's
```

A snapshot that an orca attempt committed is **made** by that attempt: its
run, pipeline, step, partition and code, and the snapshots it read, its
**inputs**. The walks go over those edges in SQL (recursive CTEs):

- **`back`** from a snapshot: the attempt that made it, then the attempts
  that made each of its inputs, and so on, to inputs no orca attempt made
  (sources: a table written by something else, or a read of a dataset
  before it had any snapshot, which isn't listed).
- **`forward`** from a snapshot: every snapshot made by an attempt that read
  it, then what was made from those, and so on.

Both stop at `depth` steps (default 20), so a deep graph is cut rather than
slow. A snapshot recorded by two attempts (a retry that recognised its
earlier attempt's commit) is listed once, as the nearer and earlier. Only
datasets take part: a table outside the warehouse has no name to follow.
The same edges are in the snapshots' own summaries, as `orca.reads`.

## Structs

### `SnapshotRef`

```mojo
struct SnapshotRef
```

Implements: `Copyable`, `Equatable`, `Movable`, `Writable`

Fields:

- `dataset: String`
- `snapshot: Int64`

#### `__eq__`

```mojo
def __eq__(self, other: Self) -> Bool
```

#### `write_to`

```mojo
def write_to(self, mut writer: T)
```

### `Made`

```mojo
struct Made
```

A snapshot, the attempt that made it, and what that attempt read.

Implements: `Copyable`, `Movable`

Fields:

- `at: SnapshotRef`
- `depth: Int`: Steps from the snapshot the walk started at: 0 for it, 1 for its inputs' (back) or for what was made from it (forward).
- `pipeline: String`
- `run: Int`
- `step: String`
- `partition: String`
- `attempt: Int`
- `code: String`
- `ended: String`
- `inputs: List[SnapshotRef]`

### `Dataset`

```mojo
struct Dataset
```

Implements: `Copyable`, `Movable`

Fields:

- `name: String`
- `writers: List[String]`: The pipelines whose steps declare it in `writes=`.
- `readers: List[String]`
- `commits: Int`
- `latest: Int64`: The newest snapshot an orca attempt committed, or -1.
- `latest_run: Int`

### `Flow`

```mojo
struct Flow
```

An edge of the graph of pipelines and datasets: a pipeline reads a dataset, or writes one.

Implements: `Copyable`, `Movable`

Fields:

- `pipeline: String`
- `dataset: String`
- `writes: Bool`

### `Commit`

```mojo
struct Commit
```

A snapshot of a dataset an orca attempt committed.

Implements: `Copyable`, `Movable`

Fields:

- `snapshot: Int64`
- `pipeline: String`
- `run: Int`
- `step: String`
- `partition: String`
- `ended: String`

## Functions

### `back`

```mojo
def back(mut db: State, start: SnapshotRef) -> List[Made]
```

The snapshot, if an orca attempt made it, and everything it was made from: one `Made` per snapshot an orca attempt made.

### `forward`

```mojo
def forward(mut db: State, start: SnapshotRef) -> List[Made]
```

Every snapshot made from `start`, directly or not.

### `latest`

```mojo
def latest(mut db: State, dataset: String) -> Int64
```

The dataset's newest snapshot orca knows of, committed or read, or -1 if it knows of none.

### `datasets`

```mojo
def datasets(mut db: State, only: String = "") -> List[Dataset]
```

Every dataset a pipeline declares or an attempt touched, or the one named, by name.

### `flows`

```mojo
def flows(mut db: State) -> List[Flow]
```

Every pipeline's declared reads and writes.

### `commits`

```mojo
def commits(mut db: State, dataset: String, limit: Int = Int(50)) -> List[Commit]
```

The dataset's snapshots orca committed, newest first.

## Aliases

### `DEPTH`

```mojo
comptime DEPTH = 20
```