# parquet.assemble

> A module of parquet's API.

Dremel record assembly: definition and repetition levels back into Arrow.

Every leaf of a Parquet file carries, for each of its value slots, the
definition level (how deep the path to the value is actually present) and the
repetition level (at which level of nesting the value repeats). Those two
numbers are enough to rebuild the whole nested structure, and that is what
`build_field` does — one Arrow array per Arrow field, recursively.

The key idea is a *presence threshold*. A field is asked to build only the
slots whose definition level is at least `require_def`, which its parent sets:

* the root asks for `require_def = 0`, so every slot is a row;
* a `LIST` or `MAP` asks its child for `require_def = elem_def_level`, the
  definition level at which the repeated group actually has an entry — so the
  child array comes back already containing exactly the list's elements, with
  the empty and null containers dropped;
* a `STRUCT` passes its own threshold straight through, because a struct does
  not change granularity.

Levels come from the field's first descendant leaf. Every leaf under a field
agrees about the levels at and above that field, so any one of them will do —
which is what makes it possible to build a struct's children independently and
still have them line up.

## Structs

### `LeafSlice`

```mojo
struct LeafSlice
```

The slot range of one leaf that a batch covers, and where its values start in the chunk's value buffer.

Implements: `Copyable`, `Defaultable`, `Movable`

Fields:

- `s0: Int`
- `s1: Int`
- `v0: Int`

#### `__init__`

```mojo
def __init__(out self)
```

## Functions

### `first_leaf`

```mojo
def first_leaf(s: ParquetSchema, fi: Int) -> Int
```

The first descendant leaf of Arrow field `fi` — its level source.

### `first_included_leaf`

```mojo
def first_included_leaf(s: ParquetSchema, fi: Int, include: List[Bool]) -> Int
```

`first_leaf`, restricted to the fields `include` selects.

A pruned read skips whole sub-trees, so the level source of a struct can no
longer be its first child: it has to be the first child actually being
built. Every leaf under a field agrees about the levels at and above that
field, so any included one will do.

### `build_field`

```mojo
def build_field(s: ParquetSchema, fi: Int, require_def: Int, chunks: List[ColumnData], slices: List[LeafSlice], mut arena: ArrayArena, include: List[Bool]) -> Int
```

Build the Arrow array for field `fi`; return its index in `arena`.

`include`, when it is not empty, is one flag per entry of `s.fields`: a
struct builds only the children it selects, so a projection that wants
`a.b` and nothing else decodes exactly the leaves under `a.b`. A list and
a map always build their element (and a map its key), because their
offsets mean nothing without one.