# objectstore.ranges

> A module of objectstore's API.

`objectstore.ranges` — byte ranges, and the plan that turns many into few.

A Parquet scan does not read a file, it reads a list of spans: the footer,
then one span per column chunk of each row group it kept. Over local storage
that is a list of seeks and costs nothing. Over HTTP it is a list of *requests*,
and even with connection reuse each one still costs a round trip — so the
useful optimisation is not making requests faster, it is making fewer of them.

Column chunks of the same row group are laid out consecutively, which means the
spans a scan asks for are usually adjacent or separated by the columns it
skipped. Fetching across a small hole and throwing the hole away is cheaper
than a second round trip: this module works out where those holes are.

The bound is `max_gap` (1 MiB by default), applied per join, so the bytes read
and discarded are at most `max_gap` times the number of joins — never more than
the caller asked for plus that.

This module deliberately knows nothing about transports; `fileio` and `httpio`
execute the plan.

## Structs

### `ByteRange`

```mojo
struct ByteRange
```

`length` bytes starting at `offset`.

A negative `offset` means what it means to `read_range` — a suffix range,
"the last `length` bytes" — and is never coalesced, because where it lands
is not known until the length is.

Implements: `Copyable`, `Movable`, `Writable`

Fields:

- `offset: Int`
- `length: Int`

#### `end`

```mojo
def end(self) -> Int
```

#### `write_to`

```mojo
def write_to(self, mut writer: T)
```

### `CoalescedPlan`

```mojo
struct CoalescedPlan
```

How to answer `n` ranges with `len(fetches)` requests.

For each input range `i`: `group[i]` is the fetch it comes from, or -1 if
it has to be read on its own (a suffix or empty range), and `group_offset[i]`
is where inside that fetch its bytes begin.

Implements: `Copyable`, `Movable`

Fields:

- `fetches: List[ByteRange]`
- `group: List[Int]`
- `group_offset: List[Int]`

#### `__init__`

```mojo
def __init__(out self)
```

#### `requests_saved`

```mojo
def requests_saved(self, asked: Int) -> Int
```

## Traits

### `RangeReader`

```mojo
undefined
```

The one method coalescing needs. `InputFile` is a superset.

Implements: `Copyable`, `Movable`

#### `read_range`

```mojo
def read_range(self, offset: Int, length: Int) -> List[UInt8]
```

## Functions

### `plan_ranges`

```mojo
def plan_ranges(ranges: List[ByteRange], max_gap: Int = Int(1048576)) -> CoalescedPlan
```

Groups ranges that are adjacent, overlapping, or `max_gap` apart.

The input order is preserved in the result — a caller that asked for the
footer last still gets the footer last — and ranges are sorted internally
only to find the groups.

### `read_ranges_coalesced`

```mojo
def read_ranges_coalesced[T: RangeReader](reader: T, ranges: List[ByteRange], max_gap: Int = Int(1048576)) -> List[List[UInt8]]
```

Fetches the plan's groups one by one and slices the caller's ranges back out. Backends that can fetch groups at once do that instead and call `assemble`.

### `assemble`

```mojo
def assemble[T: RangeReader](reader: T, plan: CoalescedPlan, ranges: List[ByteRange], var buffers: List[List[UInt8]]) -> List[List[UInt8]]
```

The caller's ranges, in order, cut from `buffers` (one per fetch of `plan`). A range a fetch answers exactly gets that buffer, uncopied; a suffix or empty range, which no fetch covers, is read on its own.

## Aliases

### `DEFAULT_COALESCE_GAP`

```mojo
comptime DEFAULT_COALESCE_GAP = 1048576
```

Holes up to this size are read and discarded rather than skipped. A MiB is roughly where a loopback round trip stops being cheaper than the bytes; over a real network the crossover is much higher, so this is the conservative end.