`objectstore.httpio`

objectstore.httpio — a read-only InputFile over http(s)://.

Two things make this worth having beyond "download the file". A HEAD gives the length without a body, and a Range request gives an arbitrary slice — so a Parquet file published over plain HTTP is as lazily readable as one in S3. And a presigned S3 URL is, at the point of use, just an HTTPS URL with a signature in the query string: this is the backend that reads it, with no credentials involved.

Servers are allowed to ignore Range and answer 200 with the whole body. That is handled rather than trusted: a 200 where a 206 was expected is sliced locally, so the caller always gets the bytes it asked for.

Structs

`HttpInputFile`

struct HttpInputFile

Implements Copyable, Movable, RangeReader

Fields: url: String; client: HttpClient; headers: List[Header]: Sent with every request. This is what makes the module reusable for GCS (Authorization: Bearer …) and Azure (x-ms-version) — both reduce to an HTTPS URL plus a fixed set of headers.

`init`

def __init__(out self, var url: String)

def __init__(out self, var url: String, var client: HttpClient)

`location`

def location(self) -> String

`exists`

def exists(self) -> Bool

`length`

def length(self) -> Int

`read_all`

def read_all(self) -> List[UInt8]

`read_range`

def read_range(self, offset: Int, length: Int) -> List[UInt8]

length bytes from offset. A negative offset is a suffix range (bytes=-N), the way HTTP itself spells "the last N bytes".

`read_ranges`

def read_ranges(self, ranges: List[ByteRange]) -> List[List[UInt8]]

One request per group of nearby ranges rather than one per range.

Connection reuse made a request cheap; this makes it rare. A Parquet row group whose column chunks are laid out consecutively becomes a single Range request, and the bytes of the columns the scan skipped are read and thrown away — which is cheaper than the round trip that would have avoided them. The groups are fetched at once.

`HttpOutputFile`

struct HttpOutputFile

A write-only file behind a single PUT.

Only usable where a plain PUT to a URL means "store this object" — a presigned S3 upload URL, GCS's XML endpoint with a bearer token, an Azure blob URL with a SAS token. It is not a general HTTP writer, because HTTP itself has no create-versus-overwrite distinction: create is implemented as "check, then write", which is racy and says so.

Implements Copyable, Movable

Fields: url: String; client: HttpClient; headers: List[Header]

`init`

def __init__(out self, var url: String, var headers: List[Header])

`location`

def location(self) -> String

`exists`

def exists(self) -> Bool

`create`

def create(self, data: Span[UInt8])

`overwrite`

def overwrite(self, data: Span[UInt8])

Functions

`http_delete`

def http_delete(url: String, headers: List[Header], client: HttpClient)