Docs
The UI
orca ui serves pipelines, runs and datasets as web pages, from the same
orca binary, on 127.0.0.1:8270 unless
ORCA_UI_ADDR says otherwise. It holds no state: it reads the tables the
scheduler and the runs write, connecting to Postgres per request, so it can run beside
the scheduler or anywhere else that reaches the database. Nothing is fetched from
elsewhere, so it works offline.
Pages
ORCA_DATABASE_URL=postgresql:///orca ./build/orca ui
curl -H 'Accept: application/json' http://127.0.0.1:8270/runs/7| Path | What it shows |
|---|---|
/ | pipelines, with their schedules, next runs and last runs, and runs per day over the last 30 days |
/pipelines/<name> | a pipeline's DAG, partitions, schedule, next runs, versions and runs, its code, and the forms to start a run, backfill a range, cancel its runs and pause it; a keyed one's keys, done, live and failed |
/pipelines/<name>/keys/<key> | one key of a keyed pipeline: its steps in every run it was in |
/runs | runs, newest first; ?pipeline=, ?status=, ?trigger= |
/runs/<id> | a run's DAG in its steps' colours, its steps and partitions, attempts, errors, reads and commits, its source and code, its timeline, machines and instances, and resume, rerun from a step and cancel |
/runs/<id>/attempts/<attempt> | an attempt's log, its last MiB, with its error |
/runs/<id>/timeline.json | the run's Dista timeline, for Perfetto |
/runs/<id>/driver.log | the run's driver output, as the scheduler kept it |
/runs/<id>/cluster/<role>-<instance>.log | a head's or worker's output, from an EC2 run |
/datasets | every dataset with its writers and readers, and the graph of pipelines and datasets |
/datasets/<name> | a dataset's place in that graph, its snapshots, its rebuilds and the form to start one |
/datasets/<name>/snapshots/<id> | a snapshot's lineage, back and forward, as a graph and as tables |
/rebuilds/<id> | a rebuild's pipelines by level, with their runs |
/login, /logout | a session for the actions, as a cookie |
Every path also answers JSON, to Accept: application/json or
?format=json, built from the same rows, so a script and a person see the
same thing. While a run or an attempt is going on, its page refreshes its DAG, steps
or log every two seconds, and stops once it has ended.
Actions
| Action | What it does |
|---|---|
| Start | a queued run over the partitions entered, trigger ui |
| Backfill | what orca backfill does: a queued run for each partition without one, and with and what is ticked, the done ones too |
| Resume | a failed, cancelled or stalled run queued again, to resume at what hasn't succeeded; a run whose code has changed asks for confirmation first |
| Rerun from a step | a new run, trigger rerun, with the steps before the chosen one copied from the old run, so only it and what follows run |
| Cancel | a queued or failed run cancelled at once; a running one stops before its next attempt, and the attempt already going finishes |
| Pause, Unpause | on a pipeline's page: what orca pause and orca unpause do |
| Cancel queued runs | on a pipeline's page: what orca cancel does, over @all or the partitions entered |
| Rebuild | on a dataset's page: what orca rebuild does, over the partitions entered |
The UI never starts a pipeline binary. An action writes rows in Postgres, and the
scheduler starts the queued runs on its next tick, within max_runs, so
the UI does nothing without a scheduler running. It binds to localhost unless told
otherwise, and refuses a POST whose Origin, or
Referer without one, isn't its own, so another site's page can't post to
it through a browser.
Login
Anyone who can reach the UI can read it. Its actions need a login once there's a
user, or with ORCA_UI_LOGIN=required before there is one:
./build/orca user add alice # reads the password from standard input
./build/orca user list
./build/orca user remove alice # and ends their sessions
Passwords are kept as bcrypt hashes made by Postgres. A login is a session cookie,
HttpOnly and SameSite=Strict, and Secure with
ORCA_UI_SECURE_COOKIES=1 behind HTTPS. Every action is recorded with
its user and shown on its run's page, and so is every failed login: after 5 for one
name, or 20 from one address, in 15 minutes, /login answers 429 without
checking the password. Behind a proxy, ORCA_UI_TRUST_PROXY=1 takes the
address from X-Forwarded-For.
A run's page
A run with Dista tasks draws its timeline: one lane per worker, each task a bar
coloured by its function. With ORCA_UI_URL set to the UI's public
address, an Open in Perfetto link opens the full trace there. Its
machine metrics are drawn under its steps. A
run on EC2 lists its instances, with the number of workers over the run, why each
instance ended, and each one's log. The step names, the pipeline and its Dista tasks
link to their code (versions and code
links), in a new tab, and the headings link to these docs
(ORCA_DOCS_URL changes where).