Add POST /runs/import for runs executed elsewhere

A history recorded before the move (MLflow experiments) had nowhere to
go, since runs were only made by running a flow. Each entry is stored as
it ended: parameters completed and checked like a submit, cause
"import", empty commit and code digest, metrics and artifacts written
as given, nothing queued or announced. Run.external_id (new migration)
makes a second import a no-op and is an export column so a row joins
back. The request is all or nothing and capped at 200 entries;
Client.import_runs pages a longer history.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QgG6TJw5qBZzaLghQSRVrq
This commit is contained in:
2026-09-16 17:01:07 +02:00
co-authored by Claude Opus 5
parent eeaa46d4ae
commit d9630f66f5
12 changed files with 848 additions and 32 deletions
+11
View File
@@ -103,6 +103,7 @@ published to. Flows own the namespace; everything else is a client of it.
|---|---|---|
| `POST` | `/runs/flows/{name}` | queue one run — `{"params": {...}, "seed": 7, "draft": false, "no_cache": false}`. `"cause"` says where it came from — `api` (the default), `cli` or `sdk` |
| `POST` | `/runs/flows/{name}/sweep` | queue many, sharing a `group_id` |
| `POST` | `/runs/import` | record runs executed somewhere else, up to 200 per request. Answers `[{"external_id", "id", "created"}]` |
| `GET` | `/runs` | the queryable history: `?flow=`, `?status=`, `?group=`, `?digest=`, `?since=`, `?before=`, `?limit=`, `?offset=` |
| `DELETE` | `/runs?flow=` | every run of one flow, and everything hanging off them. `flow` is required. Refused with 409 while one of them is still going |
| `GET` | `/runs/overview` | one row per flow that has runs, with how many are running or queued |
@@ -120,6 +121,16 @@ Submitting answers immediately with a `queued` run. Wrong parameters (an
undeclared name, a value of the wrong type) come back as a 422 naming the
problem, before anything executes.
`/runs/import` takes `{"runs": [...]}`. Each entry carries `flow`,
`external_id`, `params`, `seed`, `status` (`ok`, `error` or `cancelled`),
`created_at`, `started_at`, `finished_at`, `result`, `metrics` (`name`,
`step`, `ts`, `value`), `artifacts` (`name`, `filename`, `media_type`,
`content_base64`), `labels` and `group_id`. The parameters are completed and
checked as a submit's are, the run is stored with the cause `import` and an
empty commit and code digest, and nothing is queued. The request is checked
whole before anything is written, and an `external_id` already imported
answers with that run and `"created": false`.
`?digest=` filters by the hash of the parameters, which is how you find "every
run that used exactly this configuration".
+33
View File
@@ -383,6 +383,39 @@ path (`metric.final_metrics.train_loss`) and `metrics="final_metrics.train_loss"
selects one. Inputs work the same way, and are compared leaf by leaf, so two
configurations differing in one field give that field as the axis.
### Bringing in runs made elsewhere
Runs are made by running a flow, so a history recorded before the move (an
MLflow experiment, say) has to be imported instead. Each entry names the flow,
the id the run had where it ran, and what it ended with:
```python
Client().import_runs([
{
"flow": "train",
"external_id": "mlflow:study-1/3f2a…",
"params": {"lr": 0.01},
"seed": 1001,
"status": "ok",
"created_at": "2026-03-01T10:00:00Z",
"started_at": "2026-03-01T10:00:01Z",
"finished_at": "2026-03-01T10:40:12Z",
"result": {"final_metrics": {"train_loss": 0.08}},
"metrics": [{"name": "train.train_loss", "step": 0, "ts": 1772359201.0, "value": 2.3}],
"artifacts": [{"name": "weights", "filename": "model.txt",
"media_type": "text/plain", "content_base64": "…"}],
"labels": ["imported"],
"group_id": "mlflow:study-1",
},
])
```
The parameters are completed with the flow's declared values and checked the
way a submit checks them. The run is stored with the cause `import` and no
commit or code digest, since this engine never saw the code it ran. Importing
the same `external_id` again adds nothing, and `fluksio export runs` carries
it as a column, so an exported row joins back to where it came from.
### A dashboard, read against runs
A run records values under the same names a dashboard binds to (a run of