Fluksio/app
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

History is kept by flow id: runs, cascades, rollups and events file the id in their flow column and nodes as <flow id>.<node>, the API answers flow plus flow_name and shows and takes nodes as <key>.<node>; events carry flow_id from the moment they are published; run queue items, the stage cache and cached curves keep ids and keys apart; endpoints, the brain graph, device bindings and search name the id to open

Browse Source
This commit is contained in:
stroblme
2026-11-02 19:01:19 +01:00
parent e6027af8fe
commit d4a1b1db47
34 changed files with 1374 additions and 329 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/code/api.md
+33 -6
View File
@@ -86,6 +86,12 @@ Deleting a flow keeps its run history, which is shown under the key the flow
last had. Delete runs separately. The flow's values and queued work go with
it, so its key is free for a new flow, which starts with an empty history.
Answers that lead to a flow or dashboard carry its id next to the key. An
endpoint in `GET /flows/{id}` has `owner_id`, a neuron in `GET /flows/graph`
lists `flow_ids` in the order of its `flows`, a device binding has `flow_id`
and a search entry has `id`. Open things by these ids, not by the part of a
`<key>.<node>` before the dot.
### Node source
```sh
@@ -106,7 +112,8 @@ Ordinary source edits stay in the draft until publishing. Shared library
source changes affect its users immediately, and the audit trail records them
like a publish.
`POST /flows/{id}/nodes/{node}/share` promotes a source to the library;
`/unshare` makes it private again. `GET /flows/library` lists shared sources.
`/unshare` makes it private again. `GET /flows/library` lists shared sources
and the `<key>.<node>` of every node using each.
`DELETE /flows/library/{name}` retires one, and answers 409 while nodes still
use it. With `?detach=true` each of those nodes keeps a copy of the code, in
the published flow and the draft, and the shared source is deleted.
@@ -149,8 +156,8 @@ values. You can publish only to declared messages, just as dashboard controls do
| Method | Path | Action |
|---|---|---|
| `POST` | `/runs/flows/{name}` | queue a run with `{"params": {...}, "seed": 7}` |
| `POST` | `/runs/flows/{name}/sweep` | queue runs with a shared group |
| `POST` | `/runs/flows/{flow id}` | queue a run with `{"params": {...}, "seed": 7}` |
| `POST` | `/runs/flows/{flow id}/sweep` | queue runs with a shared group |
| `POST` | `/runs/import` | import up to 200 completed runs |
| `GET` | `/runs` | query history |
| `GET` | `/runs/overview` | summarize runs per flow |
@@ -158,7 +165,7 @@ values. You can publish only to declared messages, just as dashboard controls do
| `GET` | `/runs/{id}/nodes/{node}/outputs` | read what one node returned |
| `POST` | `/runs/{id}/cancel`, `/runs/{id}/retry` | cancel, or create a retry |
| `DELETE` | `/runs/{id}` | delete a finished run and its records |
| `DELETE` | `/runs?flow={name}` | delete a flow's runs; refused while any are active. `history=true` also clears its cascades from the history |
| `DELETE` | `/runs?flow={flow id}` | delete a flow's runs; refused while any are active. `history=true` also clears its cascades from the history |
| `GET` | `/runs/{id}/metrics?name=&stride=` | read metric series |
| `GET` | `/runs/metrics/names` | list metric names across a selection |
| `GET` | `/runs/series/compare?ids=a,b,c&metric=&x=` | compare curves by step, time, or another metric |
@@ -172,19 +179,29 @@ History filters include `flow`, `status`, `group`, `digest`, `since`,
`before`, `limit`, and `offset`. Rows arrive newest first; use the last
row's `created_at` as `before` for stable pagination.
History is kept by flow id, so it stays with a flow whose key changes and
stays apart from a later flow that takes a deleted one's key. In a run, a
cascade, an engine event and a metrics rollup, `flow` is the flow's id and
`flow_name` its key, or the key it last had once it is deleted. A `flow`
filter, here and under `/observability`, takes the id and only the id. Nodes
are shown and taken as `<key>.<node>`, and metric and artifact names keep the
key too (`train.loss`).
A node's outputs come from the stage cache, so only a node that succeeded
and may be cached has them, up to 256 kB of JSON. Any other node answers
`404`.
Exports also accept `ids` and `until`, with `format=csv` or `jsonl`.
The metrics export is a long table; the runs export is one row per run.
The metrics export is a long table; the runs export is one row per run, with
the flow's id in `flow` and its key in `flow_name`.
Use `params` and `metrics` to choose columns, including dotted paths into
record outputs. An export covers at most the newest 10 000 runs. When a
selection matches more, the response carries `X-Truncated` with that cap and
`X-Total-Count` with the number of runs the selection matched.
Import takes `{"runs": [...]}`, records completed results without executing
code, and deduplicates by `external_id`. See
code, and deduplicates by `external_id`. Each run's `flow` is the flow's id,
and its metric names are qualified with the key. See
[Runs](../concepts/runs.md) for examples, artifacts, caching, and exports.
## Artifacts
@@ -281,6 +298,12 @@ Connector state keys can also be listed and cleared by a superuser; see
Changing flavors requires a superuser. See [Remote workers](workers.md).
The history under `/observability` is kept by flow id as described under
[Runs](#runs): `flow` is the id, `flow_name` the key, and `node` a
`<key>.<node>`, which is also how `timeseries` takes its `node` filter. Dead
letters are work the engine gave up on and name the flow and node by key, as
the engine runs them.
## Live events
Connect to `ws://127.0.0.1:8000/api/v1/flows/ws` with an
@@ -289,6 +312,10 @@ Connect to `ws://127.0.0.1:8000/api/v1/flows/ws` with an
Browsers first call `POST /flows/ws-ticket`, then connect with
`?ticket=<ticket>`. Tickets are single-use and expire after 30 seconds.
The socket sends a snapshot followed by node, value, health, and run events.
An event about a flow names it by key in `flow`, the way its nodes and
messages are named, and by id in `flow_id`, which is empty for the engine as
a whole. `dashboard_changed` names the dashboard by id, or nothing when a
device's own wiring moved.
A client asks for the bytes of the media it shows with
`{"type": "media", "names": [...]}`, which replaces what it asked for before,
docs/concepts/runs.md
+13 -6
View File
@@ -42,7 +42,9 @@ curl -X POST "$FLUKSIO/runs/flows/train" \
```
Here `$FLUKSIO` is the API base URL ending in `/api/v1`, and `$TOKEN`
is an access token. See [The HTTP API](../code/api.md) for authentication.
is an access token. The path takes the flow's id, here a flow whose id is
its key, as for one made before ids existed. See
[The HTTP API](../code/api.md) for authentication.
Submission returns immediately with status `queued`. Read
`GET /api/v1/runs/{id}` for status and results. Unknown parameters or wrong
@@ -68,7 +70,7 @@ These four runs share a `group_id` and execute concurrently within the
engine's limits. Each has its own state, so one run cannot overwrite another's
messages. Vary seeds with `--param seed=1,2,3` if the flow declares that input.
The API equivalent is `POST /api/v1/runs/flows/{name}/sweep` with a
The API equivalent is `POST /api/v1/runs/flows/{flow id}/sweep` with a
`runs` list of parameter dictionaries and optional seeds. Filter history
with `GET /api/v1/runs?group=<group-id>`.
@@ -210,7 +212,11 @@ fluksio export runs --flow train --status ok -o results.csv
Both include run IDs. Result rows also include input values, final numeric
outputs, status, duration and code identifiers. Nested values use dotted paths,
such as `metric.final_metrics.train_loss`.
such as `metric.final_metrics.train_loss`. A row names the flow by its id in
`flow` and by its key in `flow_name`. History stays with the id, so a flow
keeps its runs whatever its key is later, and a new flow that takes the key
of a deleted one starts without them. Metric names keep the key, as in
`train.loss`.
Python returns the same rows directly:
@@ -243,8 +249,9 @@ Client().import_runs([{
}])
```
The flow must exist, and parameters are checked against its inputs. Repeated
imports of the same `external_id` reuse the existing record. Imported runs
The flow must exist, and parameters are checked against its inputs. Over the
HTTP API an entry's `flow` is the flow's id. Repeated imports of the same
`external_id` reuse the existing record. Imported runs
have no code commit or digest because the engine did not execute them.
Entries may also carry metric points and base64-encoded artifacts; see the
API schema for their fields.
@@ -257,7 +264,7 @@ the first run, and controls are disabled.
If the flow has no dashboard, the same menu can create one from its streaming
ports and outputs. The API equivalent is
`POST /api/v1/dashboards/from-flow/{name}`. The generated dashboard is
`POST /api/v1/dashboards/from-flow/{flow id}`. The generated dashboard is
published and can be edited like any other dashboard.
To keep a chart showing recent experiments, set its source to **Runs** and