Let a sweep run more than four at a time, and name the run a failure was in
Docs / docs (push) Successful in 29s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m33s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m3s
pre-commit / pre-commit (push) Failing after 3m9s
Test Backend / test-backend (push) Successful in 2m46s
Compose Smoke Test / test-compose (push) Successful in 39s
Playwright Tests / merge-reports (push) Successful in 1m47s
Docs / docs (push) Successful in 29s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m33s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m3s
pre-commit / pre-commit (push) Failing after 3m9s
Test Backend / test-backend (push) Successful in 2m46s
Compose Smoke Test / test-compose (push) Successful in 39s
Playwright Tests / merge-reports (push) Successful in 1m47s
Concurrent runs sat at 4 whatever FLOW_MAX_CASCADES said: that setting bounds cascades, and the run drivers read a hardcoded MAX_PARALLEL nobody could reach. FLOW_MAX_RUNS is the knob they read now, --max-runs/--max-cascades/--max-workers are the same three as flags on serve, and the engine says which numbers it started with — which is the only way to tell that a settings file was read. Events keep the run they happened in. The payload always carried it and the persist path dropped it, so reading one run's failures meant filtering the engine-wide list; a batch run's id reaches those events now too, since a run has no journaled item to name itself by. Also: a provisioner's 0 means "no deadline" rather than "cancel on the next reconcile", and a command that reaches no engine says how to start one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015sbYeYaVgYQqm1sbx7wPdL
This commit is contained in:
+1
-1
@@ -176,7 +176,7 @@ read one back.
|
||||
| `/observability/timeseries` | executions and failures over a window |
|
||||
| `/observability/flows` | per-flow rollups with a 60-slice trend |
|
||||
| `/observability/runs` | recent cascades, with `?flow=`, `?since=`, `?until=` |
|
||||
| `/observability/events?kind=failure\|audit` | what went wrong, or who changed what |
|
||||
| `/observability/events?kind=failure\|audit` | what went wrong, or who changed what; narrows by `?flow=`, `?run=`, `?since=`, `?until=` |
|
||||
| `/observability/dead-letter` | work the engine gave up on |
|
||||
|
||||
`GET /utils/health/` is the deep health check the container probe uses: it
|
||||
|
||||
@@ -204,6 +204,10 @@ never attaches within `provision_timeout_s` is `scancel`led, as is anything
|
||||
outstanding when the engine stops. `--max-idle` is what ends the job at the
|
||||
other end, so an allocation goes back rather than idling to its walltime.
|
||||
|
||||
Both take `0` for "no limit": `provision_timeout_s: 0` waits for as long as the
|
||||
queue does, which is what a cluster that queues overnight needs, and
|
||||
`max_idle_s: 0` keeps the machine for the job's whole walltime.
|
||||
|
||||
`GET /workers/resources` reports what is outstanding and what last went wrong.
|
||||
|
||||
!!! note "It needs a route out"
|
||||
|
||||
@@ -125,6 +125,7 @@ warning into a refusal to start.
|
||||
|---|---|---|
|
||||
| `FLOW_MAX_WORKERS` | `4` | node-code subprocesses run in parallel |
|
||||
| `FLOW_MAX_CASCADES` | `4` | cascades in flight at once; throughput is this over the mean cascade time, so raise it where nodes wait on a network rather than a CPU |
|
||||
| `FLOW_MAX_RUNS` | `4` | batch runs driven at once. A different limit from the one above: a run drives a whole graph, and its nodes are bounded by `FLOW_MAX_WORKERS`. This is what a sweep queues behind |
|
||||
| `FLOW_NODE_TIMEOUT` | `0` | seconds a node may be silent, unless it sets its own; 0 is no limit |
|
||||
| `FLOW_CPUS` | `0` | cores nodes that declare `resources` may be given; 0 works it out as every core but two, which are what keeps the engine answering while the machine is busy |
|
||||
| `FLOW_GPUS` | `0` | GPUs on this machine, each held by one node at a time. Not detected — say how many there are |
|
||||
@@ -132,6 +133,10 @@ warning into a refusal to start.
|
||||
| `ARTIFACT_GC_INTERVAL_S` | `3600` | how often artifact bytes nothing refers to are swept away; 0 never sweeps |
|
||||
| `ARTIFACT_GC_GRACE_S` | `3600` | how long a freshly written artifact is spared, whatever refers to it |
|
||||
|
||||
The three concurrency limits are also flags on `fluksio serve` — `--max-workers`,
|
||||
`--max-cascades`, `--max-runs` — which outrank the file, and the engine says
|
||||
which numbers it started with in its first lines.
|
||||
|
||||
An artifact is referred to by a run that recorded it or by a message currently
|
||||
holding it; anything else is what a camera published four hours ago, and the
|
||||
sweep is what keeps a flow streaming media from filling the disk. It stands
|
||||
|
||||
Reference in New Issue
Block a user