Let a sweep run more than four at a time, and name the run a failure was in

Concurrent runs sat at 4 whatever FLOW_MAX_CASCADES said: that setting bounds
cascades, and the run drivers read a hardcoded MAX_PARALLEL nobody could reach.
FLOW_MAX_RUNS is the knob they read now, --max-runs/--max-cascades/--max-workers
are the same three as flags on serve, and the engine says which numbers it
started with — which is the only way to tell that a settings file was read.

Events keep the run they happened in. The payload always carried it and the
persist path dropped it, so reading one run's failures meant filtering the
engine-wide list; a batch run's id reaches those events now too, since a run
has no journaled item to name itself by.

Also: a provisioner's 0 means "no deadline" rather than "cancel on the next
reconcile", and a command that reaches no engine says how to start one.
This commit is contained in:
2026-08-27 14:17:51 +02:00
parent 2c081fe3ed
commit 081d499edc
23 changed files with 288 additions and 18 deletions
+10 -3
View File
@@ -237,6 +237,7 @@ class Pipeline:
"observer",
"emission_observer",
"run_cache",
"run_id",
)
def __init__(
@@ -252,6 +253,7 @@ class Pipeline:
observer: Callable[[NodeOutcome], None] | None = None,
emission_observer: Callable[[str, dict[str, Any]], None] | None = None,
run_cache: RunCacheLookup | None = None,
run_id: str = "",
) -> None:
self._nodes = nodes or []
# Stopped flows are stored and survive a restart; paused ones are a
@@ -286,6 +288,11 @@ class Pipeline:
# none: a cascade is about what just happened, not about what a node
# once returned for the same inputs.
self.run_cache = run_cache
# The batch run this pipeline belongs to, if any. A live cascade names
# the journaled item it came from instead, which is what the events
# below carry; a run has no such item, so without this its failures
# would be recorded belonging to nothing.
self.run_id = run_id
# How deep to keep each message's series; a chart asking for more
# than the default puts its message in here. Swapped, never mutated.
self.history_limits: dict[str, int] = {}
@@ -853,7 +860,7 @@ class Pipeline:
"flow": node.flow,
"node": node.id,
"error": error,
"run": entry_id,
"run": entry_id or self.run_id,
"ts": time.time(),
}
)
@@ -916,7 +923,7 @@ class Pipeline:
"node": node.id,
"outputs": len(outputs or {}),
"duration_ms": 0.0,
"run": entry_id,
"run": entry_id or self.run_id,
"ts": time.time(),
}
)
@@ -1007,7 +1014,7 @@ class Pipeline:
# which is a different thing to show than one that emitted.
"outputs": len(result or {}),
"duration_ms": duration_ms,
"run": entry_id,
"run": entry_id or self.run_id,
"ts": time.time(),
}
)