Let a sweep run more than four at a time, and name the run a failure was in
Docs / docs (push) Successful in 29s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m33s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m3s
pre-commit / pre-commit (push) Failing after 3m9s
Test Backend / test-backend (push) Successful in 2m46s
Compose Smoke Test / test-compose (push) Successful in 39s
Playwright Tests / merge-reports (push) Successful in 1m47s
Docs / docs (push) Successful in 29s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m33s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m3s
pre-commit / pre-commit (push) Failing after 3m9s
Test Backend / test-backend (push) Successful in 2m46s
Compose Smoke Test / test-compose (push) Successful in 39s
Playwright Tests / merge-reports (push) Successful in 1m47s
Concurrent runs sat at 4 whatever FLOW_MAX_CASCADES said: that setting bounds cascades, and the run drivers read a hardcoded MAX_PARALLEL nobody could reach. FLOW_MAX_RUNS is the knob they read now, --max-runs/--max-cascades/--max-workers are the same three as flags on serve, and the engine says which numbers it started with — which is the only way to tell that a settings file was read. Events keep the run they happened in. The payload always carried it and the persist path dropped it, so reading one run's failures meant filtering the engine-wide list; a batch run's id reaches those events now too, since a run has no journaled item to name itself by. Also: a provisioner's 0 means "no deadline" rather than "cancel on the next reconcile", and a command that reaches no engine says how to start one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015sbYeYaVgYQqm1sbx7wPdL
This commit is contained in:
@@ -68,9 +68,12 @@ LEASE_INTERVAL_S = 20.0
|
||||
LEASE_STALE_S = 90.0
|
||||
#: How often stale leases are looked for.
|
||||
SWEEP_INTERVAL_S = 30.0
|
||||
#: Runs driven at once. Node bodies are bounded by the worker pool anyway;
|
||||
#: this only bounds how many graphs are in flight.
|
||||
#: Runs driven at once, unless `FLOW_MAX_RUNS` says otherwise. Node bodies are
|
||||
#: bounded by the worker pool anyway; this only bounds how many graphs are in
|
||||
#: flight.
|
||||
MAX_PARALLEL = 4
|
||||
#: Runs claimed per poll, at least. Raising the pool raises this with it, so a
|
||||
#: queue of ready runs fills the drivers in one pass rather than four a second.
|
||||
CLAIM_COUNT = 4
|
||||
CLAIM_BLOCK_MS = 1000
|
||||
ERROR_CAP = 2000
|
||||
@@ -706,7 +709,10 @@ class RunService:
|
||||
# the isolation it wants, minus surviving the process.
|
||||
self._state_factory = state_factory or (lambda _ns: MemoryState())
|
||||
self.engine_name = f"{socket.gethostname()}-{os.getpid()}"[:64]
|
||||
self._pool = ThreadPoolExecutor(max_workers=parallel, thread_name_prefix="run")
|
||||
self.parallel = max(1, parallel)
|
||||
self._pool = ThreadPoolExecutor(
|
||||
max_workers=self.parallel, thread_name_prefix="run"
|
||||
)
|
||||
self._stop = threading.Event()
|
||||
self._consumer: threading.Thread | None = None
|
||||
self._keeper: threading.Thread | None = None
|
||||
@@ -882,7 +888,9 @@ class RunService:
|
||||
# Runs put back to wait for a worker come due here. The claim
|
||||
# below blocks for a second, so this is about once a second.
|
||||
self.queue.move_due(time.time())
|
||||
items = self.queue.claim(CLAIM_COUNT, CLAIM_BLOCK_MS)
|
||||
items = self.queue.claim(
|
||||
max(CLAIM_COUNT, self.parallel), CLAIM_BLOCK_MS
|
||||
)
|
||||
failures = 0
|
||||
except Exception as exc:
|
||||
failures += 1
|
||||
|
||||
Reference in New Issue
Block a user