Schedule a node across every machine, not just this one

The engine answered "where does this node run" twice, in two ways that could
not see each other: a device sent it to a worker carrying that label, and
resources were counted against the engine's own cores. Declaring both meant the
second answer won and nothing was counted at all — which the data-science
getting-started page and the worked example both do.

One question now, in flow/placement.py: of every machine attached, which could
grant what this node asked for, and which of those has it free. The books move
onto each machine — one accountant per worker, built from the inventory it
reported — and the waiting moves above them, where one condition variable can
be woken by a release anywhere or by a worker attaching. Locks go one way:
placer, then a machine's books, never back.

So a node asking for a card now finds the box that has one, rather than being
clamped down to none and run here. When nothing can grant the ask at all it is
still cut down and run — a flow written on a cluster has to work on a laptop —
but the ceiling is one real machine now, since taking the largest of each
dimension separately can describe a machine nobody has.

Two things fixed on the way. A device on a connector node held every batch run
of its flow forever, waiting for a worker that could never run an entry point.
And `prefer` falling back to the engine skipped the books, so the fallback held
nothing.

The bench flow's node has taken a `params` argument that with_settings has not
forwarded for some time, so the benchmark could not run at all: 62 ms median
submit-to-result with this, against the 61 ms on record.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
This commit is contained in:
2026-08-27 08:49:36 +02:00
co-authored by Claude Opus 5
parent 1a9753fa9d
commit 6ff56533f5
14 changed files with 1214 additions and 326 deletions
+10
View File
@@ -60,6 +60,16 @@ class Resources(BaseModel):
"runs, which is what keeps two preallocating processes apart."
),
)
ram: int | None = Field(
default=None,
ge=1,
description=(
"Megabytes held for the whole execution. Counted against machines "
"that said how much they have, and ignored by those that did not — "
"which is a machine with nothing to say about memory, not one with "
"none."
),
)
env: dict[str, str] = Field(
default_factory=dict,
description=(