Document the scheduler
Docs / docs (push) Successful in 32s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m56s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m0s
pre-commit / pre-commit (push) Failing after 3m8s
Test Backend / test-backend (push) Successful in 2m45s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Successful in 1m15s

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
This commit is contained in:
2026-08-27 09:36:39 +02:00
co-authored by Claude Opus 5
parent 7d87dfffdd
commit e2f97d36e4
3 changed files with 63 additions and 7 deletions
+40 -5
View File
@@ -299,16 +299,51 @@ threshold, a message on its way somewhere. Those share the engine's worker pool
and are given a fair share of `FLOW_CPUS` as a thread cap, which is what stops
several of them at once from each sizing to the whole box.
### Ask for a size by name
Cores and gigabytes are a property of the machines you have, and those change.
A node that names a **flavor** keeps meaning something afterwards:
```python
@node(..., resources="gpu-small")
def finetune(checkpoint):
...
@node(..., resources={"flavor": "medium", "duration_s": "2h"})
def fit(dataset, lr, epochs=25):
...
```
The flavor is read every time the node is built, so editing it changes what the
next run gets. `fluksio flavors` lists them; the Workers screen edits them.
`ram` takes `"2G"` and `duration_s` takes `"30m"`, and a flavor already says how
much — pass one or the numbers, not both. `duration_s` is a statement about the
node for whoever is planning around it, not a limit; the limit is `timeout`.
### Where it runs
This is one decision, not two. Of every machine attached — this engine and each
worker — the node goes to one that can grant what it asked for and has it free.
So a node asking for a card finds the box that has one, without naming it:
```python
@node(..., resources={"gpus": 1}) # wherever there is a card
@node(..., device="gpu", resources="gpu-small") # that box, and this much of it
```
If nothing attached can ever grant the ask, it is cut down to what is here and
runs anyway — a flow written on a cluster still has to work on a laptop. If
something *could* but is busy, the node waits and says so.
Ask what is free, and what is waiting for it, at
`GET /api/v1/workers/resources` — a node queued for cores looks exactly like a
node that has hung unless you can see the queue.
`GET /api/v1/workers/resources`, on the Workers screen, or in `fluksio status`.
!!! note "Accounted, not enforced"
Nothing stops a node that ignores its declaration; the numbers are
bookkeeping plus the environment its libraries read. Scheduling across
several machines, named hardware flavours and real enforcement are the
next steps, not this one.
bookkeeping plus the environment its libraries read. Real enforcement —
cgroups, rlimits — is a next step, not this one.
## Say which nodes make a flow