Bound warm workers' host memory and account local RAM

A worker over FLOW_WORKER_MAX_RSS_MB (--worker-max-rss) when its node
returns is retired instead of kept warm, in the shared pool and in every
declared one; 0, the default, keeps them all. The engine now accounts its
own memory (FLOW_RAM / --ram, else the box or the cgroup limit less 1 GB),
so a node's `ram` declaration holds locally. A declared environment the
shared pool already has is that pool rather than a duplicate set of warm
workers.
This commit is contained in:
2026-09-27 16:11:13 +02:00
parent 7240b7b823
commit 0649062327
10 changed files with 143 additions and 8 deletions
+7
View File
@@ -315,6 +315,13 @@ given `CUDA_VISIBLE_DEVICES`, and leaves a warm worker holding whatever it
preallocated. Declaring the card is what makes it a worker the engine knows
to retire, and what stops two such nodes running at once in the first place.
Host memory is held the same way, and there the declaration is not needed. A
worker that fitted a model keeps what it allocated, and on a laptop a few warm
ones are the machine. `fluksio serve --worker-max-rss 1024` retires any worker
holding more than that when its node returns, whichever pool it is in. A
node's `ram` is counted against this machine's memory, so declaring it is what
stops three large fits from starting at once.
Declaring nothing is the default and is right for most nodes: a poll, a
threshold, a message on its way somewhere. Those share the engine's worker pool
and are given a fair share of `FLOW_CPUS` as a thread cap, which is what stops