Bound warm workers' host memory and account local RAM
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 12s
Playwright Tests / test-playwright (2, 2) (push) Failing after 9s
pre-commit / pre-commit (push) Failing after 1m56s
Test Backend / test-backend (push) Failing after 2m50s
Compose Smoke Test / test-compose (push) Failing after 9s
Playwright Tests / merge-reports (push) Failing after 2m16s

A worker over FLOW_WORKER_MAX_RSS_MB (--worker-max-rss) when its node
returns is retired instead of kept warm, in the shared pool and in every
declared one; 0, the default, keeps them all. The engine now accounts its
own memory (FLOW_RAM / --ram, else the box or the cgroup limit less 1 GB),
so a node's `ram` declaration holds locally. A declared environment the
shared pool already has is that pool rather than a duplicate set of warm
workers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-27 16:11:13 +02:00
co-authored by Claude Opus 5.5
parent cc4091166e
commit dafdcb76c4
10 changed files with 143 additions and 8 deletions
+15
View File
@@ -19,6 +19,7 @@ from fluksio.flow.resources import (
derive_env,
fair_share_env,
machine_gpus,
machine_ram_mb,
)
from fluksio.flow.schemas import NodeDef, Resources
@@ -44,6 +45,20 @@ def test_the_cards_are_counted_from_the_devices_the_driver_makes(monkeypatch):
assert machine_gpus() == 0
def test_memory_is_the_containers_limit_where_that_is_lower(monkeypatch, tmp_path):
"""The host's total is memory a container would be killed for using."""
from fluksio.flow import resources
box = os.sysconf("SC_PHYS_PAGES") * os.sysconf("SC_PAGE_SIZE") // 2**20
limit = tmp_path / "memory.max"
monkeypatch.setattr(resources, "CGROUP_MEMORY_MAX", limit)
limit.write_text("max\n")
assert machine_ram_mb() == box - resources.ENGINE_RAM_RESERVE_MB
limit.write_text(f"{2048 * 2**20}\n")
assert machine_ram_mb() == 2048 - resources.ENGINE_RAM_RESERVE_MB
def test_what_is_free_is_what_was_handed_out():
accountant = ResourceAccountant(cpus=4, gpus=0)
held = accountant.try_take(cpus=3, gpus=0)