Close eight open SDK tasks: the pidfile, the log, cards, names and a live curve
Each was a loose end recorded under `### SDK` in the notepad. `serve` takes its own pidfile down on SIGTERM. uvicorn restores the handler it found and re-raises the signal it stopped on, so the default handler ended the process without unwinding and the `finally` never ran — which is what a stop sends, and what left `serve.pid` behind. `serve.log` is cut back past 5 MB by the engine rather than by the screen that started it, so an adopted engine is bounded too. Gated on its own stdout being an appended regular file, which is what makes the cut safe: the kernel then puts the next write at the new end. Cards are counted from `/dev/nvidia[0-9]*`, so `FLOW_GPUS`/`--gpus` of 0 means "work it out" the way `FLOW_CPUS` always has. The engine counts, not the accountant — a remote worker builds one of those from its own inventory, and detecting there would hand it the engine host's cards. The worker counts last: what a batch job says it was granted still wins. `GET /runs/metrics/names` is the distinct over a selection that `--list` and the terminal's metric picker were approximating by reading the newest run that had measured anything, which missed a name only an older run ever wrote. `MetricSink` announces each batch it has written (`run_metric`, carrying the names). Not a per-point event: one covers up to 500 points or two seconds of them, and the rows stay the record. The terminal comparison fills in as the first readings land instead of staying blank until reopened, and the browser refetches the run and any comparison rather than the list behind them. `retry --group` pages the list route by `before` instead of stopping at 500. The terminal dashboard takes the terminal's colours (`ansi-dark`), and the web UI can re-pair from Settings without disconnecting first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRQ9bmTvCbqCwXo9mxZzzV
This commit is contained in:
@@ -18,10 +18,32 @@ from fluksio.flow.resources import (
|
||||
ResourceAccountant,
|
||||
derive_env,
|
||||
fair_share_env,
|
||||
machine_gpus,
|
||||
)
|
||||
from fluksio.flow.schemas import NodeDef, Resources
|
||||
|
||||
|
||||
def test_the_cards_are_counted_from_the_devices_the_driver_makes(monkeypatch):
|
||||
"""A box with cards and an engine told nothing used to have none of them.
|
||||
|
||||
NVIDIA's own device nodes, counted — no vendor tool, so the one dependency
|
||||
does not become two. The control files beside them are not cards.
|
||||
"""
|
||||
from fluksio.flow import resources
|
||||
|
||||
monkeypatch.setattr(
|
||||
resources.glob,
|
||||
"glob",
|
||||
lambda pattern: (
|
||||
["/dev/nvidia0", "/dev/nvidia1"] if pattern == "/dev/nvidia[0-9]*" else []
|
||||
),
|
||||
)
|
||||
assert machine_gpus() == 2
|
||||
|
||||
monkeypatch.setattr(resources.glob, "glob", lambda pattern: [])
|
||||
assert machine_gpus() == 0
|
||||
|
||||
|
||||
def test_what_is_free_is_what_was_handed_out():
|
||||
accountant = ResourceAccountant(cpus=4, gpus=0)
|
||||
held = accountant.try_take(cpus=3, gpus=0)
|
||||
|
||||
@@ -296,6 +296,30 @@ def test_emissions_reach_the_run_as_a_series_with_a_step_each():
|
||||
assert sink._steps == {"study.loss": 1}
|
||||
|
||||
|
||||
def test_a_written_batch_says_it_is_there():
|
||||
"""What a screen watching a live run waits on.
|
||||
|
||||
The rows are still the record — this is a nudge carrying names, one per
|
||||
batch rather than one per point, so nothing lands on the hot path.
|
||||
"""
|
||||
said = []
|
||||
sink = MetricSink("run-1", batch=1, on_flush=said.append)
|
||||
|
||||
sink.handle("study.train", {"study.loss": 1.0, "study.tag": "ignored"})
|
||||
assert said == [["study.loss"]]
|
||||
|
||||
# Held back until the batch is due, then announced once for all of it.
|
||||
quiet = MetricSink("run-2", batch=10, interval=3600, on_flush=said.append)
|
||||
quiet.handle("study.train", {"study.loss": 1.0, "study.acc": 0.5})
|
||||
assert said == [["study.loss"]]
|
||||
quiet.flush()
|
||||
assert said[-1] == ["study.acc", "study.loss"]
|
||||
|
||||
# Nothing to write is nothing to say.
|
||||
quiet.flush()
|
||||
assert len(said) == 2
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------------
|
||||
# What a caller may ask for
|
||||
# -----------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user