A cached node keeps its curve, and any input can name a run's output
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m42s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m41s
pre-commit / pre-commit (push) Failing after 2m53s
Test Backend / test-backend (push) Successful in 2m21s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Failing after 1m6s

A cache hit still replays no emissions — those values were the story of an
execution that is not happening — but the run they were recorded in is now
written on the row (`run_node.cached_from`), and the metrics endpoints read the
series back from there. So a reused run answers `run.metrics("train.loss")`
with the same points the run that trained did, rather than looking like a run
that produced no numbers at all. Pointed at rather than copied: a sweep of 500
reusing one frozen node would otherwise duplicate its curve 500 times.

That needed the cross-flow restore fixed first. The cache key has no flow in
it while the stored outputs are named for the flow that produced them, so
`quick.prepare` getting a hit from `train` wrote `train.dataset` into `quick`'s
state and the next node was called without its argument. One rule now covers
both halves: `requalify` reads a name owned by one flow as the same name in
another, applied to the restored outputs, to the node id behind the pointer,
and to the series names on the way out. Reuse across flows is kept.

Also: `@run:<id>.<output>` and a bare `sha256:` digest resolve on every input,
not only artifacts. Chaining a run's json config into the next one from a shell
meant pasting the whole object inline, and the CLI could not even send the
spelling — `_coerce` died in `json.loads` before the engine saw it. Both
spellings are reserved on every input now, `str` included, and `_from_run`
returns whatever the run's result holds rather than only a reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp9L6gakMVro1K2C5zdtBE
This commit is contained in:
2026-08-25 15:12:28 +02:00
co-authored by Claude Opus 5
parent aa2bd1e665
commit 3c15964364
18 changed files with 442 additions and 106 deletions
+33 -4
View File
@@ -11,7 +11,7 @@ import pytest
from fluksio.flow.events import EventBus
from fluksio.flow.messages import DType, MessageSpec
from fluksio.flow.nodes import Node
from fluksio.flow.pipeline import NodeOutcome, Pipeline, run_cache_key
from fluksio.flow.pipeline import CacheHit, NodeOutcome, Pipeline, run_cache_key
from fluksio.flow.runs import (
MetricSink,
RunRejected,
@@ -351,15 +351,20 @@ def test_a_flow_without_a_seed_input_ignores_the_runs_seed():
class FakeCache:
"""A stage cache with no database behind it, and a record of what it was asked."""
def __init__(self, entries: dict[str, dict | None] | None = None) -> None:
def __init__(
self, entries: dict[str, dict | None] | None = None, flow: str = "study"
) -> None:
self.entries = entries or {}
self.flow = flow
self.asked: list[str] = []
def lookup(self, key: str):
self.asked.append(key)
if key in self.entries:
return True, self.entries[key]
return False, None
return CacheHit(
flow=self.flow, outputs=self.entries[key], metrics_run="earlier"
)
return None
def counting_node(flow: str = "study") -> tuple[Node, list[int]]:
@@ -394,6 +399,30 @@ def test_a_cache_hit_restores_the_outputs_without_running_the_node():
# downstream of it looks — its namespace holds nothing otherwise.
assert collect_result(flow, state) == {"loss": 99.0}
assert seen[0].cached and seen[0].cache_key == key
# Where its series is, since the hit replayed none of it.
assert seen[0].cached_from == "earlier"
def test_a_hit_from_another_flow_restores_under_this_flow_s_names():
"""The same node reached through two flows publishes under two names."""
flow = double_flow()
node, calls = counting_node()
state = MemoryState()
seen: list[NodeOutcome] = []
key = run_cache_key("fp-train", {"study.lr": 0.5})
# Recorded by a run of "other", which is what a node shared between two
# flows gets a hit from — the values are the same, the namespace is not.
cache = FakeCache({key: {"other.loss": 99.0}}, flow="other")
Pipeline(nodes=[node], state=state, observer=seen.append, run_cache=cache).run(
seed_values(flow, {"lr": 0.5})
)
assert calls == []
assert collect_result(flow, state) == {"loss": 99.0}
# And stored that way too, so the next run to reuse this one finds names
# it can requalify from a flow that really did publish them.
assert seen[0].output_values == {"study.loss": 99.0}
def test_a_miss_runs_the_node_and_carries_what_would_be_stored():