Stage caching for batch runs, and an engine that lives in the command
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -175,6 +175,7 @@ class NodeSpec:
|
||||
timeout: float | None,
|
||||
device: str | None,
|
||||
device_policy: str,
|
||||
cache: bool = True,
|
||||
) -> None:
|
||||
self.fn = fn
|
||||
self.id = id
|
||||
@@ -186,6 +187,7 @@ class NodeSpec:
|
||||
self.timeout = timeout
|
||||
self.device = device
|
||||
self.device_policy = device_policy
|
||||
self.cache = cache
|
||||
|
||||
def rebind(
|
||||
self, *, id: str = "", wire: dict[str, str] | None = None, **settings: Any
|
||||
@@ -216,6 +218,7 @@ class NodeSpec:
|
||||
timeout=self.timeout,
|
||||
device=self.device,
|
||||
device_policy=self.device_policy,
|
||||
cache=self.cache,
|
||||
)
|
||||
|
||||
|
||||
@@ -246,6 +249,7 @@ def node(
|
||||
timeout: float | None = None,
|
||||
device: str | None = None,
|
||||
device_policy: str = "require",
|
||||
cache: bool = True,
|
||||
) -> Callable[[F], F]:
|
||||
"""Mark a function as a node, declaring its ports.
|
||||
|
||||
@@ -257,6 +261,10 @@ def node(
|
||||
the canvas can tune them without editing code. ``device`` picks the worker
|
||||
the code runs on — ``"gpu"``, say — and ``device_policy="prefer"`` runs it
|
||||
locally when no such worker is attached rather than waiting for one.
|
||||
|
||||
A batch run skips this node when an earlier one already ran the same source
|
||||
with the same settings and the same input values, and restores what it
|
||||
returned. ``cache=False`` says not to: the answer can change on its own.
|
||||
"""
|
||||
if device_policy not in ("require", "prefer"):
|
||||
raise SyncError("device_policy is 'require' or 'prefer'")
|
||||
@@ -275,6 +283,7 @@ def node(
|
||||
timeout=timeout,
|
||||
device=device,
|
||||
device_policy=device_policy,
|
||||
cache=cache,
|
||||
)
|
||||
_check_signature(spec)
|
||||
fn.__fluksio__ = spec # type: ignore[attr-defined]
|
||||
@@ -417,6 +426,7 @@ def _check(spec: NodeSpec, mode: str) -> dict[str, Any]:
|
||||
"timeout": spec.timeout,
|
||||
"device": spec.device,
|
||||
"device_policy": spec.device_policy,
|
||||
"cache": spec.cache,
|
||||
}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user