A cached node keeps its curve, and any input can name a run's output
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m42s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m41s
pre-commit / pre-commit (push) Failing after 2m53s
Test Backend / test-backend (push) Successful in 2m21s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Failing after 1m6s
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m42s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m41s
pre-commit / pre-commit (push) Failing after 2m53s
Test Backend / test-backend (push) Successful in 2m21s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Failing after 1m6s
A cache hit still replays no emissions — those values were the story of an
execution that is not happening — but the run they were recorded in is now
written on the row (`run_node.cached_from`), and the metrics endpoints read the
series back from there. So a reused run answers `run.metrics("train.loss")`
with the same points the run that trained did, rather than looking like a run
that produced no numbers at all. Pointed at rather than copied: a sweep of 500
reusing one frozen node would otherwise duplicate its curve 500 times.
That needed the cross-flow restore fixed first. The cache key has no flow in
it while the stored outputs are named for the flow that produced them, so
`quick.prepare` getting a hit from `train` wrote `train.dataset` into `quick`'s
state and the next node was called without its argument. One rule now covers
both halves: `requalify` reads a name owned by one flow as the same name in
another, applied to the restored outputs, to the node id behind the pointer,
and to the series names on the way out. Reuse across flows is kept.
Also: `@run:<id>.<output>` and a bare `sha256:` digest resolve on every input,
not only artifacts. Chaining a run's json config into the next one from a shell
meant pasting the whole object inline, and the CLI could not even send the
spelling — `_coerce` died in `json.loads` before the engine saw it. Both
spellings are reserved on every input now, `str` included, and `_from_run`
returns whatever the run's result holds rather than only a reference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp9L6gakMVro1K2C5zdtBE
This commit is contained in:
+9
-7
@@ -193,19 +193,21 @@ is labelled with is the number your code actually drew from, instead of merely
|
||||
looking like it. A flow that declares no such input still records it, and
|
||||
nothing reads it. Sweep over seeds with `--param seed=1,2,3`.
|
||||
|
||||
An input declared as an `artifact` takes the file a previous run produced,
|
||||
named rather than typed out:
|
||||
Any input takes what a previous run produced, named rather than typed out —
|
||||
a checkpoint, but equally a config object nobody wants to paste into a shell:
|
||||
|
||||
```sh
|
||||
fluksio run evaluate --dataset @run:1758042000123-9f2ab41c.dataset
|
||||
fluksio run evaluate --dataset sha256:6dd1f0…
|
||||
fluksio run train --meta @run:1758042000123-9f2ab41c.dataset_meta
|
||||
```
|
||||
|
||||
`@run:<id>.<output>` is what that run's output was, and a bare digest is the
|
||||
content itself; the engine resolves either into the reference. Passing the
|
||||
whole reference as JSON still works and is what a script that already holds one
|
||||
does — which is the same thing `flow.submit(dataset=run.result["dataset"])`
|
||||
does from Python.
|
||||
`@run:<id>.<output>` is whatever that run's output was, whole and with its own
|
||||
type; a bare digest is the content itself, resolved into a reference. Both
|
||||
spellings are reserved on every input, `str` included, so an input that has to
|
||||
carry one of them literally cannot. Passing the value as JSON still works and
|
||||
is what a script that already holds one does — the same thing
|
||||
`flow.submit(dataset=run.result["dataset"])` does from Python.
|
||||
|
||||
Run a flow with no parameters at a terminal and it asks for them, one line per
|
||||
declared input, with the declared value in brackets:
|
||||
|
||||
+11
-9
@@ -202,11 +202,13 @@ canvas and the API can change it too — or for one run with
|
||||
`fluksio run --no-cache`, `fluksio sweep --no-cache`, or `"no_cache": true` in
|
||||
the submission body.
|
||||
|
||||
What a cached node does not bring back is what it emitted on the way. Its
|
||||
returned outputs are restored; the values it published mid-execution are not,
|
||||
because those were the story of an execution that is not happening this time.
|
||||
So a skipped training node contributes no loss curve to the new run — if you
|
||||
want the curve, that run has to actually train.
|
||||
A cached node replays no emissions — those values were the story of an
|
||||
execution that is not happening this time — so its series is not rewritten
|
||||
either. The run it was restored from is recorded instead, and that is where the
|
||||
curve is read back from: asking the reusing run for its metrics answers with
|
||||
the same points, under its own flow's names. The one way to be left with a
|
||||
result and no curve is for that earlier run to have been deleted, which
|
||||
deleting its flow does.
|
||||
|
||||
## Objects that cannot be serialized
|
||||
|
||||
@@ -331,10 +333,10 @@ sweep, or specific runs. It re-reads on its own and whenever a run finishes.
|
||||
|
||||
### When a run draws nothing
|
||||
|
||||
A node restored from the [stage cache](#stage-caching) replays no emissions —
|
||||
a cache hit returns what the node returned, not what it emitted on the way. So
|
||||
a run that reused an earlier one has a result and an empty curve, and the
|
||||
chart says so rather than looking broken.
|
||||
A node restored from the [stage cache](#stage-caching) has its curve read back
|
||||
from the run that recorded it. Delete that run — deleting its flow does — and
|
||||
the reusing run is left with a result and an empty curve, and the chart says so
|
||||
rather than looking broken.
|
||||
|
||||
## What this costs, compared
|
||||
|
||||
|
||||
@@ -99,12 +99,6 @@ stays valid wherever the store is reachable from, including on another machine.
|
||||
Node code produces one with `fluksio.save_artifact` and opens one with
|
||||
`fluksio.load_artifact`. See [Writing node code](../code/nodes.md#bytes-artifacts).
|
||||
|
||||
As a *run parameter* it is also accepted as text, since nobody wants to paste
|
||||
the object into a shell: `@run:<id>.<output>` names what a run produced, and a
|
||||
bare `sha256:…` digest names the content. Both resolve to the reference above
|
||||
before the run starts, so the CLI, the run dialog and a python caller all mean
|
||||
the same thing.
|
||||
|
||||
### `json`
|
||||
|
||||
Anything JSON-serializable. The escape hatch, and the right answer when a
|
||||
@@ -113,6 +107,18 @@ payload genuinely has no fixed shape.
|
||||
Reach for it last. A `json` port tells the canvas, the widget picker and the
|
||||
next author nothing.
|
||||
|
||||
## Naming a run's output
|
||||
|
||||
Any *run parameter* is also accepted as text, since nobody wants to paste an
|
||||
object into a shell. `@run:<id>.<output>` names what an earlier run produced —
|
||||
whatever its type, an `artifact` reference or a `json` config alike — and a bare
|
||||
`sha256:…` digest names content in the artifact store. Both resolve before the
|
||||
run starts, so the CLI, the run dialog and a python caller all mean the same
|
||||
thing by the same string.
|
||||
|
||||
Both spellings are reserved on every input, `str` included: an input that has to
|
||||
carry one of them literally is asking for a value this engine reads as a name.
|
||||
|
||||
## What a widget will bind to
|
||||
|
||||
| Widget | Accepts |
|
||||
|
||||
Reference in New Issue
Block a user