Stage caching for batch runs, and an engine that lives in the command
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -342,6 +342,8 @@ class Run(SQLModel, table=True):
|
||||
parent_id: str | None = Field(default=None, max_length=64)
|
||||
#: api, hook, sweep or cli.
|
||||
cause: str = Field(default="api", max_length=32)
|
||||
#: Re-execute every node, whatever the stage cache holds for it.
|
||||
no_cache: bool = False
|
||||
#: queued, running, ok, error, cancelled or abandoned.
|
||||
status: str = Field(default="queued", index=True, max_length=16)
|
||||
#: Why it is where it is: what it waits for, or what went wrong.
|
||||
@@ -392,10 +394,14 @@ class RunNode(SQLModel, table=True):
|
||||
worker: str = Field(default="local", max_length=64)
|
||||
error: str = ""
|
||||
logs: str = ""
|
||||
#: Everything this node's output depends on, hashed. Recorded from the
|
||||
#: start so that skipping a stage whose inputs have not changed is later a
|
||||
#: lookup rather than a migration.
|
||||
#: Everything this node's output depends on, hashed: its source, its
|
||||
#: settings and the values it read. What a later run looks itself up by.
|
||||
cache_key: str = Field(default="", index=True, max_length=64)
|
||||
#: What it returned, as canonical JSON, so a node with this key can be
|
||||
#: skipped and its outputs restored. None when it may not be reused —
|
||||
#: opted out, too large, or a value JSON cannot carry. Written with
|
||||
#: `cache_key` or not at all, so a keyed row is always restorable.
|
||||
outputs: str | None = None
|
||||
|
||||
|
||||
class RunMetric(SQLModel, table=True):
|
||||
|
||||
Reference in New Issue
Block a user