Stage caching for batch runs, and an engine that lives in the command
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -30,6 +30,8 @@ class RunCreate(BaseModel):
|
||||
seed: int | None = None
|
||||
#: Run the unpublished draft instead of what is published.
|
||||
draft: bool = False
|
||||
#: Execute every node, whatever an earlier run already worked out.
|
||||
no_cache: bool = False
|
||||
|
||||
|
||||
class SweepEntry(BaseModel):
|
||||
@@ -40,6 +42,7 @@ class SweepEntry(BaseModel):
|
||||
class SweepCreate(BaseModel):
|
||||
runs: list[SweepEntry] = Field(default_factory=list)
|
||||
draft: bool = False
|
||||
no_cache: bool = False
|
||||
|
||||
|
||||
class RunNodeRow(BaseModel):
|
||||
@@ -50,6 +53,9 @@ class RunNodeRow(BaseModel):
|
||||
worker: str
|
||||
error: str
|
||||
logs: str
|
||||
#: What this node's result was looked up by. Empty when it may not be
|
||||
#: reused; `status` is "cached" when it was.
|
||||
cache_key: str = ""
|
||||
|
||||
|
||||
class ArtifactRow(BaseModel):
|
||||
@@ -96,6 +102,7 @@ class MetricPoint(BaseModel):
|
||||
step: int
|
||||
ts: float
|
||||
value: float
|
||||
name: str = ""
|
||||
|
||||
|
||||
class MetricSeries(BaseModel):
|
||||
@@ -132,6 +139,7 @@ async def create_run(
|
||||
cause="api",
|
||||
actor=user.email,
|
||||
draft=body.draft,
|
||||
no_cache=body.no_cache,
|
||||
)
|
||||
except FlowNotFound as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
@@ -168,6 +176,7 @@ async def create_sweep(
|
||||
cause="sweep",
|
||||
actor=user.email,
|
||||
draft=body.draft,
|
||||
no_cache=body.no_cache,
|
||||
)
|
||||
for entry in body.runs
|
||||
]
|
||||
@@ -233,17 +242,18 @@ async def cancel_run(run_id: str, request: Request, session: SessionDep) -> Any:
|
||||
|
||||
|
||||
@router.get("/{run_id}/metrics", response_model=list[MetricPoint])
|
||||
def read_metrics(run_id: str, session: SessionDep, name: str, stride: int = 1) -> Any:
|
||||
"""One metric's series, in step order.
|
||||
def read_metrics(
|
||||
run_id: str, session: SessionDep, name: str = "", stride: int = 1
|
||||
) -> Any:
|
||||
"""One metric's series, in step order — or every one of them, unnamed.
|
||||
|
||||
``stride`` thins a long curve down: 3000 steps drawn on a 400-pixel chart
|
||||
is 3000 points nobody can see.
|
||||
"""
|
||||
statement = (
|
||||
select(RunMetric)
|
||||
.where(col(RunMetric.run_id) == run_id, col(RunMetric.name) == name)
|
||||
.order_by(col(RunMetric.step))
|
||||
)
|
||||
statement = select(RunMetric).where(col(RunMetric.run_id) == run_id)
|
||||
if name:
|
||||
statement = statement.where(col(RunMetric.name) == name)
|
||||
statement = statement.order_by(col(RunMetric.name), col(RunMetric.step))
|
||||
rows = list(session.exec(statement))
|
||||
if stride > 1:
|
||||
rows = rows[:: max(1, stride)]
|
||||
|
||||
Reference in New Issue
Block a user