Stage caching for batch runs, and an engine that lives in the command
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1446,6 +1446,11 @@ export const MetricPointSchema = {
|
||||
value: {
|
||||
type: 'number',
|
||||
title: 'Value'
|
||||
},
|
||||
name: {
|
||||
type: 'string',
|
||||
title: 'Name',
|
||||
default: ''
|
||||
}
|
||||
},
|
||||
type: 'object',
|
||||
@@ -1628,6 +1633,12 @@ export const NodeDef_InputSchema = {
|
||||
title: 'Device Policy',
|
||||
description: 'What to do when no worker carries `device`: wait for one, or run locally anyway.',
|
||||
default: 'require'
|
||||
},
|
||||
cache: {
|
||||
type: 'boolean',
|
||||
title: 'Cache',
|
||||
description: 'Whether a batch run may reuse an earlier execution of this node with the same source, settings and inputs. Turn it off for a function whose answer can change on its own.',
|
||||
default: true
|
||||
}
|
||||
},
|
||||
type: 'object',
|
||||
@@ -1717,6 +1728,12 @@ export const NodeDef_OutputSchema = {
|
||||
title: 'Device Policy',
|
||||
description: 'What to do when no worker carries `device`: wait for one, or run locally anyway.',
|
||||
default: 'require'
|
||||
},
|
||||
cache: {
|
||||
type: 'boolean',
|
||||
title: 'Cache',
|
||||
description: 'Whether a batch run may reuse an earlier execution of this node with the same source, settings and inputs. Turn it off for a function whose answer can change on its own.',
|
||||
default: true
|
||||
}
|
||||
},
|
||||
type: 'object',
|
||||
@@ -2399,6 +2416,11 @@ export const RunCreateSchema = {
|
||||
type: 'boolean',
|
||||
title: 'Draft',
|
||||
default: false
|
||||
},
|
||||
no_cache: {
|
||||
type: 'boolean',
|
||||
title: 'No Cache',
|
||||
default: false
|
||||
}
|
||||
},
|
||||
type: 'object',
|
||||
@@ -2551,6 +2573,11 @@ export const RunNodeRowSchema = {
|
||||
logs: {
|
||||
type: 'string',
|
||||
title: 'Logs'
|
||||
},
|
||||
cache_key: {
|
||||
type: 'string',
|
||||
title: 'Cache Key',
|
||||
default: ''
|
||||
}
|
||||
},
|
||||
type: 'object',
|
||||
@@ -2781,6 +2808,11 @@ export const SweepCreateSchema = {
|
||||
type: 'boolean',
|
||||
title: 'Draft',
|
||||
default: false
|
||||
},
|
||||
no_cache: {
|
||||
type: 'boolean',
|
||||
title: 'No Cache',
|
||||
default: false
|
||||
}
|
||||
},
|
||||
type: 'object',
|
||||
|
||||
@@ -1820,7 +1820,7 @@ export class RunsService {
|
||||
|
||||
/**
|
||||
* Read Metrics
|
||||
* One metric's series, in step order.
|
||||
* One metric's series, in step order — or every one of them, unnamed.
|
||||
*
|
||||
* ``stride`` thins a long curve down: 3000 steps drawn on a 400-pixel chart
|
||||
* is 3000 points nobody can see.
|
||||
|
||||
@@ -556,6 +556,7 @@ export type MetricPoint = {
|
||||
step: number;
|
||||
ts: number;
|
||||
value: number;
|
||||
name?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -620,6 +621,10 @@ export type NodeDef_Input = {
|
||||
* What to do when no worker carries `device`: wait for one, or run locally anyway.
|
||||
*/
|
||||
device_policy?: 'require' | 'prefer';
|
||||
/**
|
||||
* Whether a batch run may reuse an earlier execution of this node with the same source, settings and inputs. Turn it off for a function whose answer can change on its own.
|
||||
*/
|
||||
cache?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -656,6 +661,10 @@ export type NodeDef_Output = {
|
||||
* What to do when no worker carries `device`: wait for one, or run locally anyway.
|
||||
*/
|
||||
device_policy?: 'require' | 'prefer';
|
||||
/**
|
||||
* Whether a batch run may reuse an earlier execution of this node with the same source, settings and inputs. Turn it off for a function whose answer can change on its own.
|
||||
*/
|
||||
cache?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -864,6 +873,7 @@ export type RunCreate = {
|
||||
};
|
||||
seed?: (number | null);
|
||||
draft?: boolean;
|
||||
no_cache?: boolean;
|
||||
};
|
||||
|
||||
export type RunDetail = {
|
||||
@@ -902,6 +912,7 @@ export type RunNodeRow = {
|
||||
worker: string;
|
||||
error: string;
|
||||
logs: string;
|
||||
cache_key?: string;
|
||||
};
|
||||
|
||||
export type RunPage = {
|
||||
@@ -982,6 +993,7 @@ export type ShareRequest = {
|
||||
export type SweepCreate = {
|
||||
runs?: Array<SweepEntry>;
|
||||
draft?: boolean;
|
||||
no_cache?: boolean;
|
||||
};
|
||||
|
||||
export type SweepEntry = {
|
||||
@@ -1606,7 +1618,7 @@ export type RunsCancelRunData = {
|
||||
export type RunsCancelRunResponse = (fluksio__api__routes__runs__RunRow);
|
||||
|
||||
export type RunsReadMetricsData = {
|
||||
name: string;
|
||||
name?: string;
|
||||
runId: string;
|
||||
stride?: number;
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user