15 Commits

Include renames

Author	SHA1	Message	Date
stroblmeandClaude Opus 5	f370601aec	Draw curves in the terminal, and stop waiting five seconds to hear ... `space` ticks runs on the serve dashboard's table and `enter` compares them: one metric in braille, five distinct hues, and the table of what actually differs under it. Both halves are routes that already existed — the browser's own comparison endpoint for the curves, the runs export for the table, whose input columns are filtered to the ones that vary. The palette is hue rather than the web's lightness ramp on purpose: five steps of one brand hue collapse to a single colour on a 16-colour tty. The screen also subscribes to the engine's event bus over the same websocket a browser uses, so a run that starts and finishes inside a tick is seen rather than only recorded. `a` lists what a run left behind and fetches it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014SmyLMSqcJQ8tUL2qLj21s	2026-08-30 12:51:08 +02:00
stroblmeandClaude Opus 5	73cd37a608	Export every recorded input, not only the ones that vary ... `export runs` dropped a `param.*` column whose value was constant across the exported runs, so a downstream filter broke depending on which runs the selection happened to hold. Every input the selection recorded is a column now; `--params` still narrows it to a sweep's axis. The metrics default is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Hra4ndWMCLU5F3KjUuVAc	2026-08-29 13:47:00 +02:00
stroblmeandClaude Opus 5	f8a7dfb101	A sync that changed nothing says so again ... Comparing the per-node digest against an engine that does not record one is comparing against nothing, and reporting every node as changed on every sync for ever — which is what a client newer than its engine did, since `NodeDef` drops fields it has never heard of. A node is named now only when both sides carry a digest, so a no-op sync is `unchanged` again and the signal one syncs for is back. That silence had also been the only sign of the mismatch, so sync now names it: one line saying the engine stored no record of what a node's code reaches, with both versions in it and what to run. Bumped to 0.1.6 — the digest changed the stored document's shape, and a version that does not move makes two different engines indistinguishable, which is the thing it was made load-bearing for a day ago. `— draft` was printed whenever there was simply nothing to publish, which reads as work left unfinished. It is said only when a draft is genuinely there, and `— published` when one was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa	2026-08-27 22:58:31 +02:00
stroblmeandClaude Opus 5	91ef2bbe9a	Key a node on the code it reaches, not on the repository around it ... `sync` follows each node function's imports through the project's own modules — stopping at the standard library, at anything installed, and at Fluksio itself, whose checkout would otherwise be most of every digest — and records the file list with what it hashed to. The engine hashes those files again when the run is claimed, so the fingerprint is live rather than a snapshot, and falls back to what sync recorded when it cannot see them: a remote worker's runs used to share one empty digest, and therefore one key. Three things follow. Editing a helper a node calls into re-runs that node, as before. Editing something the node never reaches no longer re-runs anything — a notebook two directories away was invalidating every arm. And `Run.code_digest` is now the hash of its nodes' digests, so it is neither looser nor tighter than "the code behind these numbers", which is what makes it worth joining an exported table on. `sync` says so too: it compares the per-node digest against the stored one, so a helper edit prints `train: updated (flow, fit)` instead of `unchanged`. The digest is read when the document is built rather than when the flow is declared, so a second `sync()` in one process sees an edit between them. Also: `fluksio runs` shows only the inputs that differ from what the flow declares, fitted to the terminal, so a flow taking a few kB of json no longer wraps every line. Every existing cache entry misses once — the fingerprint changed shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa	2026-08-27 22:35:23 +02:00
stroblmeandClaude Opus 5	51464941ac	Export runs and their curves as tables an analysis reads ... `fluksio export metrics` is the long table — a row per run, metric and step — and `fluksio export runs` the wide one, a row per run with the inputs that *vary* across the selection as columns beside its final numbers, status, duration and the commit and digest of the code it ran. Both carry the run id on every row, which is the join back to the run page and what makes an exported file auditable. `Client.export_metrics`/`export_runs` answer the same rows to a notebook. The engine streams csv or jsonl from two routes declared above `/{run_id}`; parquet is a client-side conversion behind the new `fluksio[parquet]` extra, so nobody pays for pyarrow who does not want dtypes kept. The long export reads each run through `_series`, so a cached node's curve comes with it, and `--stride` thins each series rather than the concatenation of all of them. Two things they needed on the way: `GET /runs` takes `?since=` and `?before=`, so a long history pages by the last row's own timestamp instead of an offset that shifts under it; and a read that reaches no engine now says so in half a second rather than seven, because `runs`, `flavors`, `export` and an unwatched `status` pass `retries=0`. Everything that submits keeps them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa	2026-08-27 17:43:30 +02:00
stroblmeandClaude Opus 5	37a7df9d24	Let a sweep run more than four at a time, and name the run a failure was in ... Concurrent runs sat at 4 whatever FLOW_MAX_CASCADES said: that setting bounds cascades, and the run drivers read a hardcoded MAX_PARALLEL nobody could reach. FLOW_MAX_RUNS is the knob they read now, --max-runs/--max-cascades/--max-workers are the same three as flags on serve, and the engine says which numbers it started with — which is the only way to tell that a settings file was read. Events keep the run they happened in. The payload always carried it and the persist path dropped it, so reading one run's failures meant filtering the engine-wide list; a batch run's id reaches those events now too, since a run has no journaled item to name itself by. Also: a provisioner's 0 means "no deadline" rather than "cancel on the next reconcile", and a command that reaches no engine says how to start one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015sbYeYaVgYQqm1sbx7wPdL	2026-08-27 14:17:51 +02:00
stroblmeandClaude Opus 5	a82f88cf0a	Name the sizes a node can ask for ... Raw cpus and gpus are a property of the machines an installation has, so a node written against a cluster quietly stops meaning anything when the cluster is replaced. A node says "gpu-small" instead, and what that is stored here — editable, and read again every time the node is built, so changing the flavor changes what the next run gets. Memory joins the schema properly (`ram`, in MB, accepting "2G"), along with `duration_s` for how long a node is expected to take. That one is recorded and shown and nothing else yet: a statement for whoever is planning around the node, not a limit — the limit is still `timeout`. A flavor and a number for the same thing is refused, compared by value so an editor writing the whole object back with its defaults still round-trips. A name nothing stores is refused at the save, which covers the canvas and `fluksio sync` at once, and deleting one a node still asks for says which node. Four sizes are seeded on an installation that has none, and never re-seeded: re-adding one somebody deliberately removed is an argument nobody wins. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW	2026-08-27 08:59:10 +02:00
stroblmeandClaude Opus 5	4a38c6ed31	Name the code a run ran, and let an interrupted sync finish ... Three faults with one root: the stored body of a code-defined node is an import shim, and nothing that mattered was ever read from the code itself. - The run stamp could not identify what ran. The shim imports whatever is on disk when the worker starts, and an uncommitted tree stamps <commit>-dirty for every run it ever produces. Run.code_digest hashes the repository's .py files, memoized on their stat state, and it is read again when the run is actually claimed -- so a sweep queued for hours records the code each of its runs executed, not the code that was there when it was submitted. - The stage cache adopted code that was too new. The fingerprint hashed the shim, which is invariant under any edit to the imported function or anything it calls into, so a re-run was served from cache and answered without the outputs the edit added. It now carries the repo digest and the node's declared ports. Every fingerprint changes once, which invalidates the existing cache; a canvas flow has no repository and keys as before. - An interrupted sync looked like a hand-edited canvas. The engine answers a new-node template for a node with no stored body, and the template carries no marker, so the drift check read "somebody edited this" and demanded --force -- for the one state that re-running the sync is the fix for. NodeSource.missing states the fact, and sync skips those and reuses the bodies it read instead of asking for each one twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 21:27:10 +02:00
stroblmeandClaude Opus 5	1f7c6646f1	Survive a busy engine: retry, idempotent submit, resilient waiting ... A driver script died of one slow answer: httpx.ReadTimeout out of RunHandle.refresh() with a 30 s read timeout and no retry anywhere, which cost a sweep 78 of its 84 runs. - Split the timeout (5 s connect, 120 s read): a wrong URL fails at once, and a busy engine gets longer than the slowest thing it does on purpose (a 60 s compile, a 15 s rebuild wait). - Retry idempotent calls three times on a transport error or 502/503/504. 503 is the engine's own "ask again" — it is what RebuildBusy answers. - Submit carries a key the engine stores with the run, so a retry after a timeout returns that run instead of starting a second. A sweep keys every entry, so a half-created one recreates only what is missing. - wait() and --follow tolerate five failed polls in a row; a 404 still stops at once, because that is an answer rather than a gap. - CLI says "engine not answering" and names the run still on the engine, instead of printing a traceback. - runs: clamp the params column to 80 characters; events() takes the flow/since/until the endpoint already had; RunHandle.failures answers "what killed this run" from the run's own node rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 21:19:09 +02:00
stroblmeandClaude Opus 5	1148e54c9e	A run says whether the dashboard, the CLI or a script asked for it ... `POST /runs/flows/{name}` hardcoded `cause: "api"`, so every row in the history claimed the same origin. The body now carries an optional `cause`, closed to the values the column knows — the dashboard sends nothing and stays "api", `fluksio run` says "cli", and the SDK client says "sdk". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK	2026-08-25 22:06:29 +02:00
stroblmeandClaude Opus 5	60757fa7fa	Style the engine's own logs, notice enrolment while serving, say more in status ... Four things from a testing pass. `fluksio serve` printed its own lines through the root logger, which has no handler and falls back to `INFO:fluksio.cloud.connector:...` — beside uvicorn's aligned output it reads like something went wrong. The engine's loggers and alembic's now use uvicorn's own handler. Named rather than configuring the root: httpx logs every portal call at INFO and none of that is printed today. `fluksio enroll` writes its config from another process, so an engine already serving never learned it had been paired. It now looks for one every few seconds and dials when it appears. `load()` rather than `exists()`, or a file that does not parse would be restarted forever. `fluksio status` says where the installation stands with its portal — never paired, linked, or paired and unreachable, which is the one worth acting on. `--seed` and `--timeout` had no help text at all. Both say what they are for now, and the docs say what a seed is actually for: recorded on the run, part of its input digest, and passed to an input named `seed` when the flow declares one, so the number a run is labelled with is the one the code drew from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69	2026-08-25 09:29:22 +02:00
stroblmeandClaude Opus 5	4941c2787c	Add fluksio status, and ask for a run's parameters at a terminal ... Two halves of the same gap: the CLI could start work but not show you any. `fluksio status` draws the home screen's top half in a terminal — health and what is wrong with it, every flow with its state and node count, and the recent runs and failures under them. `--watch` keeps it there. Rich does the drawing; it was already installed under fastapi's own CLI, and is named now because a command depends on it. `fluksio run` with no parameters at a terminal asks for them, one line per declared input with its declared value in brackets — so Enter through the lot is what running the defaults looks like, and an artifact input takes the `@run:` spelling the engine now resolves. A scripted run is untouched: passing any parameter, or piping the command, skips the questions, as does --defaults. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69	2026-08-25 07:42:09 +02:00
stroblmeandClaude Opus 5	400d7d9c5c	Stage caching for batch runs, and an engine that lives in the command ... A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-24 20:31:31 +02:00
stroblmeandClaude Fable 5	99f6530698	One installation per project, and no login to reach it ... Two things a local install should not have asked for. `fluksio serve` now signs you in. Logging in to your own machine was a formality — the password was printed by the same process that would have checked it, and the database it authenticates against sits in the directory the token goes into — so `serve` mints the token itself and says where it put it. `fluksio login` is left for an engine somewhere else. And an installation is `.fluksio` beside the code, found the way `.git` is, rather than one `~/.fluksio` for the machine. A repository with its own venv was already getting its own engine; it now gets its own flows, run history and token too, instead of three repositories sharing one database and fighting over one port. `--global` asks for the shared one, `--data-dir` still names any directory, and when both exist the banner says which you are looking at and how to reach the other. The directory ignores itself from within — a `.gitignore` of `*`, the way uv writes one into `.venv` — because it holds a credential and a database, and neither belongs in anybody's history. The token is written mode 600. A login an older version wrote to ~/.config/fluksio is still read, so nothing that worked stops working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU	2026-08-24 16:13:35 +02:00
stroblmeandClaude Fable 5	a38e2745eb	Add a Python SDK: flows declared in your own repository ... A data scientist keeps their code where it is and decorates it: `@node` declares a function's ports beside the function, `Flow(name, nodes=[...])` says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds one for a single flow. `fluksio sync` uploads the document plus a generated import shim per node, so the store still holds a complete, runnable, git-versioned definition while the code it imports stays theirs. `fluksio login|run|runs` and `flow.submit().wait()` are the client half, over the run endpoints that already existed. Runs record the user repository's commit beside the store's, so "what code produced this number" is answerable on the side that now holds the code. - `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim generator, an HTTP client and sync. Standard library only at import, so `from fluksio import node` in a training script pulls in no engine. - `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the repository's commit; `POST /modules/refresh` retires the workers without an install, which every sync calls — a worker holds the imported package in memory, so an edit to it is invisible until the process goes. - The canvas shows a generated body read-only and names the repository to edit instead; a body edited there stops the next sync rather than being discarded. - The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since the shim imports a module whose first line declares them. - `examples/myresearch` is the worked example, `make sync-example` uploads it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU	2026-08-23 20:16:08 +02:00