17 Commits

Include renames

Author	SHA1	Message	Date
stroblmeandClaude Opus 5	8a94bf10d7	Close the nine open SDK tasks: one engine per directory, a tabbed dashboard, re-pairing, run recovery ... serve: refuse a second engine for one data directory whatever port it was asked for, using the pidfile and a token this directory signed. The check runs before the database is touched and before the credential is written, which is what left every later CLI call pointing at a dead port. The terminal dashboard is three tabs (Overview, Runs, Logs) with the toolbar following the focused pane, the engine's output goes to serve.log rather than down a pipe, and closing the screen stops both reader threads so the prompt comes back. It adopts a running engine on every start, so stop/start and restart work on one it did not start, and a stop waits for the process to be gone before the next start. Enrolment reports itself in the modal. enroll: a new claim code replaces the pairing instead of being refused. The code is redeemed before anything is written, mappings to a portal being left are cleared, and a running engine redials when the stored enrolment changes. runs: an engine re-queues the runs left `queued` by the one before it, and `fluksio retry <id>` / `retry --group <sweep>` submits an interrupted run again with the same inputs and group, recorded through Run.parent_id. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U9BoNGq6V9MdRWAte7JBuC	2026-08-31 17:37:13 +02:00
stroblmeandClaude Opus 5	bdad6d7fc2	Make the docs state things rather than argue them ... The site read as a design journal: rationale paragraphs, hedges ("deliberately", "on purpose", "genuinely"), meta-commentary about the docs themselves, and one em-dash every ten lines carrying an aside. Roughly twenty rationale blocks are gone or reduced to what a reader needs in order to use the thing. Em-dashes go from 507 to 135, and what is left is structural rather than prose: list and definition separators, table cells, and four inside code blocks that quote what the CLI actually prints. Also: api.example.com becomes api.fluksio.com (the emails stay, since bootstrap.py really defaults to admin@example.com and RFC 2606 reserves it); the mqtt table gains the two settings it had drifted behind on and inject's wording matches the engine; llms.txt lists the two connector pages that were in the nav but not in it; and the two device/device_policy notes now agree. Builds clean under `zensical build --strict`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015YrQnKV3bnQd4K342y8tKj	2026-08-31 10:49:58 +02:00
stroblmeandClaude Opus 5	d01a8dad37	Rename Installation to Instance ... Follows the portal: the noun is "instance" everywhere the app says it — UI strings, CLI output, error details, docs and comments. The wire keys (`instance_id`, `instance_token`) and the hub route this calls move with it. An existing cloud.json is adopted rather than refused: without the key alias the dataclass fails to parse, which the caller swallows and reads as "never enrolled" instead of "reconnect". `instance_key` on a node type becomes `target_key`. It means the outside thing a node points at, which is a different sense of the word, and keeping both would put two meanings of "instance" in one codebase. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015YrQnKV3bnQd4K342y8tKj	2026-08-31 10:12:01 +02:00
stroblmeandClaude Opus 5	70e542ec3c	Surface a failing connector poll as node health and a flow issue ... The poll loop remembered what it read rather than what it published, so a value the node could not publish counted as said: the next poll skipped it, succeeded, and health went back to ok with the port still dark. Remember it only after inject returns, and report ok last. A node reporting itself down is now derived into its flow's issues on read and counted on the health summary, so the canvas marks it and Home says so. Being down does not stop the flow, and the issue clears by itself when the node reports well again. The repeating poll warning is logged once per outage rather than once per tick. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K1moruzue2kTJd3uVisgNk	2026-08-28 12:22:55 +02:00
stroblmeandClaude Opus 5	91ef2bbe9a	Key a node on the code it reaches, not on the repository around it ... `sync` follows each node function's imports through the project's own modules — stopping at the standard library, at anything installed, and at Fluksio itself, whose checkout would otherwise be most of every digest — and records the file list with what it hashed to. The engine hashes those files again when the run is claimed, so the fingerprint is live rather than a snapshot, and falls back to what sync recorded when it cannot see them: a remote worker's runs used to share one empty digest, and therefore one key. Three things follow. Editing a helper a node calls into re-runs that node, as before. Editing something the node never reaches no longer re-runs anything — a notebook two directories away was invalidating every arm. And `Run.code_digest` is now the hash of its nodes' digests, so it is neither looser nor tighter than "the code behind these numbers", which is what makes it worth joining an exported table on. `sync` says so too: it compares the per-node digest against the stored one, so a helper edit prints `train: updated (flow, fit)` instead of `unchanged`. The digest is read when the document is built rather than when the flow is declared, so a second `sync()` in one process sees an edit between them. Also: `fluksio runs` shows only the inputs that differ from what the flow declares, fitted to the terminal, so a flow taking a few kB of json no longer wraps every line. Every existing cache entry misses once — the fingerprint changed shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa	2026-08-27 22:35:23 +02:00
stroblmeandClaude Opus 5	4479eeb726	Follow a record into its fields, name the metrics, name the version ... Three things the first export pass got wrong for a real study. **Dotted paths.** A node returns a record, not a scalar — the numbers arrive inside `final_metrics` — so `--metrics final_metrics.train_loss` yielded an empty column and `--metrics final_metrics` yielded the whole record in one cell. Both sides of the wide table now take dotted paths, and the defaults reach the same depth: every number a result carries is a column named by its path, and inputs are compared leaf by leaf, so two configurations differing in one field give that field as the axis rather than two blobs that are merely not equal. Lists stay whole — a curve belongs in the long table. **`--list`.** Metric names are flow-qualified, so `--name train_loss` matched nothing and said only that. `fluksio export metrics --list` prints the names the selection carries, and an empty export made with `--name` points at it. **A version to compare.** The CLI ships ahead of the engine and a stale one answered a flat 404 with nothing anywhere in the API to tell how old it was. The engine reports `version` on `/observability/summary`, `fluksio status` prints it, and a 404 from export now names both versions — or says "older" when the field itself predates the engine. Bumped to 0.1.5, which is what makes the number worth reading. Also formats `flow/metrics.py`, which had been committed unformatted and was the last `ruff format --check` failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa	2026-08-27 20:42:19 +02:00
stroblmeandClaude Opus 5	51464941ac	Export runs and their curves as tables an analysis reads ... `fluksio export metrics` is the long table — a row per run, metric and step — and `fluksio export runs` the wide one, a row per run with the inputs that *vary* across the selection as columns beside its final numbers, status, duration and the commit and digest of the code it ran. Both carry the run id on every row, which is the join back to the run page and what makes an exported file auditable. `Client.export_metrics`/`export_runs` answer the same rows to a notebook. The engine streams csv or jsonl from two routes declared above `/{run_id}`; parquet is a client-side conversion behind the new `fluksio[parquet]` extra, so nobody pays for pyarrow who does not want dtypes kept. The long export reads each run through `_series`, so a cached node's curve comes with it, and `--stride` thins each series rather than the concatenation of all of them. Two things they needed on the way: `GET /runs` takes `?since=` and `?before=`, so a long history pages by the last row's own timestamp instead of an offset that shifts under it; and a read that reaches no engine now says so in half a second rather than seven, because `runs`, `flavors`, `export` and an unwatched `status` pass `retries=0`. Everything that submits keeps them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa	2026-08-27 17:43:30 +02:00
stroblmeandClaude Opus 5	0ffcabfdb9	Media dtypes: image, audio and video as narrowed artifact references ... A port may now declare `image`, `audio` or `video`. Each is the artifact reference the engine already had, narrowed by the `media_type` on it, so a speech recogniser declares what it eats rather than taking any bytes at all and finding out. Bytes still never travel as a message and nothing on the wire stops being JSON: a camera publishes one reference per frame, a microphone one per chunk, and a reference may carry a `meta` dict nothing here interprets. Streaming media is therefore an ordinary streaming port — with one change to what that means. An emission used to journal an item with no payload, so downstream read whatever was current when the item was claimed; a consumer slower than its producer saw only the newest chunk and the ones between were lost. That is right for a training curve and wrong for a second of speech, so an emission now journals a `kind="emission"` item carrying its values, and the executor hands them to the nodes reading that message instead of writing them to state again. The value in state stays the latest, which is what everything else reads, and the wave is filtered by what actually changed rather than walking everything reachable. No queue serialization change — the existing `outputs` field carries it. Continuous media makes the store's missing GC a real problem, so this closes it: `sweep_artifacts` runs hourly, keeps every digest a `run_artifact` row records or a live message holds, spares anything written in the last hour, and stands aside entirely while a run is in flight, since a node may store a checkpoint long before it returns the reference to it. That also collects the orphans a deleted flow has always left behind. `ARTIFACT_GC_INTERVAL_S=0` turns it off. Around the edges: `GET /artifacts/{digest}` serves the media type the caller passes and answers ranged requests, so a browser plays a clip rather than downloading it; `PUT` spools to disk instead of holding the whole body in memory, as does `save_artifact` given a path; a Media widget draws whatever its message points at, and a wall panel may fetch the bytes its own tiles are showing and nothing else; and a connector gets `save_artifact`, for a device whose readings are bytes. What this cannot do is live video: a frame every second or two is a glance, and the honest answer above that is the camera's own stream, which the widget takes as a URL and the browser plays from source. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 23:44:55 +02:00
stroblmeandClaude Opus 5	4a38c6ed31	Name the code a run ran, and let an interrupted sync finish ... Three faults with one root: the stored body of a code-defined node is an import shim, and nothing that mattered was ever read from the code itself. - The run stamp could not identify what ran. The shim imports whatever is on disk when the worker starts, and an uncommitted tree stamps <commit>-dirty for every run it ever produces. Run.code_digest hashes the repository's .py files, memoized on their stat state, and it is read again when the run is actually claimed -- so a sweep queued for hours records the code each of its runs executed, not the code that was there when it was submitted. - The stage cache adopted code that was too new. The fingerprint hashed the shim, which is invariant under any edit to the imported function or anything it calls into, so a re-run was served from cache and answered without the outputs the edit added. It now carries the repo digest and the node's declared ports. Every fingerprint changes once, which invalidates the existing cache; a canvas flow has no repository and keys as before. - An interrupted sync looked like a hand-edited canvas. The engine answers a new-node template for a node with no stored body, and the template carries no marker, so the drift check read "somebody edited this" and demanded --force -- for the one state that re-running the sync is the fix for. NodeSource.missing states the fact, and sync skips those and reuses the bodies it read instead of asking for each one twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 21:27:10 +02:00
stroblmeandClaude Opus 5	266acd1661	Document the cascade guarantee and the cascade ceiling ... A node runs when something it reads was published, not merely when something upstream of it ran — worth saying now that it is true, since it is what a reader relies on when a rate-limited port goes quiet. Plus `FLOW_MAX_CASCADES` in the engine's configuration table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf	2026-08-26 10:27:23 +02:00
stroblmeandClaude Opus 5	a3a234756c	A batch flow's input is a run parameter, not a missing value ... Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UytviPMJbXzD8P84nLvXcq	2026-08-25 18:51:54 +02:00
stroblme	4350916bd8	Scroll the runs table, pick a range, and choose what a comparison plots against	2026-08-25 16:32:34 +02:00
stroblmeandClaude Opus 5	3c15964364	A cached node keeps its curve, and any input can name a run's output ... A cache hit still replays no emissions — those values were the story of an execution that is not happening — but the run they were recorded in is now written on the row (`run_node.cached_from`), and the metrics endpoints read the series back from there. So a reused run answers `run.metrics("train.loss")` with the same points the run that trained did, rather than looking like a run that produced no numbers at all. Pointed at rather than copied: a sweep of 500 reusing one frozen node would otherwise duplicate its curve 500 times. That needed the cross-flow restore fixed first. The cache key has no flow in it while the stored outputs are named for the flow that produced them, so `quick.prepare` getting a hit from `train` wrote `train.dataset` into `quick`'s state and the next node was called without its argument. One rule now covers both halves: `requalify` reads a name owned by one flow as the same name in another, applied to the restored outputs, to the node id behind the pointer, and to the series names on the way out. Reuse across flows is kept. Also: `@run:<id>.<output>` and a bare `sha256:` digest resolve on every input, not only artifacts. Chaining a run's json config into the next one from a shell meant pasting the whole object inline, and the CLI could not even send the spelling — `_coerce` died in `json.loads` before the engine saw it. Both spellings are reserved on every input now, `str` included, and `_from_run` returns whatever the run's result holds rather than only a reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dp9L6gakMVro1K2C5zdtBE	2026-08-25 15:12:28 +02:00
stroblme	03e12b7647	Document the runs screen, the run context and the generated dashboard	2026-08-25 12:44:19 +02:00
stroblmeandClaude Opus 5	93374a310e	Refuse what a node cannot publish, and stop timing out work that is fine ... Four things the python SDK turned up, each fixed where every client sees it. A key no port declares is now an error rather than a silent drop, on the return, the yield and the emit alike — the contract the docs already stated. The SDK reads literal yields at sync time, so a typo fails before anything runs, and an emission of one fails the call rather than being logged where nobody looks. NaN and infinity are refused at the port. JSON cannot spell either, so one that travelled came back as a 500, a socket frame that stopped the canvas, or a metric batch the database dropped whole. An artifact input takes `@run:<id>.<output>` or a bare digest, resolved on the engine — so the CLI, the run dialog and a python caller mean the same thing, and a sweep can pass one at all. Node timeouts are off by default. The clock measured silence, which a training node is full of, and remote workers had already stopped enforcing it — their heartbeat reset it. Now a heartbeat proves the agent rather than the node, ninety seconds of nothing fails the call either way, and the engine touches work it is still running so a long node is not redelivered at sixty seconds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69	2026-08-25 07:30:14 +02:00
stroblmeandClaude Opus 5	400d7d9c5c	Stage caching for batch runs, and an engine that lives in the command ... A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-24 20:31:31 +02:00
stroblmeandClaude Opus 5	11e032386b	Publish the documentation site: docs.fluksio.com ... A zensical site under docs/, served by a new `docs` compose service behind Traefik, built with --strict in CI. Same pattern the sibling n3xd workspace uses. Getting started splits the way the landing page does — one path is `pip install fluksio` and a training script, the other is a Docker stack and an afternoon in the browser — because the two audiences will not spend the same amount of time. Everything after that is shared: the concepts, the web interface (app and portal), the CLI and the API, and a reference for node types, payload types and configuration. The three flow guides move here from the docs submodule rather than being copied, so there is one version of them. Styling mirrors DESIGN-GUIDELINES.md: the app's token palette remapped onto Material's variables in both schemes, Inter, the 16px panel radius, and the one terracotta accent spent on the facility lane of the audience split. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M7Xv3cJEW5c8AXxn2hoojV	2026-08-22 05:55:34 +02:00