16 Commits

Include renames

Author	SHA1	Message	Date
stroblmeandClaude Opus 5	37a7df9d24	Let a sweep run more than four at a time, and name the run a failure was in ... Concurrent runs sat at 4 whatever FLOW_MAX_CASCADES said: that setting bounds cascades, and the run drivers read a hardcoded MAX_PARALLEL nobody could reach. FLOW_MAX_RUNS is the knob they read now, --max-runs/--max-cascades/--max-workers are the same three as flags on serve, and the engine says which numbers it started with — which is the only way to tell that a settings file was read. Events keep the run they happened in. The payload always carried it and the persist path dropped it, so reading one run's failures meant filtering the engine-wide list; a batch run's id reaches those events now too, since a run has no journaled item to name itself by. Also: a provisioner's 0 means "no deadline" rather than "cancel on the next reconcile", and a command that reaches no engine says how to start one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015sbYeYaVgYQqm1sbx7wPdL	2026-08-27 14:17:51 +02:00
stroblmeandClaude Opus 5	a82f88cf0a	Name the sizes a node can ask for ... Raw cpus and gpus are a property of the machines an installation has, so a node written against a cluster quietly stops meaning anything when the cluster is replaced. A node says "gpu-small" instead, and what that is stored here — editable, and read again every time the node is built, so changing the flavor changes what the next run gets. Memory joins the schema properly (`ram`, in MB, accepting "2G"), along with `duration_s` for how long a node is expected to take. That one is recorded and shown and nothing else yet: a statement for whoever is planning around the node, not a limit — the limit is still `timeout`. A flavor and a number for the same thing is refused, compared by value so an editor writing the whole object back with its defaults still round-trips. A name nothing stores is refused at the save, which covers the canvas and `fluksio sync` at once, and deleting one a node still asks for says which node. Four sizes are seeded on an installation that has none, and never re-seeded: re-adding one somebody deliberately removed is an argument nobody wins. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW	2026-08-27 08:59:10 +02:00
stroblmeandClaude Opus 5	0ffcabfdb9	Media dtypes: image, audio and video as narrowed artifact references ... A port may now declare `image`, `audio` or `video`. Each is the artifact reference the engine already had, narrowed by the `media_type` on it, so a speech recogniser declares what it eats rather than taking any bytes at all and finding out. Bytes still never travel as a message and nothing on the wire stops being JSON: a camera publishes one reference per frame, a microphone one per chunk, and a reference may carry a `meta` dict nothing here interprets. Streaming media is therefore an ordinary streaming port — with one change to what that means. An emission used to journal an item with no payload, so downstream read whatever was current when the item was claimed; a consumer slower than its producer saw only the newest chunk and the ones between were lost. That is right for a training curve and wrong for a second of speech, so an emission now journals a `kind="emission"` item carrying its values, and the executor hands them to the nodes reading that message instead of writing them to state again. The value in state stays the latest, which is what everything else reads, and the wave is filtered by what actually changed rather than walking everything reachable. No queue serialization change — the existing `outputs` field carries it. Continuous media makes the store's missing GC a real problem, so this closes it: `sweep_artifacts` runs hourly, keeps every digest a `run_artifact` row records or a live message holds, spares anything written in the last hour, and stands aside entirely while a run is in flight, since a node may store a checkpoint long before it returns the reference to it. That also collects the orphans a deleted flow has always left behind. `ARTIFACT_GC_INTERVAL_S=0` turns it off. Around the edges: `GET /artifacts/{digest}` serves the media type the caller passes and answers ranged requests, so a browser plays a clip rather than downloading it; `PUT` spools to disk instead of holding the whole body in memory, as does `save_artifact` given a path; a Media widget draws whatever its message points at, and a wall panel may fetch the bytes its own tiles are showing and nothing else; and a connector gets `save_artifact`, for a device whose readings are bytes. What this cannot do is live video: a frame every second or two is a glance, and the honest answer above that is the camera's own stream, which the widget takes as a URL and the browser plays from source. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 23:44:55 +02:00
stroblmeandClaude Opus 5	608d30d884	Let a node say how much of the machine it takes ... Five concurrent training nodes, each sizing its thread pool to every core, left the engine's own event loop unscheduled: the API stopped answering within 10 s and every client died. The same shape on a GPU deadlocked a run for 21 minutes at 0% utilisation with nothing failing and nothing to read -- it just sat in `running`. @node(resources={"cpus": 2}) is the declaration. The engine holds that much for the length of the execution, so more of them than the machine has room for wait their turn rather than oversubscribing it, and a `gpus` node holds its card exclusively. FLOW_CPUS defaults to every core but two, and those two are what keeps the engine answering. Because a thread cap is read when the process imports the library, a warm worker cannot be told a different one -- so an environment gets a pool of its own and nodes deriving the same one share it, rather than paying a cold start per call on exactly the nodes whose imports are slowest. XLA_FLAGS is never derived: it is a composed, version-dependent string, so it travels in resources.env where it is visible. A node that declares nothing is not accounted for and behaves as it always did -- it just gets FLOW_CPUS/FLOW_MAX_WORKERS as a thread cap, which is the half of this that fixes the reported incident without anybody declaring anything. An operator who set OMP_NUM_THREADS themselves still wins. Resources are claimed strictly before a worker slot, so the two blocking waits cannot deadlock. A node queued for them publishes node_queued and shows on GET /workers/resources, because waiting and hanging looked identical. Accounted, not enforced: no cgroups, no rlimits. Scheduling across machines, flavours and enforcement are the next steps. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 21:36:59 +02:00
stroblmeandClaude Opus 5	4a38c6ed31	Name the code a run ran, and let an interrupted sync finish ... Three faults with one root: the stored body of a code-defined node is an import shim, and nothing that mattered was ever read from the code itself. - The run stamp could not identify what ran. The shim imports whatever is on disk when the worker starts, and an uncommitted tree stamps <commit>-dirty for every run it ever produces. Run.code_digest hashes the repository's .py files, memoized on their stat state, and it is read again when the run is actually claimed -- so a sweep queued for hours records the code each of its runs executed, not the code that was there when it was submitted. - The stage cache adopted code that was too new. The fingerprint hashed the shim, which is invariant under any edit to the imported function or anything it calls into, so a re-run was served from cache and answered without the outputs the edit added. It now carries the repo digest and the node's declared ports. Every fingerprint changes once, which invalidates the existing cache; a canvas flow has no repository and keys as before. - An interrupted sync looked like a hand-edited canvas. The engine answers a new-node template for a node with no stored body, and the template carries no marker, so the drift check read "somebody edited this" and demanded --force -- for the one state that re-running the sync is the fix for. NodeSource.missing states the fact, and sync skips those and reuses the bodies it read instead of asking for each one twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 21:27:10 +02:00
stroblmeandClaude Opus 5	1f7c6646f1	Survive a busy engine: retry, idempotent submit, resilient waiting ... A driver script died of one slow answer: httpx.ReadTimeout out of RunHandle.refresh() with a 30 s read timeout and no retry anywhere, which cost a sweep 78 of its 84 runs. - Split the timeout (5 s connect, 120 s read): a wrong URL fails at once, and a busy engine gets longer than the slowest thing it does on purpose (a 60 s compile, a 15 s rebuild wait). - Retry idempotent calls three times on a transport error or 502/503/504. 503 is the engine's own "ask again" — it is what RebuildBusy answers. - Submit carries a key the engine stores with the run, so a retry after a timeout returns that run instead of starting a second. A sweep keys every entry, so a half-created one recreates only what is missing. - wait() and --follow tolerate five failed polls in a row; a 404 still stops at once, because that is an answer rather than a gap. - CLI says "engine not answering" and names the run still on the engine, instead of printing a traceback. - runs: clamp the params column to 80 characters; events() takes the flow/since/until the endpoint already had; RunHandle.failures answers "what killed this run" from the run's own node rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 21:19:09 +02:00
stroblmeandClaude Opus 5	1148e54c9e	A run says whether the dashboard, the CLI or a script asked for it ... `POST /runs/flows/{name}` hardcoded `cause: "api"`, so every row in the history claimed the same origin. The body now carries an optional `cause`, closed to the values the column knows — the dashboard sends nothing and stays "api", `fluksio run` says "cli", and the SDK client says "sdk". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK	2026-08-25 22:06:29 +02:00
stroblmeandClaude Opus 5	3c15964364	A cached node keeps its curve, and any input can name a run's output ... A cache hit still replays no emissions — those values were the story of an execution that is not happening — but the run they were recorded in is now written on the row (`run_node.cached_from`), and the metrics endpoints read the series back from there. So a reused run answers `run.metrics("train.loss")` with the same points the run that trained did, rather than looking like a run that produced no numbers at all. Pointed at rather than copied: a sweep of 500 reusing one frozen node would otherwise duplicate its curve 500 times. That needed the cross-flow restore fixed first. The cache key has no flow in it while the stored outputs are named for the flow that produced them, so `quick.prepare` getting a hit from `train` wrote `train.dataset` into `quick`'s state and the next node was called without its argument. One rule now covers both halves: `requalify` reads a name owned by one flow as the same name in another, applied to the restored outputs, to the node id behind the pointer, and to the series names on the way out. Reuse across flows is kept. Also: `@run:<id>.<output>` and a bare `sha256:` digest resolve on every input, not only artifacts. Chaining a run's json config into the next one from a shell meant pasting the whole object inline, and the CLI could not even send the spelling — `_coerce` died in `json.loads` before the engine saw it. Both spellings are reserved on every input now, `str` included, and `_from_run` returns whatever the run's result holds rather than only a reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dp9L6gakMVro1K2C5zdtBE	2026-08-25 15:12:28 +02:00
stroblmeandClaude Opus 5	60757fa7fa	Style the engine's own logs, notice enrolment while serving, say more in status ... Four things from a testing pass. `fluksio serve` printed its own lines through the root logger, which has no handler and falls back to `INFO:fluksio.cloud.connector:...` — beside uvicorn's aligned output it reads like something went wrong. The engine's loggers and alembic's now use uvicorn's own handler. Named rather than configuring the root: httpx logs every portal call at INFO and none of that is printed today. `fluksio enroll` writes its config from another process, so an engine already serving never learned it had been paired. It now looks for one every few seconds and dials when it appears. `load()` rather than `exists()`, or a file that does not parse would be restarted forever. `fluksio status` says where the installation stands with its portal — never paired, linked, or paired and unreachable, which is the one worth acting on. `--seed` and `--timeout` had no help text at all. Both say what they are for now, and the docs say what a seed is actually for: recorded on the run, part of its input digest, and passed to an input named `seed` when the flow declares one, so the number a run is labelled with is the one the code drew from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69	2026-08-25 09:29:22 +02:00
stroblmeandClaude Opus 5	7c5b212f43	Let fluksio status read a local installation ... `runs --local` already exists for the reason status needs it: the flows and the history are in this directory, and looking at them should not need a server. Watching still does — an in-process engine is the command itself, so nothing could change under it — and says so rather than sitting on a still frame. Found setting up a sandbox that drives the SDK entirely through --local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69	2026-08-25 08:21:27 +02:00
stroblmeandClaude Opus 5	4941c2787c	Add fluksio status, and ask for a run's parameters at a terminal ... Two halves of the same gap: the CLI could start work but not show you any. `fluksio status` draws the home screen's top half in a terminal — health and what is wrong with it, every flow with its state and node count, and the recent runs and failures under them. `--watch` keeps it there. Rich does the drawing; it was already installed under fastapi's own CLI, and is named now because a command depends on it. `fluksio run` with no parameters at a terminal asks for them, one line per declared input with its declared value in brackets — so Enter through the lot is what running the defaults looks like, and an artifact input takes the `@run:` spelling the engine now resolves. A scripted run is untouched: passing any parameter, or piping the command, skips the questions, as does --defaults. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69	2026-08-25 07:42:09 +02:00
stroblmeandClaude Opus 5	93374a310e	Refuse what a node cannot publish, and stop timing out work that is fine ... Four things the python SDK turned up, each fixed where every client sees it. A key no port declares is now an error rather than a silent drop, on the return, the yield and the emit alike — the contract the docs already stated. The SDK reads literal yields at sync time, so a typo fails before anything runs, and an emission of one fails the call rather than being logged where nobody looks. NaN and infinity are refused at the port. JSON cannot spell either, so one that travelled came back as a 500, a socket frame that stopped the canvas, or a metric batch the database dropped whole. An artifact input takes `@run:<id>.<output>` or a bare digest, resolved on the engine — so the CLI, the run dialog and a python caller mean the same thing, and a sweep can pass one at all. Node timeouts are off by default. The clock measured silence, which a training node is full of, and remote workers had already stopped enforcing it — their heartbeat reset it. Now a heartbeat proves the agent rather than the node, ninety seconds of nothing fails the call either way, and the engine touches work it is still running so a long node is not redelivered at sixty seconds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69	2026-08-25 07:30:14 +02:00
stroblmeandClaude Opus 5	400d7d9c5c	Stage caching for batch runs, and an engine that lives in the command ... A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-24 20:31:31 +02:00
stroblmeandClaude Fable 5	7e4f03369b	Three fewer things to remember ... **The portal link puts itself back up.** It already retried a connection that raised, but a session that ended *cleanly* — a portal restarting, a proxy closing an idle socket — returned normally and went straight back round the loop with no wait at all, so an engine could spin against a portal that was merely saying goodbye politely. Every ending now reconnects on a delay, and the delay turns on whether the attempt got as far as attaching: one that stood up and dropped is a network event and retries at once, one that never stood up waits longer each time. Jittered, so a portal coming back is not met by every installation it serves in the same instant. Ping timeouts are named rather than defaulted, since they are what bounds how long a suspended laptop's dead socket looks alive, and the keepalive task is awaited so the reason a link went reaches the log instead of the garbage collector. **`fluksio enroll <code>`** is the whole command now; hub.fluksio.com is the default and `--portal` names another. The one command run before anything works should not need two flags. **`fluksio run` syncs first.** The reason a run exists is usually the edit before it, so remembering to sync was remembering to do something the computer could do — including the worker refresh, which is what makes an edit to your own package take effect at all. `--no-sync` opts out for a tight loop. That last one needed discovery fixed: it only ever looked at top-level `*.py`, so a repository whose code is in a package — the ordinary shape — found nothing from its own root. It now descends into the packages it holds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU	2026-08-24 18:17:16 +02:00
stroblmeandClaude Fable 5	99f6530698	One installation per project, and no login to reach it ... Two things a local install should not have asked for. `fluksio serve` now signs you in. Logging in to your own machine was a formality — the password was printed by the same process that would have checked it, and the database it authenticates against sits in the directory the token goes into — so `serve` mints the token itself and says where it put it. `fluksio login` is left for an engine somewhere else. And an installation is `.fluksio` beside the code, found the way `.git` is, rather than one `~/.fluksio` for the machine. A repository with its own venv was already getting its own engine; it now gets its own flows, run history and token too, instead of three repositories sharing one database and fighting over one port. `--global` asks for the shared one, `--data-dir` still names any directory, and when both exist the banner says which you are looking at and how to reach the other. The directory ignores itself from within — a `.gitignore` of `*`, the way uv writes one into `.venv` — because it holds a credential and a database, and neither belongs in anybody's history. The token is written mode 600. A login an older version wrote to ~/.config/fluksio is still read, so nothing that worked stops working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU	2026-08-24 16:13:35 +02:00
stroblmeandClaude Fable 5	a38e2745eb	Add a Python SDK: flows declared in your own repository ... A data scientist keeps their code where it is and decorates it: `@node` declares a function's ports beside the function, `Flow(name, nodes=[...])` says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds one for a single flow. `fluksio sync` uploads the document plus a generated import shim per node, so the store still holds a complete, runnable, git-versioned definition while the code it imports stays theirs. `fluksio login|run|runs` and `flow.submit().wait()` are the client half, over the run endpoints that already existed. Runs record the user repository's commit beside the store's, so "what code produced this number" is answerable on the side that now holds the code. - `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim generator, an HTTP client and sync. Standard library only at import, so `from fluksio import node` in a training script pulls in no engine. - `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the repository's commit; `POST /modules/refresh` retires the workers without an install, which every sync calls — a worker holds the imported package in memory, so an edit to it is invisible until the process goes. - The canvas shows a generated body read-only and names the repository to edit instead; a body edited there stops the next sync rather than being discarded. - The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since the shim imports a module whose first line declares them. - `examples/myresearch` is the worked example, `make sync-example` uploads it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU	2026-08-23 20:16:08 +02:00