0578a30a3a46a7eda5fcf423c1df7f12c4a449cf
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  stroblmeandClaude Opus 5
|
3c15964364 |
A cached node keeps its curve, and any input can name a run's output
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m42s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m41s
pre-commit / pre-commit (push) Failing after 2m53s
Test Backend / test-backend (push) Successful in 2m21s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Failing after 1m6s
A cache hit still replays no emissions — those values were the story of an
execution that is not happening — but the run they were recorded in is now
written on the row (`run_node.cached_from`), and the metrics endpoints read the
series back from there. So a reused run answers `run.metrics("train.loss")`
with the same points the run that trained did, rather than looking like a run
that produced no numbers at all. Pointed at rather than copied: a sweep of 500
reusing one frozen node would otherwise duplicate its curve 500 times.
That needed the cross-flow restore fixed first. The cache key has no flow in
it while the stored outputs are named for the flow that produced them, so
`quick.prepare` getting a hit from `train` wrote `train.dataset` into `quick`'s
state and the next node was called without its argument. One rule now covers
both halves: `requalify` reads a name owned by one flow as the same name in
another, applied to the restored outputs, to the node id behind the pointer,
and to the series names on the way out. Reuse across flows is kept.
Also: `@run:<id>.<output>` and a bare `sha256:` digest resolve on every input,
not only artifacts. Chaining a run's json config into the next one from a shell
meant pasting the whole object inline, and the CLI could not even send the
spelling — `_coerce` died in `json.loads` before the engine saw it. Both
spellings are reserved on every input now, `str` included, and `_from_run`
returns whatever the run's result holds rather than only a reference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp9L6gakMVro1K2C5zdtBE
|
||
|
stroblmeandClaude Opus 5
|
60757fa7fa |
Style the engine's own logs, notice enrolment while serving, say more in status
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 4m24s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m37s
pre-commit / pre-commit (push) Failing after 3m14s
Test Backend / test-backend (push) Successful in 2m15s
Compose Smoke Test / test-compose (push) Successful in 34s
Playwright Tests / merge-reports (push) Failing after 1m3s
Four things from a testing pass. `fluksio serve` printed its own lines through the root logger, which has no handler and falls back to `INFO:fluksio.cloud.connector:...` — beside uvicorn's aligned output it reads like something went wrong. The engine's loggers and alembic's now use uvicorn's own handler. Named rather than configuring the root: httpx logs every portal call at INFO and none of that is printed today. `fluksio enroll` writes its config from another process, so an engine already serving never learned it had been paired. It now looks for one every few seconds and dials when it appears. `load()` rather than `exists()`, or a file that does not parse would be restarted forever. `fluksio status` says where the installation stands with its portal — never paired, linked, or paired and unreachable, which is the one worth acting on. `--seed` and `--timeout` had no help text at all. Both say what they are for now, and the docs say what a seed is actually for: recorded on the run, part of its input digest, and passed to an input named `seed` when the flow declares one, so the number a run is labelled with is the one the code drew from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69 |
||
|
stroblmeandClaude Opus 5
|
7c5b212f43 |
Let fluksio status read a local installation
`runs --local` already exists for the reason status needs it: the flows and the history are in this directory, and looking at them should not need a server. Watching still does — an in-process engine is the command itself, so nothing could change under it — and says so rather than sitting on a still frame. Found setting up a sandbox that drives the SDK entirely through --local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69 |
||
|
stroblmeandClaude Opus 5
|
4941c2787c |
Add fluksio status, and ask for a run's parameters at a terminal
Two halves of the same gap: the CLI could start work but not show you any. `fluksio status` draws the home screen's top half in a terminal — health and what is wrong with it, every flow with its state and node count, and the recent runs and failures under them. `--watch` keeps it there. Rich does the drawing; it was already installed under fastapi's own CLI, and is named now because a command depends on it. `fluksio run` with no parameters at a terminal asks for them, one line per declared input with its declared value in brackets — so Enter through the lot is what running the defaults looks like, and an artifact input takes the `@run:` spelling the engine now resolves. A scripted run is untouched: passing any parameter, or piping the command, skips the questions, as does --defaults. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69 |
||
|
stroblmeandClaude Opus 5
|
93374a310e |
Refuse what a node cannot publish, and stop timing out work that is fine
Four things the python SDK turned up, each fixed where every client sees it. A key no port declares is now an error rather than a silent drop, on the return, the yield and the emit alike — the contract the docs already stated. The SDK reads literal yields at sync time, so a typo fails before anything runs, and an emission of one fails the call rather than being logged where nobody looks. NaN and infinity are refused at the port. JSON cannot spell either, so one that travelled came back as a 500, a socket frame that stopped the canvas, or a metric batch the database dropped whole. An artifact input takes `@run:<id>.<output>` or a bare digest, resolved on the engine — so the CLI, the run dialog and a python caller mean the same thing, and a sweep can pass one at all. Node timeouts are off by default. The clock measured silence, which a training node is full of, and remote workers had already stopped enforcing it — their heartbeat reset it. Now a heartbeat proves the agent rather than the node, ninety seconds of nothing fails the call either way, and the engine touches work it is still running so a long node is not redelivered at sixty seconds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69 |
||
|
stroblmeandClaude Opus 5
|
400d7d9c5c |
Stage caching for batch runs, and an engine that lives in the command
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Fable 5
|
7e4f03369b |
Three fewer things to remember
**The portal link puts itself back up.** It already retried a connection that raised, but a session that ended *cleanly* — a portal restarting, a proxy closing an idle socket — returned normally and went straight back round the loop with no wait at all, so an engine could spin against a portal that was merely saying goodbye politely. Every ending now reconnects on a delay, and the delay turns on whether the attempt got as far as attaching: one that stood up and dropped is a network event and retries at once, one that never stood up waits longer each time. Jittered, so a portal coming back is not met by every installation it serves in the same instant. Ping timeouts are named rather than defaulted, since they are what bounds how long a suspended laptop's dead socket looks alive, and the keepalive task is awaited so the reason a link went reaches the log instead of the garbage collector. **`fluksio enroll <code>`** is the whole command now; hub.fluksio.com is the default and `--portal` names another. The one command run before anything works should not need two flags. **`fluksio run` syncs first.** The reason a run exists is usually the edit before it, so remembering to sync was remembering to do something the computer could do — including the worker refresh, which is what makes an edit to your own package take effect at all. `--no-sync` opts out for a tight loop. That last one needed discovery fixed: it only ever looked at top-level `*.py`, so a repository whose code is in a package — the ordinary shape — found nothing from its own root. It now descends into the packages it holds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Fable 5
|
99f6530698 |
One installation per project, and no login to reach it
Two things a local install should not have asked for. `fluksio serve` now signs you in. Logging in to your own machine was a formality — the password was printed by the same process that would have checked it, and the database it authenticates against sits in the directory the token goes into — so `serve` mints the token itself and says where it put it. `fluksio login` is left for an engine somewhere else. And an installation is `.fluksio` beside the code, found the way `.git` is, rather than one `~/.fluksio` for the machine. A repository with its own venv was already getting its own engine; it now gets its own flows, run history and token too, instead of three repositories sharing one database and fighting over one port. `--global` asks for the shared one, `--data-dir` still names any directory, and when both exist the banner says which you are looking at and how to reach the other. The directory ignores itself from within — a `.gitignore` of `*`, the way uv writes one into `.venv` — because it holds a credential and a database, and neither belongs in anybody's history. The token is written mode 600. A login an older version wrote to ~/.config/fluksio is still read, so nothing that worked stops working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Opus 5
|
d4a9406c51 |
Fix the CI gates: Python 3.13, concurrency groups, hook violations
Docs / docs (push) Successful in 49s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m11s
Playwright Tests / test-playwright (2, 2) (push) Failing after 23s
pre-commit / pre-commit (push) Successful in 3m2s
Test Backend / test-backend (push) Successful in 2m22s
Compose Smoke Test / test-compose (push) Failing after 22s
Playwright Tests / merge-reports (push) Canceled after 1s
The gates have never gone green on the new runners. Three separate reasons: - backend/Dockerfile shipped Python 3.10 while the code imports typing.Self and datetime.UTC, so the container exited on import and the suite could not even load its conftest. The image moves to 3.13 and the packages declare >=3.12, which is the floor the tests actually pass on; ruff's target follows and rewrites timezone.utc and asyncio.TimeoutError accordingly. Relocking drops the 3.10 branch, which bumps FastAPI and so regenerates the SDK. - frontend/README.md had no trailing newline and two dashboard widgets used arbitrary text-[…] sizes. Both are em-relative on purpose, so they move to the inline style the neighbouring ramp already uses. - Every commit left its own run queued: without a concurrency group a runner that was offline for a while works through a backlog nobody reads. A stack that fails to come up now prints its logs before the teardown removes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Fable 5
|
fea57064f9 |
Run node code on the venv Fluksio was installed into
The workflow this serves: make a venv, install what you work with, then `pip install fluksio` into the same one. Building a second environment beside it was exactly wrong — the packages the nodes need are already here, and the Modules screen was asking for them a second time. `NODE_VENV=auto` (the default) adopts that venv. It declines in the three cases where adopting would be wrong: `managed` says otherwise, a managed venv already exists and may hold packages somebody installed on purpose, or the engine is not running from a venv at all. The images set `managed`, since the venv in them holds the app and nothing of anybody else's. An adopted venv is never written to. `uv pip sync` makes a venv hold exactly the manifest, so pointed at somebody's own environment it uninstalls their work and the engine with it — `sync()` refuses outright and `reconcile()` returns before it can be called at startup, which is where that would have happened first. The Modules screen lists what is installed and drops its editor; `pip` is how that environment changes. `fluksio serve` now names the interpreter node code runs on, which is the thing a data scientist most needs to know at that moment. `fluksio-worker` already defaulted `--python` to its own interpreter, so a GPU box works the same way — that was only ever undocumented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Fable 5
|
a38e2745eb |
Add a Python SDK: flows declared in your own repository
A data scientist keeps their code where it is and decorates it: `@node` declares a function's ports beside the function, `Flow(name, nodes=[...])` says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds one for a single flow. `fluksio sync` uploads the document plus a generated import shim per node, so the store still holds a complete, runnable, git-versioned definition while the code it imports stays theirs. `fluksio login|run|runs` and `flow.submit().wait()` are the client half, over the run endpoints that already existed. Runs record the user repository's commit beside the store's, so "what code produced this number" is answerable on the side that now holds the code. - `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim generator, an HTTP client and sync. Standard library only at import, so `from fluksio import node` in a training script pulls in no engine. - `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the repository's commit; `POST /modules/refresh` retires the workers without an install, which every sync calls — a worker holds the imported package in memory, so an edit to it is invisible until the process goes. - The canvas shows a generated body read-only and names the repository to edit instead; a body edited there stops the next sync rather than being discarded. - The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since the shim imports a module whose first line declares them. - `examples/myresearch` is the worked example, `make sync-example` uploads it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Opus 5
|
11e032386b |
Publish the documentation site: docs.fluksio.com
A zensical site under docs/, served by a new `docs` compose service behind Traefik, built with --strict in CI. Same pattern the sibling n3xd workspace uses. Getting started splits the way the landing page does — one path is `pip install fluksio` and a training script, the other is a Docker stack and an afternoon in the browser — because the two audiences will not spend the same amount of time. Everything after that is shared: the concepts, the web interface (app and portal), the CLI and the API, and a reference for node types, payload types and configuration. The three flow guides move here from the docs submodule rather than being copied, so there is one version of them. Styling mirrors DESIGN-GUIDELINES.md: the app's token palette remapped onto Material's variables in both schemes, Inter, the 16px panel radius, and the one terracotta accent spent on the facility lane of the audience split. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M7Xv3cJEW5c8AXxn2hoojV |