A worker reported its labels and nothing about the machine behind them, so the
engine could route a node to a GPU box but not tell whether that box had a GPU
free. Inventory — cores, GPUs, memory — now arrives with the hello frame, and
the run frame carries back what the engine allocated for that call.
Which is protocol 2 on both ends. GPUs are never probed: asking a vendor tool
would make the one dependency two, so a GPU is what the batch job says it was
given or what --gpus says. A worker that reports nothing still attaches and is
scheduled by its label alone.
Two things a job scheduler needs: --max-idle stops a worker started for one job
rather than letting it hold its allocation to the walltime, and a refusal is now
fatal instead of a reconnect loop that reads as a hang in a job's log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
A port may now declare `image`, `audio` or `video`. Each is the artifact
reference the engine already had, narrowed by the `media_type` on it, so a
speech recogniser declares what it eats rather than taking any bytes at all and
finding out. Bytes still never travel as a message and nothing on the wire
stops being JSON: a camera publishes one reference per frame, a microphone one
per chunk, and a reference may carry a `meta` dict nothing here interprets.
Streaming media is therefore an ordinary streaming port — with one change to
what that means. An emission used to journal an item with no payload, so
downstream read whatever was current when the item was claimed; a consumer
slower than its producer saw only the newest chunk and the ones between were
lost. That is right for a training curve and wrong for a second of speech, so
an emission now journals a `kind="emission"` item carrying its values, and the
executor hands them to the nodes reading that message instead of writing them
to state again. The value in state stays the latest, which is what everything
else reads, and the wave is filtered by what actually changed rather than
walking everything reachable. No queue serialization change — the existing
`outputs` field carries it.
Continuous media makes the store's missing GC a real problem, so this closes
it: `sweep_artifacts` runs hourly, keeps every digest a `run_artifact` row
records or a live message holds, spares anything written in the last hour, and
stands aside entirely while a run is in flight, since a node may store a
checkpoint long before it returns the reference to it. That also collects the
orphans a deleted flow has always left behind. `ARTIFACT_GC_INTERVAL_S=0` turns
it off.
Around the edges: `GET /artifacts/{digest}` serves the media type the caller
passes and answers ranged requests, so a browser plays a clip rather than
downloading it; `PUT` spools to disk instead of holding the whole body in
memory, as does `save_artifact` given a path; a Media widget draws whatever its
message points at, and a wall panel may fetch the bytes its own tiles are
showing and nothing else; and a connector gets `save_artifact`, for a device
whose readings are bytes.
What this cannot do is live video: a frame every second or two is a glance, and
the honest answer above that is the camera's own stream, which the widget takes
as a URL and the browser plays from source.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The worker held each yield one behind, because the last one is the node's
result when the generator returns nothing of its own. Only the engine knows
what ports a node declared, so the check happened when the *next* yield
arrived — a pass late, which for a training loop is however long one epoch
takes.
The worker now sends every yield as it happens and returns whatever its
generator returned; EmitSink holds the last one back and decides at the end
of the call what it was. Old "emit" frames are still handled, so a remote
agent that has not been restarted keeps working.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A driver script died of one slow answer: httpx.ReadTimeout out of
RunHandle.refresh() with a 30 s read timeout and no retry anywhere, which
cost a sweep 78 of its 84 runs.
- Split the timeout (5 s connect, 120 s read): a wrong URL fails at once,
and a busy engine gets longer than the slowest thing it does on purpose
(a 60 s compile, a 15 s rebuild wait).
- Retry idempotent calls three times on a transport error or 502/503/504.
503 is the engine's own "ask again" — it is what RebuildBusy answers.
- Submit carries a key the engine stores with the run, so a retry after a
timeout returns that run instead of starting a second. A sweep keys every
entry, so a half-created one recreates only what is missing.
- wait() and --follow tolerate five failed polls in a row; a 404 still stops
at once, because that is an answer rather than a gap.
- CLI says "engine not answering" and names the run still on the engine,
instead of printing a traceback.
- runs: clamp the params column to 80 characters; events() takes the
flow/since/until the endpoint already had; RunHandle.failures answers
"what killed this run" from the run's own node rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four things the python SDK turned up, each fixed where every client sees it.
A key no port declares is now an error rather than a silent drop, on the
return, the yield and the emit alike — the contract the docs already stated.
The SDK reads literal yields at sync time, so a typo fails before anything
runs, and an emission of one fails the call rather than being logged where
nobody looks.
NaN and infinity are refused at the port. JSON cannot spell either, so one
that travelled came back as a 500, a socket frame that stopped the canvas, or
a metric batch the database dropped whole.
An artifact input takes `@run:<id>.<output>` or a bare digest, resolved on the
engine — so the CLI, the run dialog and a python caller mean the same thing,
and a sweep can pass one at all.
Node timeouts are off by default. The clock measured silence, which a training
node is full of, and remote workers had already stopped enforcing it — their
heartbeat reset it. Now a heartbeat proves the agent rather than the node,
ninety seconds of nothing fails the call either way, and the engine touches
work it is still running so a long node is not redelivered at sixty seconds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69
The gates have never gone green on the new runners. Three separate reasons:
- backend/Dockerfile shipped Python 3.10 while the code imports typing.Self
and datetime.UTC, so the container exited on import and the suite could not
even load its conftest. The image moves to 3.13 and the packages declare
>=3.12, which is the floor the tests actually pass on; ruff's target follows
and rewrites timezone.utc and asyncio.TimeoutError accordingly. Relocking
drops the 3.10 branch, which bumps FastAPI and so regenerates the SDK.
- frontend/README.md had no trailing newline and two dashboard widgets used
arbitrary text-[…] sizes. Both are em-relative on purpose, so they move to
the inline style the neighbouring ramp already uses.
- Every commit left its own run queued: without a concurrency group a runner
that was offline for a while works through a backlog nobody reads. A stack
that fails to come up now prints its logs before the teardown removes it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every other README carries the AGPL line; these two did not, and they are the
ones that become pypi.org/project/fluksio and /fluksio-worker — the most
public surface the project has, and the place a copyleft licence is least
useful to have to go looking for. §13 is named outright for the same reason.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
Replace the empty LICENSE placeholders with the verbatim GNU AGPL v3 text,
fix the invalid "AGPLv3" SPDX string in the package metadata, and name
Melvin Strobl as the copyright holder wherever the old footers said
"Fluksio ... all rights reserved".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The release workflow was already right; what it would have uploaded was not.
`fluksio` had no readme, so its PyPI page would have been blank — the app
repo's own README is a contributor's map of `frontend/` and `docker/`, which
is the wrong front page for `pip install fluksio`. It now has one of its own,
aimed at somebody who landed on the project page. Both distributions gain
authors, urls, keywords and classifiers; `twine check` passes clean on all
four artifacts where it warned on two before.
The workflow publishes `fluksio-worker` first, because `fluksio` depends on it
and the other order leaves a few seconds — the whole of a first release — in
which the dependency cannot be resolved. `--check-url` makes a re-run skip
what is already uploaded rather than failing on it, which matters because a
version on PyPI can never be replaced.
Licence metadata is deliberately still absent: LICENSE is MIT in somebody
else's name, inherited from the template this was scaffolded from, and whose
it should be is not a decision to make in a commit. NOTEPAD carries it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
The workflow this serves: make a venv, install what you work with, then `pip
install fluksio` into the same one. Building a second environment beside it
was exactly wrong — the packages the nodes need are already here, and the
Modules screen was asking for them a second time.
`NODE_VENV=auto` (the default) adopts that venv. It declines in the three
cases where adopting would be wrong: `managed` says otherwise, a managed venv
already exists and may hold packages somebody installed on purpose, or the
engine is not running from a venv at all. The images set `managed`, since the
venv in them holds the app and nothing of anybody else's.
An adopted venv is never written to. `uv pip sync` makes a venv hold exactly
the manifest, so pointed at somebody's own environment it uninstalls their
work and the engine with it — `sync()` refuses outright and `reconcile()`
returns before it can be called at startup, which is where that would have
happened first. The Modules screen lists what is installed and drops its
editor; `pip` is how that environment changes.
`fluksio serve` now names the interpreter node code runs on, which is the
thing a data scientist most needs to know at that moment. `fluksio-worker`
already defaulted `--python` to its own interpreter, so a GPU box works the
same way — that was only ever undocumented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
A data scientist keeps their code where it is and decorates it: `@node`
declares a function's ports beside the function, `Flow(name, nodes=[...])`
says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds
one for a single flow. `fluksio sync` uploads the document plus a generated
import shim per node, so the store still holds a complete, runnable,
git-versioned definition while the code it imports stays theirs.
`fluksio login|run|runs` and `flow.submit().wait()` are the client half, over
the run endpoints that already existed. Runs record the user repository's
commit beside the store's, so "what code produced this number" is answerable
on the side that now holds the code.
- `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim
generator, an HTTP client and sync. Standard library only at import, so
`from fluksio import node` in a training script pulls in no engine.
- `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the
repository's commit; `POST /modules/refresh` retires the workers without an
install, which every sync calls — a worker holds the imported package in
memory, so an edit to it is invisible until the process goes.
- The canvas shows a generated body read-only and names the repository to edit
instead; a body edited there stops the next sync rather than being discarded.
- The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since
the shim imports a module whose first line declares them.
- `examples/myresearch` is the worked example, `make sync-example` uploads it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
A node's timeout now covers its body only: the pool loads the source into the
worker it picked, off the node's budget, so imports that outlast the timeout no
longer make a node impossible to run. Draft checks compile without caching, so
saving does not evict what a busy node is serving calls from. Requests carry an
id the worker echoes and the pool checks, a reply is encoded once, and the
remote-exception cache is bounded.
DELETE /flows/{name} answers 409 while the flow has a running or queued run,
which is what was letting run_node rows outlive their run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StpRc2C6au1WJ1EUU7fsfu
A cluster or GPU host installs `pip install fluksio-worker` and gets the
agent and the runner, not psycopg, numpy and the MCP SDK. The engine
depends on it as a workspace member, so the file it launches node code
with is the same file a remote worker runs — which is what keeps a node
unable to tell the difference. Copying the two files by hand still works.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
stroblmeandClaude Opus 5