`GET /api/v1/search/` hands the client one flat index of everything worth
jumping to — flows and the nodes inside them, dashboards and the widgets on
them, panels, secrets, modules, workers and alert channels — and cmdk matches
it in the browser, so results narrow while typing without a round trip per
keystroke. A node hit is the one thing no list endpoint could answer: it opens
its flow with that node in focus.
The panel is reached from **Search** above Documentation in the sidebar, or
⌘K anywhere. The flow canvas palette moves to ⌘P, being the narrower of the two.
The panels dialog gains an address (`/dashboards?panels`) so a panel hit has
somewhere to land, and the sidebar logo gets `shrink-0`: the rail's width
animates while the logo is already back, and a flex item short of room is
squeezed rather than clipped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016vGH7jqcXxWKP9wZFPyVdU
The tile only exists on an installation that has run something, which is the
same "nothing has been run yet" the Runs screen goes by. The count comes from
/runs/overview, already grouped by status in the database and already polled by
the Runs screen, so Home shares its cache entry rather than paging rows or
growing a second endpoint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1moruzue2kTJd3uVisgNk
FLOW_MAX_WORKERS, FLOW_MAX_CASCADES and FLOW_MAX_RUNS are all pool sizes,
so 0 says neither "none" nor "unlimited" — it is a pool that cannot be
built. They are PositiveInt now, so a 0 fails at startup naming the
setting rather than being swallowed by `max_cascades or MAX_CASCADES`.
The consuming fallbacks take only None as "nobody said": explicit
`is None` in the executor, and no clamp on RunService.parallel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1moruzue2kTJd3uVisgNk
Say on the querying-chart page that a flow doing its own binning has to drop
the newest bucket, as the observability endpoints already do, and make plain
that `inject` is the scheduler while `trigger` is a debounce and hold.
The poll loop remembered what it read rather than what it published, so a
value the node could not publish counted as said: the next poll skipped it,
succeeded, and health went back to ok with the port still dark. Remember it
only after inject returns, and report ok last.
A node reporting itself down is now derived into its flow's issues on read
and counted on the health summary, so the canvas marks it and Home says so.
Being down does not stop the flow, and the issue clears by itself when the
node reports well again. The repeating poll warning is logged once per
outage rather than once per tick.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1moruzue2kTJd3uVisgNk
Three things the one-folder-per-study layout ran into.
**Discovery walks down.** A plain directory is now walked all the way, so
`fluksio sync dev` finds `dev/s1_baseline/study.py` and naming each study is
no longer the price of the layout. Hidden directories, `__pycache__`,
`node_modules` and virtualenvs are left alone, and a package is taken whole.
Two files that would import under one module name are refused, naming both:
Python keeps one module per name, so the second would silently *be* the first
— and a node's generated body imports by that name, so a worker would run the
wrong study's code. The message says the fix, which is an `__init__.py` per
study directory. A module that raises while importing is now a sentence
naming the file rather than an importlib traceback.
**`run` and `sweep` sync downwards too**, so the flow is found from the
repository root without the sync-then-`--no-sync` two-step. A study that will
not import is a warning rather than a stopped run, since a walk meets every
study and a half-finished one two directories away is not this run's problem.
The upload was already a no-op for a flow nothing changed in, so what the walk
costs is import time — `--sync PATH` narrows it, and skipping unchanged
subtrees would need a cache keyed on file state that is deliberately not here.
**`serve` moves off a busy default port** — 8001, 8002, up to twenty — says
which it took, and writes that one into `client.json`. A port given with
`--port` still fails when it is taken, because naming one is asking for it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa
Comparing the per-node digest against an engine that does not record one is
comparing against nothing, and reporting every node as changed on every sync
for ever — which is what a client newer than its engine did, since `NodeDef`
drops fields it has never heard of. A node is named now only when both sides
carry a digest, so a no-op sync is `unchanged` again and the signal one syncs
for is back.
That silence had also been the only sign of the mismatch, so sync now names
it: one line saying the engine stored no record of what a node's code reaches,
with both versions in it and what to run. Bumped to 0.1.6 — the digest changed
the stored document's shape, and a version that does not move makes two
different engines indistinguishable, which is the thing it was made
load-bearing for a day ago.
`— draft` was printed whenever there was simply nothing to publish, which
reads as work left unfinished. It is said only when a draft is genuinely
there, and `— published` when one was.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa
`sync` follows each node function's imports through the project's own modules
— stopping at the standard library, at anything installed, and at Fluksio
itself, whose checkout would otherwise be most of every digest — and records
the file list with what it hashed to. The engine hashes those files again when
the run is claimed, so the fingerprint is live rather than a snapshot, and
falls back to what sync recorded when it cannot see them: a remote worker's
runs used to share one empty digest, and therefore one key.
Three things follow. Editing a helper a node calls into re-runs that node, as
before. Editing something the node never reaches no longer re-runs anything —
a notebook two directories away was invalidating every arm. And
`Run.code_digest` is now the hash of its nodes' digests, so it is neither
looser nor tighter than "the code behind these numbers", which is what makes
it worth joining an exported table on.
`sync` says so too: it compares the per-node digest against the stored one, so
a helper edit prints `train: updated (flow, fit)` instead of `unchanged`. The
digest is read when the document is built rather than when the flow is
declared, so a second `sync()` in one process sees an edit between them.
Also: `fluksio runs` shows only the inputs that differ from what the flow
declares, fitted to the terminal, so a flow taking a few kB of json no longer
wraps every line.
Every existing cache entry misses once — the fingerprint changed shape.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa
Three things the first export pass got wrong for a real study.
**Dotted paths.** A node returns a record, not a scalar — the numbers arrive
inside `final_metrics` — so `--metrics final_metrics.train_loss` yielded an
empty column and `--metrics final_metrics` yielded the whole record in one
cell. Both sides of the wide table now take dotted paths, and the defaults
reach the same depth: every number a result carries is a column named by its
path, and inputs are compared leaf by leaf, so two configurations differing in
one field give that field as the axis rather than two blobs that are merely
not equal. Lists stay whole — a curve belongs in the long table.
**`--list`.** Metric names are flow-qualified, so `--name train_loss` matched
nothing and said only that. `fluksio export metrics --list` prints the names
the selection carries, and an empty export made with `--name` points at it.
**A version to compare.** The CLI ships ahead of the engine and a stale one
answered a flat 404 with nothing anywhere in the API to tell how old it was.
The engine reports `version` on `/observability/summary`, `fluksio status`
prints it, and a 404 from export now names both versions — or says "older"
when the field itself predates the engine. Bumped to 0.1.5, which is what
makes the number worth reading.
Also formats `flow/metrics.py`, which had been committed unformatted and was
the last `ruff format --check` failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa
`fluksio export metrics` is the long table — a row per run, metric and step —
and `fluksio export runs` the wide one, a row per run with the inputs that
*vary* across the selection as columns beside its final numbers, status,
duration and the commit and digest of the code it ran. Both carry the run id
on every row, which is the join back to the run page and what makes an
exported file auditable. `Client.export_metrics`/`export_runs` answer the same
rows to a notebook.
The engine streams csv or jsonl from two routes declared above `/{run_id}`;
parquet is a client-side conversion behind the new `fluksio[parquet]` extra,
so nobody pays for pyarrow who does not want dtypes kept. The long export
reads each run through `_series`, so a cached node's curve comes with it, and
`--stride` thins each series rather than the concatenation of all of them.
Two things they needed on the way: `GET /runs` takes `?since=` and `?before=`,
so a long history pages by the last row's own timestamp instead of an offset
that shifts under it; and a read that reaches no engine now says so in half a
second rather than seven, because `runs`, `flavors`, `export` and an unwatched
`status` pass `retries=0`. Everything that submits keeps them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa
Concurrent runs sat at 4 whatever FLOW_MAX_CASCADES said: that setting bounds
cascades, and the run drivers read a hardcoded MAX_PARALLEL nobody could reach.
FLOW_MAX_RUNS is the knob they read now, --max-runs/--max-cascades/--max-workers
are the same three as flags on serve, and the engine says which numbers it
started with — which is the only way to tell that a settings file was read.
Events keep the run they happened in. The payload always carried it and the
persist path dropped it, so reading one run's failures meant filtering the
engine-wide list; a batch run's id reaches those events now too, since a run
has no journaled item to name itself by.
Also: a provisioner's 0 means "no deadline" rather than "cancel on the next
reconcile", and a command that reaches no engine says how to start one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015sbYeYaVgYQqm1sbx7wPdL
The timer thread promoted due work on a fixed one-second tick, so every
delayed item was 0-1000ms late whatever the load — measured on the house
at 705ms mean on a rollershutter stop, which is 2-4% of a 26-second
travel and accumulates in the position the motor node believes it is at.
It now sleeps to the soonest deadline and is woken when a nearer one is
scheduled, which measures 0.9ms end to end through Redis.
A promoted timer also went to the back of the queue. It goes into a due
lane of its own that `claim` reads first, so work that has waited out a
deadline is not held up by work that is merely queued.
Beside it, in the same code: seeding a message now bumps its version, so
a re-put flow's synchronous nodes no longer wait forever on a value that
is sitting in state; the consumer group drops the consumers of engines
that are gone (138 had accumulated on this installation); and the cast
that closes the long-standing `xclaim` mypy error.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Slurm is not a machine that attaches and stays; it is a queue somebody else
owns. So nothing here submits a node to it. It submits a job whose payload is an
ordinary worker dialling back in, and everything downstream — the protocol, the
artifacts, cancellation, the books — already worked and did not have to learn
what Slurm is.
The alternative, which Covalent takes, is to stage a serialized call and a
runner onto the login node, poll squeue and copy the result back: a second way
of running a node beside the one that exists. The cost of not doing that is one
assumption, that a compute node can open a connection outward. Where that is
false, _payload is the single method a staged variant would replace.
Clusters are configured in provisioners.json beside the alerts, since this is
infrastructure an operator writes rather than anything a flow says. The script
is generated with the system ssh and no new dependency, and prerun owns the
environment — deliberately no pip install, because what is on a cluster is
somebody's decision.
One outstanding request per profile, cancelled if it never attaches and on the
way out. Nothing autoscales.
The run gate needed the same hook: a run held before it starts never reaches the
placer's own wait, so it would have queued forever on a machine nothing had
asked for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
A worker reported its labels and nothing about the machine behind them, so the
engine could route a node to a GPU box but not tell whether that box had a GPU
free. Inventory — cores, GPUs, memory — now arrives with the hello frame, and
the run frame carries back what the engine allocated for that call.
Which is protocol 2 on both ends. GPUs are never probed: asking a vendor tool
would make the one dependency two, so a GPU is what the batch job says it was
given or what --gpus says. A worker that reports nothing still attaches and is
scheduled by its label alone.
Two things a job scheduler needs: --max-idle stops a worker started for one job
rather than letting it hold its allocation to the walltime, and a refusal is now
fatal instead of a reconnect loop that reads as a hang in a job's log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
A port may now declare `image`, `audio` or `video`. Each is the artifact
reference the engine already had, narrowed by the `media_type` on it, so a
speech recogniser declares what it eats rather than taking any bytes at all and
finding out. Bytes still never travel as a message and nothing on the wire
stops being JSON: a camera publishes one reference per frame, a microphone one
per chunk, and a reference may carry a `meta` dict nothing here interprets.
Streaming media is therefore an ordinary streaming port — with one change to
what that means. An emission used to journal an item with no payload, so
downstream read whatever was current when the item was claimed; a consumer
slower than its producer saw only the newest chunk and the ones between were
lost. That is right for a training curve and wrong for a second of speech, so
an emission now journals a `kind="emission"` item carrying its values, and the
executor hands them to the nodes reading that message instead of writing them
to state again. The value in state stays the latest, which is what everything
else reads, and the wave is filtered by what actually changed rather than
walking everything reachable. No queue serialization change — the existing
`outputs` field carries it.
Continuous media makes the store's missing GC a real problem, so this closes
it: `sweep_artifacts` runs hourly, keeps every digest a `run_artifact` row
records or a live message holds, spares anything written in the last hour, and
stands aside entirely while a run is in flight, since a node may store a
checkpoint long before it returns the reference to it. That also collects the
orphans a deleted flow has always left behind. `ARTIFACT_GC_INTERVAL_S=0` turns
it off.
Around the edges: `GET /artifacts/{digest}` serves the media type the caller
passes and answers ranged requests, so a browser plays a clip rather than
downloading it; `PUT` spools to disk instead of holding the whole body in
memory, as does `save_artifact` given a path; a Media widget draws whatever its
message points at, and a wall panel may fetch the bytes its own tiles are
showing and nothing else; and a connector gets `save_artifact`, for a device
whose readings are bytes.
What this cannot do is live video: a frame every second or two is a glance, and
the honest answer above that is the camera's own stream, which the widget takes
as a URL and the browser plays from source.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An undeclared key fails the whole reading, so a connector whose ports vary by
model has to narrow what it publishes to what was bound. Names the idiom and
the once-not-per-poll rule for saying a declared port is missing.
Five concurrent training nodes, each sizing its thread pool to every core,
left the engine's own event loop unscheduled: the API stopped answering
within 10 s and every client died. The same shape on a GPU deadlocked a run
for 21 minutes at 0% utilisation with nothing failing and nothing to read --
it just sat in `running`.
@node(resources={"cpus": 2}) is the declaration. The engine holds that much
for the length of the execution, so more of them than the machine has room
for wait their turn rather than oversubscribing it, and a `gpus` node holds
its card exclusively. FLOW_CPUS defaults to every core but two, and those two
are what keeps the engine answering.
Because a thread cap is read when the process imports the library, a warm
worker cannot be told a different one -- so an environment gets a pool of its
own and nodes deriving the same one share it, rather than paying a cold start
per call on exactly the nodes whose imports are slowest. XLA_FLAGS is never
derived: it is a composed, version-dependent string, so it travels in
resources.env where it is visible.
A node that declares nothing is not accounted for and behaves as it always
did -- it just gets FLOW_CPUS/FLOW_MAX_WORKERS as a thread cap, which is the
half of this that fixes the reported incident without anybody declaring
anything. An operator who set OMP_NUM_THREADS themselves still wins.
Resources are claimed strictly before a worker slot, so the two blocking
waits cannot deadlock. A node queued for them publishes node_queued and shows
on GET /workers/resources, because waiting and hanging looked identical.
Accounted, not enforced: no cgroups, no rlimits. Scheduling across machines,
flavours and enforcement are the next steps.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three faults with one root: the stored body of a code-defined node is an
import shim, and nothing that mattered was ever read from the code itself.
- The run stamp could not identify what ran. The shim imports whatever is on
disk when the worker starts, and an uncommitted tree stamps <commit>-dirty
for every run it ever produces. Run.code_digest hashes the repository's .py
files, memoized on their stat state, and it is read again when the run is
actually claimed -- so a sweep queued for hours records the code each of its
runs executed, not the code that was there when it was submitted.
- The stage cache adopted code that was too new. The fingerprint hashed the
shim, which is invariant under any edit to the imported function or anything
it calls into, so a re-run was served from cache and answered without the
outputs the edit added. It now carries the repo digest and the node's
declared ports. Every fingerprint changes once, which invalidates the
existing cache; a canvas flow has no repository and keys as before.
- An interrupted sync looked like a hand-edited canvas. The engine answers a
new-node template for a node with no stored body, and the template carries
no marker, so the drift check read "somebody edited this" and demanded
--force -- for the one state that re-running the sync is the fix for.
NodeSource.missing states the fact, and sync skips those and reuses the
bodies it read instead of asking for each one twice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A driver script died of one slow answer: httpx.ReadTimeout out of
RunHandle.refresh() with a 30 s read timeout and no retry anywhere, which
cost a sweep 78 of its 84 runs.
- Split the timeout (5 s connect, 120 s read): a wrong URL fails at once,
and a busy engine gets longer than the slowest thing it does on purpose
(a 60 s compile, a 15 s rebuild wait).
- Retry idempotent calls three times on a transport error or 502/503/504.
503 is the engine's own "ask again" — it is what RebuildBusy answers.
- Submit carries a key the engine stores with the run, so a retry after a
timeout returns that run instead of starting a second. A sweep keys every
entry, so a half-created one recreates only what is missing.
- wait() and --follow tolerate five failed polls in a row; a 404 still stops
at once, because that is an answer rather than a gap.
- CLI says "engine not answering" and names the run still on the engine,
instead of printing a traceback.
- runs: clamp the params column to 80 characters; events() takes the
flow/since/until the endpoint already had; RunHandle.failures answers
"what killed this run" from the run's own node rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A node runs when something it reads was published, not merely when something
upstream of it ran — worth saying now that it is true, since it is what a
reader relies on when a rate-limited port goes quiet. Plus `FLOW_MAX_CASCADES`
in the engine's configuration table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf
`POST /runs/flows/{name}` hardcoded `cause: "api"`, so every row in the
history claimed the same origin. The body now carries an optional `cause`,
closed to the values the column knows — the dashboard sends nothing and stays
"api", `fluksio run` says "cli", and the SDK client says "sdk".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
A cache hit still replays no emissions — those values were the story of an
execution that is not happening — but the run they were recorded in is now
written on the row (`run_node.cached_from`), and the metrics endpoints read the
series back from there. So a reused run answers `run.metrics("train.loss")`
with the same points the run that trained did, rather than looking like a run
that produced no numbers at all. Pointed at rather than copied: a sweep of 500
reusing one frozen node would otherwise duplicate its curve 500 times.
That needed the cross-flow restore fixed first. The cache key has no flow in
it while the stored outputs are named for the flow that produced them, so
`quick.prepare` getting a hit from `train` wrote `train.dataset` into `quick`'s
state and the next node was called without its argument. One rule now covers
both halves: `requalify` reads a name owned by one flow as the same name in
another, applied to the restored outputs, to the node id behind the pointer,
and to the series names on the way out. Reuse across flows is kept.
Also: `@run:<id>.<output>` and a bare `sha256:` digest resolve on every input,
not only artifacts. Chaining a run's json config into the next one from a shell
meant pasting the whole object inline, and the CLI could not even send the
spelling — `_coerce` died in `json.loads` before the engine saw it. Both
spellings are reserved on every input now, `str` included, and `_from_run`
returns whatever the run's result holds rather than only a reference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp9L6gakMVro1K2C5zdtBE
Four things from a testing pass.
`fluksio serve` printed its own lines through the root logger, which has no
handler and falls back to `INFO:fluksio.cloud.connector:...` — beside uvicorn's
aligned output it reads like something went wrong. The engine's loggers and
alembic's now use uvicorn's own handler. Named rather than configuring the
root: httpx logs every portal call at INFO and none of that is printed today.
`fluksio enroll` writes its config from another process, so an engine already
serving never learned it had been paired. It now looks for one every few
seconds and dials when it appears. `load()` rather than `exists()`, or a file
that does not parse would be restarted forever.
`fluksio status` says where the installation stands with its portal — never
paired, linked, or paired and unreachable, which is the one worth acting on.
`--seed` and `--timeout` had no help text at all. Both say what they are for
now, and the docs say what a seed is actually for: recorded on the run, part of
its input digest, and passed to an input named `seed` when the flow declares
one, so the number a run is labelled with is the one the code drew from.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69
`runs --local` already exists for the reason status needs it: the flows and the
history are in this directory, and looking at them should not need a server.
Watching still does — an in-process engine is the command itself, so nothing
could change under it — and says so rather than sitting on a still frame.
Found setting up a sandbox that drives the SDK entirely through --local.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69
Two halves of the same gap: the CLI could start work but not show you any.
`fluksio status` draws the home screen's top half in a terminal — health and
what is wrong with it, every flow with its state and node count, and the
recent runs and failures under them. `--watch` keeps it there. Rich does the
drawing; it was already installed under fastapi's own CLI, and is named now
because a command depends on it.
`fluksio run` with no parameters at a terminal asks for them, one line per
declared input with its declared value in brackets — so Enter through the lot
is what running the defaults looks like, and an artifact input takes the
`@run:` spelling the engine now resolves. A scripted run is untouched: passing
any parameter, or piping the command, skips the questions, as does --defaults.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69
Four things the python SDK turned up, each fixed where every client sees it.
A key no port declares is now an error rather than a silent drop, on the
return, the yield and the emit alike — the contract the docs already stated.
The SDK reads literal yields at sync time, so a typo fails before anything
runs, and an emission of one fails the call rather than being logged where
nobody looks.
NaN and infinity are refused at the port. JSON cannot spell either, so one
that travelled came back as a 500, a socket frame that stopped the canvas, or
a metric batch the database dropped whole.
An artifact input takes `@run:<id>.<output>` or a bare digest, resolved on the
engine — so the CLI, the run dialog and a python caller mean the same thing,
and a sweep can pass one at all.
Node timeouts are off by default. The clock measured silence, which a training
node is full of, and remote workers had already stopped enforcing it — their
heartbeat reset it. Now a heartbeat proves the agent rather than the node,
ninety seconds of nothing fails the call either way, and the engine touches
work it is still running so a long node is not redelivered at sixty seconds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69
A code node in a batch run is now fingerprinted by its source, its raw
settings and the values it reads — an artifact input counting as its digest,
which is what the content addressing was always for. A run that finds the key
restores what the earlier one returned and skips the node, recorded as
`cached`. The run history is the cache: `run_node.outputs` beside the
`cache_key` the schema already had, no second store. On for code nodes, never
for the built-in and connector types that have side effects; off per node with
`@node(cache=False)` and per run with `--no-cache`.
Emissions are not replayed on a hit, so a cached training node returns its
result without redrawing its curve. Recorded in NOTEPAD.md with the two other
deliberate limits.
`fluksio run --local` boots the real app in the command's own process and
drives it through its ASGI interface behind the ordinary client, so a run no
longer needs a `serve` terminal beside it — same data directory, same history,
and the cache carries between the two. It always waits, because the engine it
starts lives exactly as long as the command.
Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists,
`run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited
run rather than abandoning it, coloured statuses on a terminal, and `name`
made optional on the metrics endpoint so a follower can ask for every series.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
**The portal link puts itself back up.** It already retried a connection that
raised, but a session that ended *cleanly* — a portal restarting, a proxy
closing an idle socket — returned normally and went straight back round the
loop with no wait at all, so an engine could spin against a portal that was
merely saying goodbye politely. Every ending now reconnects on a delay, and
the delay turns on whether the attempt got as far as attaching: one that stood
up and dropped is a network event and retries at once, one that never stood up
waits longer each time. Jittered, so a portal coming back is not met by every
installation it serves in the same instant. Ping timeouts are named rather
than defaulted, since they are what bounds how long a suspended laptop's dead
socket looks alive, and the keepalive task is awaited so the reason a link
went reaches the log instead of the garbage collector.
**`fluksio enroll <code>`** is the whole command now; hub.fluksio.com is the
default and `--portal` names another. The one command run before anything
works should not need two flags.
**`fluksio run` syncs first.** The reason a run exists is usually the edit
before it, so remembering to sync was remembering to do something the computer
could do — including the worker refresh, which is what makes an edit to your
own package take effect at all. `--no-sync` opts out for a tight loop.
That last one needed discovery fixed: it only ever looked at top-level `*.py`,
so a repository whose code is in a package — the ordinary shape — found
nothing from its own root. It now descends into the packages it holds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
Two things a local install should not have asked for.
`fluksio serve` now signs you in. Logging in to your own machine was a
formality — the password was printed by the same process that would have
checked it, and the database it authenticates against sits in the directory
the token goes into — so `serve` mints the token itself and says where it put
it. `fluksio login` is left for an engine somewhere else.
And an installation is `.fluksio` beside the code, found the way `.git` is,
rather than one `~/.fluksio` for the machine. A repository with its own venv
was already getting its own engine; it now gets its own flows, run history and
token too, instead of three repositories sharing one database and fighting
over one port. `--global` asks for the shared one, `--data-dir` still names
any directory, and when both exist the banner says which you are looking at
and how to reach the other.
The directory ignores itself from within — a `.gitignore` of `*`, the way uv
writes one into `.venv` — because it holds a credential and a database, and
neither belongs in anybody's history. The token is written mode 600. A login
an older version wrote to ~/.config/fluksio is still read, so nothing that
worked stops working.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
The gates have never gone green on the new runners. Three separate reasons:
- backend/Dockerfile shipped Python 3.10 while the code imports typing.Self
and datetime.UTC, so the container exited on import and the suite could not
even load its conftest. The image moves to 3.13 and the packages declare
>=3.12, which is the floor the tests actually pass on; ruff's target follows
and rewrites timezone.utc and asyncio.TimeoutError accordingly. Relocking
drops the 3.10 branch, which bumps FastAPI and so regenerates the SDK.
- frontend/README.md had no trailing newline and two dashboard widgets used
arbitrary text-[…] sizes. Both are em-relative on purpose, so they move to
the inline style the neighbouring ramp already uses.
- Every commit left its own run queued: without a concurrency group a runner
that was offline for a while works through a backlog nobody reads. A stack
that fails to come up now prints its logs before the teardown removes it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two looks were somebody else's language spoken well, and neither was the
product's. A dashboard nobody has dressed yet should look like the rest
of the app, so there is now a third set that follows the root
DESIGN-GUIDELINES.md to the letter — `--card` surfaces told from the page
by a hairline and a low shadow rather than by colour, every control a
pill, 16px panels, frosted floating chrome, one slate-blue accent spent
on what a person can act on — and it is what `look` means when nothing
says otherwise.
That also turns the exemption the other way round. The dashboard is still
allowed to look unlike the product; it just no longer does so by default.
An existing dashboard, which has never named a look, lands on the design
it had before any of this.
Restraint is the style rather than an omission here: no ripple, no glow,
no lift, and a press answered by the colour changing. The one deliberate
departure is the selector, which holds its choice in `--primary` rather
than the `--accent` the segmented rule asks for — that is a decision
about the widget, not about the look, and a control must not change what
it signals when the drawing changes. All three sets hold it the same way.
Several of them describe this installation specifically — its broker, its
aircon, its DMX wiring — and this repository is about to be public. The
test and client-generation scripts stay; CI depends on scripts/test.sh.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds reference/connector-contract.md and code/connectors.md to the site, with
the node-type reference finally linking the contract it describes. Strict
build passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five things a wall panel showed that a wall panel should not.
**A tile's body no longer clips.** It scrolled, and a box that scrolls
also cuts whatever crosses its edge — which took the glow off a lit
button at exactly the width where the button filled its tile, and off a
gauge's arc at exactly the height where the dial filled its own. Only
what is written or listed asks for a scroller now; everything else is a
picture drawn to fit, and what overflows is left to the frame, which
clips at the tile's edge where a shadow has already faded out. The
slider's phantom scrollbar goes with it.
**The selector is a selector.** Named for what it does rather than what
it is, and the choice it is holding is held in the dashboard's own
primary — a pill that slides between the options rather than a grey one
that had to be looked for. The stored type is untouched, so no document
changes meaning.
**The arrangement is held off the panel's edges**, by the same distance
it holds between two widgets. The ground is not held off with it: a
background covers the whole panel, and only what is arranged on it has
a margin. No stored panel loses a row to it.
**The rail is drawn on the panel.** It was chrome bolted to the edge of
the screen beside the canvas — in the app's own design rather than the
dashboard's, and on a scaled canvas not even lined up with it. It now
takes a column out of the canvas the way the margin does, scaled with
it and wearing its look. Which cell each widget sits in is unchanged;
only how big a cell is.
Two of these were the same mistake twice: an unlayered rule stating
`position` for everything wearing a class — `.gl-surface` on a rail
placed by a utility, and a blanket lift over every child of a pressable
on a pill placed by `layoutId`. Both now say it one element at a time.
The workflow this serves: make a venv, install what you work with, then `pip
install fluksio` into the same one. Building a second environment beside it
was exactly wrong — the packages the nodes need are already here, and the
Modules screen was asking for them a second time.
`NODE_VENV=auto` (the default) adopts that venv. It declines in the three
cases where adopting would be wrong: `managed` says otherwise, a managed venv
already exists and may hold packages somebody installed on purpose, or the
engine is not running from a venv at all. The images set `managed`, since the
venv in them holds the app and nothing of anybody else's.
An adopted venv is never written to. `uv pip sync` makes a venv hold exactly
the manifest, so pointed at somebody's own environment it uninstalls their
work and the engine with it — `sync()` refuses outright and `reconcile()`
returns before it can be called at startup, which is where that would have
happened first. The Modules screen lists what is installed and drops its
editor; `pip` is how that environment changes.
`fluksio serve` now names the interpreter node code runs on, which is the
thing a data scientist most needs to know at that moment. `fluksio-worker`
already defaulted `--python` to its own interpreter, so a GPU box works the
same way — that was only ever undocumented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
A dashboard is a wall panel somebody hangs in their own hallway, so it
now wears what they choose: a look, and a palette of their own colours.
Two complete component sets live under `Dashboard/ui/` — `glass`
(translucent panes over a slowly moving ground) and `material` (Material
3 tonal cards) — behind one prop contract. Every control's state,
keyboard and `aria-` live in `ui/core` and are shared, so the two sets
are the same dashboard drawn twice rather than two products: a set only
decides what a control looks like while doing it.
Four settings join the channel, each drivable by a flow like any other:
`look`, `palette`, `background` and `touch`. A palette is an ordered list
of hex colours — background, surface, primary, accent, text, then more
chart colours — pasted from a coolors.co link or typed, written onto the
canvas as the token variables everything already reads. Trailing roles
are derived, so three colours are a whole dashboard, and derived text is
held to AA rather than trusted (`theme.check.ts` measures it). A palette
also decides light or dark, since its first colour is the ground.
Widgets are measured against their own tile with container queries rather
than against the viewport, animate through `motion`, and can be drawn
without their title. The three reworks:
- a bar draws a row per reading, up to eight, each in the dashboard's own
data colours and each able to carry its own scale — replacing readings
nested in one fill, which could only ever share one colour and stop at
three. Documents written the old way are read as rows.
- a chart's range picker moved to a column down its right-hand edge, which
gives the plot back a whole row of a short tile.
- the colour wheel became a disc: hue is the angle and saturation the
distance from the middle, so a colour is one gesture rather than three,
with brightness on a slider beside it.
`index.css` and `lib/motion.ts` are untouched — the dashboard overrides
token *values* on its canvas, never the blocks the two repos share.
A data scientist keeps their code where it is and decorates it: `@node`
declares a function's ports beside the function, `Flow(name, nodes=[...])`
says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds
one for a single flow. `fluksio sync` uploads the document plus a generated
import shim per node, so the store still holds a complete, runnable,
git-versioned definition while the code it imports stays theirs.
`fluksio login|run|runs` and `flow.submit().wait()` are the client half, over
the run endpoints that already existed. Runs record the user repository's
commit beside the store's, so "what code produced this number" is answerable
on the side that now holds the code.
- `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim
generator, an HTTP client and sync. Standard library only at import, so
`from fluksio import node` in a training script pulls in no engine.
- `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the
repository's commit; `POST /modules/refresh` retires the workers without an
install, which every sync calls — a worker holds the imported package in
memory, so an edit to it is invisible until the process goes.
- The canvas shows a generated body read-only and names the repository to edit
instead; a body edited there stops the next sync rather than being discarded.
- The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since
the shim imports a module whose first line declares them.
- `examples/myresearch` is the worked example, `make sync-example` uploads it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
`FlowStore._git` now passes `-c gc.auto=0`, so `git commit` no longer forks a
background `gc --auto` that reparents onto PID 1 and stays there unreaped. With
auto-gc off nothing packs on its own, so `_commit` runs a foreground `git gc`
every 500 commits — the trigger a long-lived seeding session actually reaches.
`docker/compose.yml` takes the frontend's `VITE_API_URL` build arg from the
environment, keeping `https://api.${DOMAIN}` only as the fallback. `setup.sh`
already derives the scheme from `ENVIRONMENT`, so an `up --build` that does not
layer `compose.local.yml` stops shipping a bundle that calls `https://api.localhost`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StpRc2C6au1WJ1EUU7fsfu
Four small things, each with a device behind it.
An MQTT filter now routes what it subscribed to. `+` and `#` reached the
broker and were then looked up in an exact-match dict, so every message a
wildcard subscription received was dropped in silence.
`json_key` lifts a value out of the object a device wraps it in — Victron
publishes `{"value": 47}` on every path, which was otherwise a Python node
per port.
The trigger node learned `passthrough` and `wait_port`, because how long to
wait can be a value rather than a constant: a rollershutter takes 26 seconds
up and 28 down. A wait of zero sends nothing afterwards and still cancels
what the last message scheduled, which is how a stop is commanded once
instead of forever.
The HTTP sender takes fixed `query` parameters, so an API key is a secret
reference rather than a message on the canvas, and `send_inputs` off for a
request whose inputs are only a trigger.
Also: `delay` accepts fractional seconds, and `TZ` reaches the container, so
a cron expression means local time. Left unset it is UTC, as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A dashboard could only ever receive as a set of tiles. This adds the dashboard
itself as a receiver: `settings` maps a name to a value plus an optional
binding. Unbound, the setting is simply its value — a wall panel that is always
dark costs no flow. Bound, a flow drives it live and the value is the fallback.
Two settings are wired: `theme` (system/light/dark) and `locked` (read-only).
There is no schedule field on purpose — a node publishing to the bound message
on a cron is what a schedule is here, which is the point of a channel.
- `messages_for()` now walks a dashboard's bound settings as well as its
widgets' bindings. Without this a paired screen is refused its own theme
message, on the one surface the setting exists for; it bounds the socket too.
- `locked` is gated in `usePublish`, so every control inherits it, and each
control also draws itself disabled — a dead button reads as broken otherwise.
The panel surface says Read-only in the corner.
- The theme is a class on the dashboard's own surface, never the root: inside
the app shell it must not flip the chrome. `.light` gains the tokens `.dark`
already had (mirrored in the index repo) so both directions work on a subtree.
- Settings bindings are type-checked from the document alone, the rule widget
bindings follow, and mirrored on the server.
- A bound setting is drawn on the flow canvas as a dashboard-level endpoint.
- The demo's house flow now publishes `home.panel_theme`, which the demo
dashboard's theme binds to: the panel goes dark after sunset, at no tile cost.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018tULRZJUkZsw7rMJ3h4xvu
A custom hue ring — a conic gradient, not a canvas — with saturation and
brightness sliders beside or under it depending on the tile's shape, sized
for a wall panel and reachable from a keyboard. It publishes [h, s, v] by
default, which is what the reference installation's DMX encoders read, and
`format` switches that to [r, g, b] or "#rrggbb".
`usePublish` moves to its own module so a widget in a file of its own can
reach it without importing `widgets.tsx` back.
- Home puts the dashboards beside the flows: two equal-height columns,
capped and scrollable, most recently worked on first. Each tile is a
schematic footprint built from the stored widget placements.
- Flows and dashboards can be picked by long press or ctrl-click; the
create button becomes a trash and one dialog covers the batch.
- The offline banner is drawn on the body so it centres on the viewport,
and the live socket now releases the offline latch a stray 503 set.
- A boot spinner before React's first commit, a router pending screen for
code-split pages, and skeletons where an empty list used to flash.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018tULRZJUkZsw7rMJ3h4xvu
Three things a paired wall panel needed.
The scope check now walks the panel's widgets instead of allowing the
`/messages/` prefix wholesale: a screen may publish what its own controls and
querying charts point at, read the history of what its tiles draw, and nothing
else — the catalogue of every message in the installation included. The same
walk that already bounds its socket, so both surfaces agree.
Pending pairing codes moved out of the per-process dictionary into Redis, keyed
per code with the code's own TTL and indexed in a zset so the fifty-code cap
means the same thing to every worker. Without a Redis there is one process by
definition, and the dictionary stays.
And a per-panel nonce in the token, bumped by `POST /panels/{id}/unpair`: that
refuses the screen hanging there without touching the panel, its dashboards or
their arrangement. A save cannot write the nonce back, so a stale client cannot
undo a revocation. Only for a credential this installation signed — one the
portal minted carries no nonce and is revoked at the hub.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018tULRZJUkZsw7rMJ3h4xvu
"What it can import is what the Modules screen installed" was true and
read as a wall: every example is a self-contained file, and a function
that imports half a repository looks unsupported. It is not — the
manifest is handed to `uv pip sync` verbatim, so `-e /home/you/repo`
installs the project you already have and a node becomes a three-line
wrapper over it. The code stays in your own repository, under your own
version control, importing its own siblings.
Two caveats that are easy to lose an afternoon to, so both are written
down: an editable install reaches the venv without a reinstall but not
into a worker that already imported it — the engine's workers are
long-lived, and Apply is what retires them, while an attached worker
starts a process per call and is always fresh. And the path is resolved
on whichever machine runs the node, while the manifest is committed to
the flow repository, so a laptop path means nothing in a container.
Verified rather than assumed: `uv pip sync` takes `-e`, the shim body
loads through the real `load_function`, an edit is live in a fresh
process and stale in a reused one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guide asked for a hostname and served the interface on
`app.${DOMAIN}` with the API on `api.${DOMAIN}` — which is not what
someone with a box in a cupboard sets up, and could not be made to work
by typing an address into a browser. `compose.lan.yml` already solves it
for the dev stack; the same file layers onto the production one, so the
guide now opens with that and keeps the hostname as the other option.
The reference gained the setting behind it. `VITE_API_URL` is a build
argument of the frontend image rather than something the stack reads,
which is worth saying once: empty means the interface addresses the API
relative to whatever origin served it, so one image answers on an
address, a hostname and an ssh tunnel alike, and no CORS list has to be
kept in step.
Verified against the production compose layered with `compose.lan.yml`:
the page, `/api` on the same port, a cross-origin-free login, the flow
websocket upgrading through nginx, and `/docs` still refused there.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
stroblmeandClaude Opus 5