A `pip install` on a locked-down host — the case the CLI exists for —
may have no git, and the store shelled out to it while building the flow
repository, so the engine refused to start at all. The store is files;
git is their history. Missing it is now one warning and no commits
rather than a stack trace, which is the difference between a machine
that runs your experiments and one that does not.
Found by installing the wheels into a bare python:3.12-slim and pairing
it with the portal: `fluksio enroll` took the code, `fluksio serve`
dialled out, and the hub was proxying requests through the tunnel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`pip install fluksio && fluksio serve` on a machine with no Docker, no
database and no configuration — which is the case this is for: a node on
a cluster where ports cannot be opened. It makes its data directory, its
key and an admin account, prints the password once, and serves. Pairing
is `fluksio enroll <code> --portal …`, doing what the Settings screen
does through the same function, before the engine starts and without one
running — a machine nobody can route to has no browser pointed at it
either. The portal serves the dashboard, so nothing is served here.
Two things had to give way. `fastapi[standard]` pulls a cloud CLI that
wants sentry-sdk 2.x while we pinned below it — no pip resolution
existed, so the pin is lifted, which the comment beside it had been
waiting for and which also lets the Python cap go. And `uv` is now a
dependency rather than something to find on PATH: the Modules screen is
how a data scientist installs torch, and it was quietly falling back to
the engine's own interpreter.
The CLI imports nothing from the engine before it has set DATA_DIR — the
settings are built on the first import of core.config, and reaching it
early put the database in the working directory. There is a test for
that now, because the failure is silent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One process owns this database — the image has run a single uvicorn
worker for that reason since the four-engines bug — so a file beside the
flows is the honest shape for it, and it is what lets `fluksio serve`
need no infrastructure at all. Live values, node execution and the work
queue never came here anyway; what does is a rollup a minute at a time,
a row per cascade and the run history, and WAL keeps the readers going
while that one writer works.
DATA_DIR is now the one setting that moves everything an installation
keeps; the rest derive from it and the images still spell theirs out.
The schema is prepared in-process at startup, so the prestart service is
gone, and the ten Postgres-only revisions collapse into one portable
baseline.
Three things only worked because psycopg was casting for us: a token's
subject arriving as a string where the column is a UUID, `greatest`, and
`date_bin`. The timestamps needed a column type of their own — SQLite
stores no offset, and a naive datetime read back either raises against an
aware `now` or serialises as local time.
Postgres stays in the stack only for Umami, behind the analytics profile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A cluster or GPU host installs `pip install fluksio-worker` and gets the
agent and the runner, not psycopg, numpy and the MCP SDK. The engine
depends on it as a workspace member, so the file it launches node code
with is the same file a remote worker runs — which is what keeps a node
unable to tell the difference. Copying the two files by hand still works.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A wheel whose top-level module is `app` collides with anything else in a
user's venv, so the package that is about to be published takes the name
it is published under. Only the Python package moves; the repo, the
Docker WORKDIR and the compose project keep theirs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`make lint-frontend` was `biome check --write --unsafe ./` — a lint target that
reformatted the whole tree rather than checking it, which is why every parallel
change in this repo has had to work around it. `lint` checks now and a new
`format` writes. The pre-commit hook and CI needed no edit at all: both call
`bun run lint`, so they became checks the moment its meaning changed.
`app/Makefile` assigned DOMAIN from .env, and a plain assignment beats an
inherited environment variable and is not exported — so `cd app && make
dev-local` served localhost while the same checkout's tests targeted the
configured domain. `export DOMAIN ?=` gives the lattice that was intended:
command line, then environment, then .env.
Alongside: the backend's htmlcov bind mount created that directory as root, so
`make test-backend` died on the coverage step after every test had passed,
which reads like a test failure and is not one. The alerts screen's copy of
ALERTING_EVENTS is now checked by a test rather than trusted. And the shard
comment claimed two spec files where there are nine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
A node's error cleared the moment it ran again, so a failure that genuinely
fired an alert could leave no trace on the canvas by the time anyone looked.
The engine records it now — on the node's status, so it survives a reload and
every client agrees — and reading the traceback is what clears it. The seam is
the event bus, which is where every failing path already meets: a queued live
run, an explicit run, a preview, and a single triggered node all publish
`node_error`, while the controller's own observer would have seen only one of
them.
That was half the confusion. The other half: clicking a failed neuron on Home
often landed on a flow where everything looked fine. Nodes merge into one
neuron by instance key — every InfluxDB node pointing at the same bucket is one
neuron — and the click went to whichever flow contributed a member first, not
the one that failed. It now goes to the failing member and selects it, and the
canvas marks a failing node rather than leaving it to the dot alone.
The inject node emitted one payload to every port it declared, whatever their
types, so an inject on a bool port carrying the text "true" raised at publish
time. Each port gets its own field now, typed and parsed by that port's dtype,
and remembers what it last sent. A port that is renamed carries its value with
it; one that is removed takes its value with it. An inject written before this
keeps emitting exactly what it did.
The derived-cron chip also appeared on the delay node, where `interval` is a
rate limit and a schedule derived from it means nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
A dashboard went live the moment it was created — an empty document straight to
the panels — while a new flow starts as a draft. It now works the way flows do:
published means `dashboard.json` exists, so every dashboard on every running
installation is already published and nothing needs migrating. Only the ones
created from here on start as drafts.
Mirroring FlowStore turned up a latent 500: discarding the draft of a dashboard
that had never been published unlinked its only file, and the read that followed
raised out of a 200 handler. It answers 400 now, the way a flow does.
Publishing all of them was 2N requests, because a publish has to name the
version it expects and the summaries did not carry one. They do now — and so do
the flow summaries, which had the same defect nobody had written down.
A panel had no way to hear about any of this. A publish, or a change to which
dashboards a panel carries, now puts one event on the bus and the screen
refetches what changed: no reload, so a wall display never blanks or asks for
its credential again. The subtle half is that a socket's message allowlist was
computed once at handshake — a reassigned panel would have fetched its new
document and then shown tiles that never updated.
The panels dialog logged non-superusers out. Every write in it needs a
superuser, not only the checkboxes the report mentioned, so the dialog is
read-only for everyone else. The logout itself was `main.tsx` treating 403 as a
dead session, against the contract deps.py spells out: only a 401 ends a
session, and a 403 now says so rather than silently signing someone out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
An installation enrolled before per-user mapping has nobody mapped, and
fail-closed means its owner is refused. Re-enrolling fixes it and can only
be done from the machine's own network, which is the wrong thing to require
of a machine whose only route in is the portal.
The hub names the owner in the handshake now, and this takes it: if the
enrolling account has no portal identity and nobody else holds that one, it
is written once and every later attach is a no-op. A mapping somebody else
holds is never moved - enrolment was told who that is, and this is only a
repair. A failure to write one does not drop the link.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Remote access used to collapse every portal session onto the account that
performed the enrolment. That was the only thing it could do while nothing
here knew who was at the other end, and it is why letting a second person
in meant handing them the first one's account.
`user.portal_sub` is where a portal identity meets a local one: set for the
enrolling superuser at enrolment, and for each person a superuser admits
afterwards through Settings -> Remote access -> Add remote user. The code
they type comes from the newcomer's own portal account, and it is redeemed
against the hub with this installation's tunnel credential rather than with
a portal session, so being let in is not itself the power to let others in.
The account created is never a superuser, which closes the same door from
this side.
A proxy token now resolves through that mapping and nowhere else. An
identity nobody mapped resolves to no user rather than falling back on the
enroller, so deleting the local row under Admin -> Users is the whole of
the revocation: it bites on a credential already in flight, and it does not
wait on the portal being reachable to be told. Telling the portal is best
effort for exactly that reason.
The cost is stated where it lands, in DEPLOY.md: an installation enrolled
before this has no mapping, so its owner reconnects once with a fresh code.
Panels and the health summary still act as the enrolling account - neither
of them is a person, and neither gained a way to name one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
/observability/timeseries and /flows read every metric_minute row in the window
and folded them in Python, so the 7d preset pulled a week of rows on each 30 s
poll. date_bin() does the binning now — the row count drops to the slices asked
for, and to flows × 60 for the sparklines. A window of zero hours used to divide
by nothing and answer 500; windows are clamped to an hour at the low end and to
the retention period at the high end, past which there is nothing to find.
/observability/runs returns {data, count} rather than a bare list, so a minute
busier than the 200-row cap says so instead of quietly showing its newest 200.
The count is only queried when the page comes back full, which keeps the poll
from handing back what the fold just saved.
failures_24h leaves the summary — the Home tile counts errors over the selected
window from the rollups, and nothing had read the field since.
Deleting a flow now takes its Run rows and their nodes, metrics and artifacts
with it. This lives in the route rather than in forget_flow because renaming a
flow calls that too, and a rename must keep its history. The observability
rollups stay: they are the record of what ran, and retention already prunes them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
The queue shared the `pipeline:` prefix with flow state, so `RedisState.clear()`
could DEL the queue stream and `keys()` enumerated queue entries — only callers
filtering `__`-prefixed names kept it safe. It moves to `queue:` without a
migration: whatever is in flight at the upgrade is dropped once, documented in
DEPLOY.md rather than papered over.
Alongside it: `pool_pre_ping`, so a connection idle across a Postgres restart
costs a round trip instead of a failed request; the test suite pins
ENVIRONMENT=local and DOMAIN=localhost itself rather than inheriting a
deployment's .env; and `depth` leaves the queue stats, where it reported the
capped journal length as if it were a backlog.
ALERTS_FILE and PANELS_FILE now point at /data. They defaulted to a path on no
volume, so alert routing and every wall-panel pairing were living in the
container's writable layer and vanishing on each rebuild. Carrying the existing
files across is a manual step; DEPLOY.md has it.
development.md was still the upstream template — compose.override.yml,
localhost.tiangolo.com, `docker compose watch` as the dev flow — and said
nothing about the Playwright suite. Rewritten against what the Makefiles
actually do. deployment.md was template text too, duplicating the root
DEPLOY.md, and is gone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
Three things the security pass on the portal pairing turned up. The first two
were already true of a screen on the local network; what changed is that a
panel credential is now presentable from the internet, which is what makes
them worth closing rather than recording.
The artifact endpoint authenticates for itself, because a worker's credential
has to open it and that token is no use anywhere else. It resolved the caller
without handing over the request, so the one credential that is scoped by
route was judged by no route at all — a panel could read and write the store
as whoever approved it. It passes the request it already holds now.
The websocket has no route to judge either, and there the bound has to be on
what is sent: a panel is given the values its own dashboards draw and nothing
else — no node status, no logs, no shape of the graph. The keys stay in the
message, emptied, because a screen on a wall runs the bundle it was paired
with. `messages_for` reads that set off the published dashboards, and is the
walk the `/messages/` allowlist has wanted for a while.
And locality is no longer a header anyone can type. The marker the connector
stamps is a value minted per process, so reaching this API directly cannot buy
a device the credential meant for one that cannot reach it at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017F9RnYCJgASuBTcAjxmnsp
A screen somewhere this installation is not reachable from asks the portal for
a code instead, and the portal mints its credential — because a token signed
here is one such a device could never present.
Where it was minted changes nothing about what it may do. The panel gate moved
off the branch that decodes a local panel token and onto whatever claims name
a panel, so the portal's and this installation's are bounded by the same check
against the same panel's dashboards. A token of that scope naming no panel is
refused rather than left holding the account it borrows.
The connector marks what arrives on its socket, since that is the only thing
that makes it true, and the approval screen now names what is holding a code —
approving adopts whatever answers, so it is worth a look first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017F9RnYCJgASuBTcAjxmnsp
Two things stopped the engine commanding this house. ConnectorNode hardwired
its node function to a no-op, so an input message reaching a connector was
discarded and Art-Net's packet builder was unreachable; write() now carries
the input ports, which is additive so the contract version holds. And the MQTT
publisher JSON-encoded every payload, so "ON" went on the wire quoted and the
devices on a shared broker, which speak bare values, ignored it.
seed_house_control.py is the rig: a flow that drives the washing machine plug,
a dimmer and a colour fixture over MQTT, carries the same two as DMX on an
Art-Net node with transmit still off, and a dashboard to drive it by hand.
A python node's settings are constants of its own function, so they are passed
the way its ports are: by name. The controller binds them to the compiled
function, the `params` field is gone from the worker and remote protocols, and
a setting sharing a port's name is reported as a node error rather than
shadowing it. The panel's scaffold follows suit and keeps the header in step
with both ports and settings.
The demo's `pace` moves from a flow input to a setting of the training node,
which is what it always was.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUb8YpL2s3gmN9WTACTt4q
Both links out of the dashboard editor were built root-relative, so a portal
serving the app under `/i/{id}` got a URL to itself: the hub has no route
there and answers a bare 404. That is what a device link and "open what a
wall panel sees" both landed on.
They want different answers. The view link is for the person already looking,
so it takes the router's basepath — `appPath` in `lib/portal` is the same
prefix the router applies to every `Link`, for the places that step outside
it. The device link is for a screen, which cannot go through the portal at
all: the shell is served only to a portal session, and the credential that
page carries is the portal's rather than the panel's. So the server now says
where it answers, and `FRONTEND_HOST` is that answer — the same setting the
password-reset links already use.
Also fixes the panel branch in the query error handler, which compared a raw
pathname and so never fired under a portal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AHpLJHozysQXjsxAyU1WHj
A panel is one screen and the ordered set of whole dashboards it shows, so a
hallway tablet and a workshop tablet carry different sets without either
dashboard knowing about the other. More than one and the device draws a rail
to switch between them — the same rail the editor puts on screen, because the
wall has it and it takes room off the canvas.
A screen has no keyboard, so it pairs rather than logs in: it shows a
six-character code, somebody approves it against a panel from the dashboards
overview, and the credential that mints reaches that panel's published
dashboards and the message endpoints its widgets speak, and nothing else.
Deleting the panel revokes it.
Closes the per-device view and the kiosk credential; supersedes the
multi-page/multi-section UI, since a page is now a dashboard of its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AHpLJHozysQXjsxAyU1WHj
The brain graph counts node_executed events client-side and the websocket is
torn down on every shell change, so anything a flow published during the
navigation gap was lost. The bus now keeps a session tally per qualified node
and the snapshot hands it back, letting a reconnecting client catch up.
Both the route and the tunnel connector build that snapshot from one helper
so the portal cannot drift from the direct connection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HTsT1isxUjw5gtkJk8WhuA
Plumbing only: the widget-type literal and its dtype table on both sides,
the regenerated client, a curated lucide map and four stubs the renderers
are wired to. Also a latching switch and a segmented dropdown, both a
`style` on the control that already publishes and reads back, plus the
option editor a dropdown never had.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HTsT1isxUjw5gtkJk8WhuA
The "running" tile counted paused and invalid flows as running, so it read
"5/5" beside "1 flow(s) cannot run". Its note is now additive rather than a
precedence chain, so a quarantine no longer hides the invalid count.
Adds dur() beside si(): a ms reading picks its own unit, so a slow run reads
"1.24 s" instead of "1.2k ms".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HTsT1isxUjw5gtkJk8WhuA
An installation can be enrolled with a portal by redeeming a claim code, after
which it holds one authenticated websocket open and answers proxied API calls
over it. Requests are dispatched into this process's own ASGI app, so the HTTP
trigger routes flows install at runtime are visible to it, and the live flow
stream is bridged straight off the event bus.
decode_token grows the third branch its docstring anticipated: tokens signed by
the enrolled portal resolve to the local account that performed the enrolment,
verified against a JWKS pinned at that moment. With no enrolment the branch
raises immediately, so an offline installation is unchanged and untouched.
Disconnecting deletes one file, which is the entire local revocation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtBzdDyLsmDaF1W7DLYtYM
make seed-demo builds demo_training — prepare on the engine, a GPU-bound
train, evaluate back here — and a panel that draws the loss curve while the
training is still going. It is the session's whole argument in one flow: batch
runs with parameters and a result, a generator yielding on a declared port
rather than logging, fluksio.emit from inside a callback, artifacts carrying
the dataset and the weights between machines, and a sweep whose configs are
isolated from each other. The train node prefers its label rather than
requiring it, so it runs before a GPU box exists and says which machine and
which numeric backend it actually used.
Building it turned up two real bugs. A run waited for a worker its flow only
*preferred*, because required_labels ignored device_policy — so the example
hung on a label it did not need. And a run's seed never reached the flow, so
sweeping over seeds ran the same experiment N times; it now fills an input of
that name when the flow declares one, which is what the field looked like it
did all along.
Pressing Run on a batch flow now submits a run rather than taking the old
non-durable path — that button is the first thing anyone evaluating will press,
and it was quietly doing something else.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
The first cut had node code call fluksio.log_metric, which was a second,
undeclared way for data to leave a node: invisible to validation, absent from
the canvas, and stored where the graph could not see it. That is precisely the
MLflow discrepancy this framework exists to avoid, so it is gone.
A node that produces values over time is a generator. Every yield is a dict
keyed by output port, published the instant it happens — same port, same type
check, same place on the canvas as any other value — and what it returns is
its result. A port doing this declares stream: true, and a run keeps every
number one takes, so experiment tracking is a consequence of the graph rather
than an API beside it: a chart binds to a training curve the way it binds to a
temperature. fluksio.emit writes the same ports imperatively, for where a
yield cannot reach — inside a training framework's callback.
In a live flow an emission also wakes what is downstream, as a subscriber
publishing does; in a run it does not, because a run's graph is scheduled once
and mid-node cascades would leave 'finished' with nothing to mean. The
enqueued item carries no payload: the value is already in state, and one
carrying it would re-apply an old emission after the node returned.
Verified on the stack: 30 loss values arrived live on the flow socket during a
run, attributed to the node that produced them, and the same node run on the
remote worker streamed its curve back across the socket.
Also caches remote compile results per worker, so attaching a GPU box does not
put a network round trip in every rebuild.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
The engine runs where the automations are and the GPU is somewhere else,
usually behind a different network — so the worker connects out and the engine
answers over the socket it was given. Nothing has to expose Redis, and the
same connection works through the tunnel the hosted access will use.
What travels is the protocol the local pool already speaks, so a node cannot
tell which kind of worker it is on. A node declares device: gpu and
device_policy, the label is resolved per call (a worker attaching later needs
no rebuild), and a run whose labels nothing carries waits in the queue saying
what it waits for rather than failing — submit from the couch, the GPU box
picks it up when it is switched on.
Two things had to move with it. Compiling now happens on the machine that will
run the node: a node importing torch is correct on the GPU box and a missing
module on the engine, so checking it here failed nodes that were fine. And the
artifact endpoint accepts a worker's own credential, because storing a
checkpoint is exactly what that credential is for — and only that.
Verified against the real split: the training ran on this host (its checkpoint
names the machine and a numpy the engine does not have), streamed 40 metric
points back mid-run, and the evaluate node read the checkpoint on the engine.
Cancel kills the remote training; pulling the worker fails the run in six
seconds instead of waiting out its ten-minute timeout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
A checkpoint is not a message. DType.ARTIFACT carries a reference — digest,
size, media type, name — so everything on the wire stays JSON and thirty
megabytes never sit in Redis, which answers the vision's open binary-payload
question by narrowing it: inline codecs would only serve payloads too small to
be worth a round trip, and nothing asks for that.
The store is content-addressed rather than per-run, for three reasons that all
pay later: a sweep whose fifty configs share one preprocessed input stores it
once, a reference stays valid however it is passed around because it names
content instead of a location, and the digest is what a stage cache will
compare — so building it in now is what keeps that from being a change to the
message contract.
Node code calls fluksio.save_artifact/load_artifact and cannot tell whether it
is writing the engine's own directory or putting bytes over HTTP, which is
what will let the same flow run on a remote worker unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
A training loop has numbers worth keeping thousands of steps before it has a
result. Node code now imports fluksio and calls log_metric/progress, which
sends a line back without ending the call; the engine writes those to
run_metric in batches from the run's own driver rather than folding them off
the event bus, which drops what it cannot keep up with.
Two things fall out. Each event resets the worker deadline, so a node's
timeout measures silence rather than duration — which is what lets a two-hour
training keep a liveness contract instead of racing it. And the worker pool's
_running is now keyed by (run, node), so cancelling one config of a sweep
kills that training and leaves the rest alone.
Fixes a latent framing bug: read_line returned whatever a read had taken,
which was fine while a worker only ever sent one line per request and
unparseable as soon as it sent several. It now keeps the remainder.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
A cascade has no end worth recording; a run does. Parameters go in, the graph
executes until it drains, and the result is kept — which is what an ML
experiment is and what a CI-style job is, so both are one entity.
Each run gets a state backend namespaced to itself, so two runs of one flow
cannot overwrite each other's messages; that is a constructor argument rather
than a change to the pipeline, because every key the engine keeps already goes
through the state backend. Its record is written by the driver thread rather
than folded off the event bus, which drops what it cannot keep up with. Its
own Redis stream wakes an engine up, and from the claim onwards the database
row is the truth: redelivering hours of training because an acknowledgement
was late is not recovery, so a stale lease is what marks a run whose engine
died.
Flows gain mode: batch, which are built and validated but never activated, and
nodes gain a device label for the worker that must run them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
The canvas lays itself out: a layered graph, left to right on a desktop and
top to bottom on a phone, with room reserved for the value each edge carries.
Nodes cannot be dragged and `NodeDef.position` is gone from the document —
a graph nobody can arrange is one worth keeping small, which is what keeps
flows atomic. Endpoints join the same layout, so their lanes and the
localStorage that remembered where they were dragged go too.
Mobile, per the new Responsive section of DESIGN-GUIDELINES.md: the dock caps
its width and wraps instead of running off the screen, the dashboard stacks
into one column rather than shrinking a wall panel to a fifth of its size, and
Home stops widening its grid track past the viewport. A Playwright project at
a phone's width fails the build when a screen no longer fits.
Along the way: publish is the checkmark that was already there rather than a
button that appears and disappears, with discard beside it on both the flow
and the dashboard; the brain reveals a neuron's name on the first tap; and the
port sparklines get room to breathe.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDSXaRhvqHYNevgDGmNAto
A series, record or list message declares its shape instead of riding
DType.JSON, so a widget binds a shape rather than some JSON and a wrong
binding is refused before anything runs. A list declares its item type,
which is what keeps list[float] expressible for a pipeline.
On top of that: an agenda over a list, a notification over a record, and
a dashboard alert channel that publishes engine faults as one — so a
panel can show what went wrong without a flow wiring it by hand.
Also: only None means a node published nothing, a falsy value of the
wrong shape is now the named error it always should have been; and the
gauge's readout says its size is viewBox geometry rather than type scale.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A dependency loop is flagged on the canvas and was invisible everywhere else:
/observability/summary answered "ok" with an empty problems list while the
published flow could not run at all. It now reports the flows validation
blocks, and the brain graph carries the reason on each neuron the issue names
so the view built to find broken wiring can show it.
Node errors stay counted once, as the nodes that failed to load, and an
advisory like an unauthenticated webhook marks nothing — it is worth saying,
but the flow still runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
Both overviews carried the same toolbar twice, left-aligned, with a search
field permanently taking a row of width. One `OverviewToolbar` now serves
them: the search folds into an icon and expands again on click (Escape puts
it away and hands focus back), create is a `+`, and everything sits right of
the page. Each page keeps its own create dialog — the toolbar only renders
the trigger — so the testids the runtime spec and the capture script drive
stayed where they were.
Dashboards get the flow store's draft/publish split. The editor autosaves
`dashboard.draft.json` beside `dashboard.json`; `/view/{name}`, `bindings_for`
and `history_requirements` keep reading the published file, so a wall panel
sees an edit only once someone publishes it. `POST /dashboards/{name}/publish`
and `/discard` mirror the flow routes down to the version precondition and the
409, `GET /dashboards/{name}?draft=true` is what the editor asks for, and the
dock grows the same Publish button — which flushes a queued save first, so an
autosave in flight is not published around. Creating a dashboard still writes
the published file directly: an empty document on a panel is harmless, and it
keeps the store free of a never-published case.
"Publish all" is a checkmark in the toolbar, live only when something actually
has `has_draft`. A summary carries no version and publish needs the one it is
based on, so each document's detail is read immediately before its publish —
honest against a stale list, and no version-less backend path to maintain.
Failures are counted rather than swallowed: three of five fails says so and
names the three.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
`AlertManager.send` swallowed every delivery failure, so the alerts screen's
Test button answered 200 whatever happened — the one thing it exists for. It
takes `raise_on_error` now, which only the test route passes; the per-channel
loop keeps the swallow, because one dead channel must not stop the others
hearing about the same fault. A refused delivery answers 502 with whatever the
sender said.
Renaming a flow left its values under the old name for good: the delete path
already swept them, the rename path never did. It calls the same `forget_flow`,
which covers the messages and the `__ts__`/`__version__`/`__history__`
bookkeeping keyed by message name. Cleanup, not migration — they repopulate
under the new name on the next run.
Triggering a node by hand ran `Node.__call__` with nothing catching it, so a
node that raised produced a 500 and a stack trace in the server log, and
nothing at all on the canvas. `Pipeline.publish_error` is the reporting half of
`_execute_node` lifted out; both paths go through it, so a manual failure now
reads the same on the canvas and in the metrics as a queued one. The route
answers 400 with the node's error.
`MemoryWorkQueue.stats()` counts claimed-but-unacknowledged work rather than
reporting zero, so the health tile means something without Redis. The metrics
collector's held tracebacks are capped at `DETAIL_CAP` and swept on the same
`RUN_STALE_S` cutoff the open runs use, instead of one untruncated traceback
per node kept for the life of the process — a traceback still survives the
flush between the log and the failure it belongs to.
`GET /observability/events` takes `since`/`until`, the window `/runs` already
took, so a failures list can cover the span the charts beside it are drawn from.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
/observability/runs gains since/until, so the throughput chart's pin asks
the server for its minute instead of filtering a fixed recent list. This
engine writes ~60 runs a minute, so any minute but the newest read empty.
since is inclusive and until exclusive, matching the minute buckets the
charts are drawn from. Hover stays the client-side preview it was:
scrubbing a day would otherwise be a request per minute rested on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
The worker script is handed to the interpreter by path, so app/flow was
sys.path[0] for every node: `import queue` got the engine's. It now drops
its own directory before anything else imports, and runs with the
deployment's credentials scrubbed out of its environment.
Also: reload builds off the event loop, the pool wakes what is blocked on
it when it stops, a refused metrics flush is kept for the next one rather
than dropped, and the cascade events are paired through failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
A node type can now say which outside thing its parameters point at, and
nodes sharing one — a broker topic, a URL, a bucket — are drawn as a single
neuron on a new /brain canvas. That makes the wiring which runs between
flows through a broker visible for the first time; no single flow's canvas
can show it. The key is read off stored parameters, so a credential
reference never reaches an id.
Layout is a d3 force simulation settled once and then frozen, lit by the
socket the editor already listens to: a neuron pulses when any node behind
it publishes, and its connections light as values pass.
Fixes the message pulse while here: interpolating the stroke against the
edge's `color-mix()` resting colour went through oklab and left the gamut,
which turned every pulse on both canvases fluorescent yellow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
A second bus subscriber folds executions, errors, timings and queue lag
into per-minute rollups, keeps failures with their traceback and an audit
trail of who published what, and records one row per cascade — manual runs
and previews included, under an id of their own that writes no idempotency
markers. Read back through /observability/*, which always answers 200 so a
degraded engine still renders its own health screen.
Also fixes two things found on the way: node-health alerts read `status`
where the engine publishes `health`, so a device dropping never alerted
anyone, and the Redis queue reported `parked: 0` whatever was held.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
User code no longer execs in the engine. A pool of persistent worker
subprocesses speaks one JSON object per line; the controller installs a
proxy as the node's function, so every execution path funnels through it
and the pipeline is untouched. A crash costs one subprocess, a per-node
timeout is a kill, and cancelling from the canvas is that same kill.
The workers run a venv of the user's own on the data volume, filled from
a pip manifest versioned beside the flows. Applying it retires the
workers and rebuilds, so a package lands without restarting the engine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
Three things a pause and an interval were quietly losing:
- A rebuild builds a fresh pipeline, so nothing is paused any more and no
resume ever comes for what the old one parked. Release it on rebuild.
- POST /flows/{name}/step takes the oldest parked item and runs that one wave
while the flow stays paused, so a held-back cascade can be walked through.
Nothing parked answers plainly rather than failing.
- A per-port interval was leading-edge only: a producer going quiet inside the
window left the consumer on the value before it. The held value is kept and
a flush item scheduled on the queue's existing timer, so the window ends with
a delivery. One timer in flight per node, and none without a queue to run it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
Outputs are keyed by port, so a bare value cannot be one. The single mapping
point every caller routes through raised a bare AttributeError from
retval.items(); it now names the problem, and the message reaches the node the
way its other errors do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
Superuser-only management for agents that registered themselves: list them
with whether anyone approved them, and withdraw one without rotating the
signing key and cutting off every other agent.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
Moving a dashboard slider lit up an edge between two nodes that had done
nothing. The canvas pulsed on the message's timestamp alone, and a message
has no idea who published it — so it credited whichever node happened to
be drawn as a producer.
That was never only about dashboards. Two nodes producing one message
pulsed both their edges whichever fired, and a message produced in another
flow changed with nothing on screen to account for it at all.
Values now carry their cause: a node, a dashboard widget, another flow, an
agent or an API caller. An edge pulses only for the producer that actually
published, and the edge inspector says where a value came from when it did
not come from a node.
What is not a node in this flow is now drawn as one — a label rather than
a card, because a dashboard with twenty tiles would otherwise bury the
logic the canvas exists to show. That covers cross-flow wiring too, which
is the link in/out affordance that has been missing.
They are never part of the document. They join at render, after everything
that reads or writes the canvas nodes, so an autosave, an undo or a delete
cannot reach them — with a Playwright test that drags a node and asserts
the stored flow still holds exactly what it did.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Installing the connectors made this fail, which is the test working: it
insists every offered type is exercised. But a connector is a separate
package with its own tests, so it is not this suite's to cover.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
A dashboard is its own document rather than widgets placed in a flow.
Node-RED's dashboard tab is 260 nodes, about forty of them pure layout,
which is exactly what the small-graph principle exists to avoid — and
since the graph is already wired by message name, a widget can bind to a
name without belonging to any flow.
Stored beside the flows in the same repository, sharing their write lock
and commit, under a directory the flow listing ignores. No draft/publish
split: nothing executes a dashboard, so edit mode is its own staging area.
Two things it needs from the engine. A message catalog spanning every
flow, because a wall panel shows the heating next to the solar and the
flow-scoped API is the wrong shape for that. And a way to put a value in
without owning a node — a slider is a real value that happened to come
from a person — which runs whatever consumes it and applies the same type
check a node's output gets. Only a message some flow declares can be
published to; flows own the namespace.
Charts also need more past than the 120 points a sparkline wanted, so a
chart widget declares its depth and the engine keeps that message's
series that deep.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Every one of these could be written as a function node, and until now had
to be. But nine node types account for most of what an actual Node-RED
installation contains — inject alone is placed 58 times, change 62 — and
a rule you fill in reads better on a canvas, and survives editing better,
than the same five lines repeated eighty times.
Inject, switch, change, filter-unchanged, join, trigger, command, file
and ntfy. They configure themselves through the editor's generated form,
so none of them needed frontend work beyond an icon.
The stateful ones (filter-unchanged, trigger) keep what they remember in
the engine's own state under a reserved prefix, never as a message. That
is the line: code you write is a pure function of its inputs, node types
the engine ships may remember things.
Verified against a live instance — a hook feeding change into
filter-unchanged into switch scales a raw reading, swallows a change
inside the deadband, and routes the rest to the right branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Everything that goes wrong already travelled the event bus, but the only
subscriber was the editor's websocket — so a flow quarantined at three in
the morning was invisible until someone opened the browser.
An alert manager now watches the same bus and forwards failures to ntfy,
email or a webhook. Most of what it does is decline to send: the same
node failing every second is one alert with a count of what followed, a
connection flapping up and down is muted until it settles, and nothing
gets past ten notifications an hour. Verified against a live instance —
six identical failures produced one alert carrying the real traceback
message.
Channels and rules are configured through the API, with a test send so a
channel can be proven before anything depends on it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Execution was fire-and-forget: an MQTT message or webhook ran a cascade
on a ThreadPoolExecutor built for that one wave, and an engine that died
halfway through simply lost whatever was in flight. Concurrent triggers
each built their own pool, so load meant unbounded threads.
Every external trigger is now journaled to a Redis Streams queue before
anything runs, and acknowledged only once its cascade finishes. A
consumer thread drives cascades on one long-lived pool while node bodies
run on another, so a cascade cannot starve the nodes it is waiting for.
A reaper reclaims what a dead consumer never acknowledged — verified end
to end: work journaled while the engine was stopped runs on restart, and
work abandoned mid-cascade comes back as a second delivery.
At-least-once needs a guard, so nodes that reach outside are marked
non-idempotent and skipped on a redelivery they already completed.
Without Redis the queue degrades to an in-memory one that does not
pretend to be durable, and interactive callers still run inline.
Also fixes two things this turned up: a delay node was sleeping on a
worker thread, where a handful of them could occupy the whole pool, and
webhooks 404'd whenever MCP was enabled because the app mounted at /
answered first for every path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Logic nodes are pure functions with no state handle, but real automations
count things and remember the last reading. The shape for that is a
message a node both reads and writes: the graph already declines to make
a node depend on itself, so this worked by accident. It is now defined,
tested, and checked — a node that is the only writer of a message it
reads is told at edit time that it needs a starting value, rather than
silently never running.
Feeding a value back between two nodes was still a cycle, and rejected.
An input can now be marked non-triggering: read when the node runs, never
the reason it runs, and no dependency either way. That is what a back
edge actually means.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
stroblmeandClaude Opus 5