nodes.py had grown to 2k lines holding every integration behind a single
blanket mypy exemption. It is now a package split by the outside world
each node talks to, so the exemption shrinks to the four integration
modules; base and mlp are type-checked, which turned up a dozen missing
annotations.
The senders opened a fresh connection — and, in the MQTT case, a fresh
thread pool and event loop — for every single message. HTTP senders now
share one pooled client, and a publisher holds one broker connection for
its lifetime, fed from a bounded queue that drops the oldest value when
the broker cannot keep up.
An HTTP sender also no longer trips over a JSON reply that is not an
object: outputs are keyed by port, so a bare scalar is a valid reply with
nothing to publish.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
A node's subscription, schedule or poll loop was a bare asyncio task: one
that raised outside its own retry handling was simply gone, and the node
went on being listed as running while nothing listened any more.
Those loops now run under a supervisor that restarts them with a growing
delay and quarantines a flow that burns through five restarts in five
minutes — a flow crash-looping every second is worse than one that is
visibly stopped, and the dashboard can now say which.
The MQTT subscription loses its private five-second reconnect in the
process: one backoff policy per socket, and it belongs to the supervisor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
The API image ran four uvicorn workers, and each one built a full flow
controller — four sets of MQTT subscriptions, cron ticks and webhooks.
Runs one worker now; scaling out is the worker split, not more processes.
Adds a loop-lag watchdog and a deep /utils/health/ that fails when the
event loop is wedged or Redis is unreachable, the two failure modes a
process-alive check never sees. Autoheal restarts on that signal, behind
a compose profile because it mounts the Docker socket.
The private user-seeding routes now need an explicit opt-in rather than
just ENVIRONMENT=local, so a deployment that kept the default never
exposes them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Connectors are the device-facing node class third parties write, so the
surface they build against is versioned and documented: ConnectorNode carries
a declared contract version, a polling loop that publishes only what changed
and reports health around it, and parameters whose credential fields are
marked x-secret so the editor offers the secrets store instead of a text box.
They are found through the fluksio.node_types entry point group, with the
package's own metadata as the manifest. docs/connectors/ has the contract and
the authoring guide; connector-skeleton/ is a working one to copy.
The controller no longer knows what any node type is: start, stop and
report_health are protocol methods on Node, and the built-ins were migrated to
them first, so the hooks a connector implements are the ones the engine has
been driving all along.
Marking a node reusable moves its source to _lib/ and points the node at it by
name. Other flows instantiate it with their own ports and settings, one fix
reaches all of them, and a shared source still in use cannot be deleted.
Ports gained an interval: an output publishes, and an input wakes its node, at
most every n seconds. State keeps the latest value, so only the delivery is
skipped, and pressing Run is never throttled.
Also fixes autosave sending no version on its first save of a session, which
made every flow saved more than once conflict with itself.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Flows can now be taken off the engine and put back. Stopped state lives in a
runtime.json beside the flow, not in the flow document: the canvas autosaves
that document, so a stopped flow would otherwise start itself again on the
next edit. A stopped flow gets no subscriptions, schedules or webhooks, its
nodes are skipped by the scheduler, and running it answers 409. Pausing holds
a flow's nodes while its values keep arriving, so the canvas still shows what
is coming in.
Node code is user code and print is how it says things, so stdout is teed
through a contextvar sink active only during a node execution — one event per
execution, capped, so a chatty node cannot outrun the stream. A node that
fails sends its traceback the same way, trimmed to the author's own frames.
The dock gains a logs panel and a pause control; the dashboard replaces its
placeholder with what is running, stopped or failing; the edge inspector can
send the last message again.
Single-stepping is deferred and noted: the scheduler keeps no progress between
calls, so a step button would re-run the same node rather than advance.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Edits autosave to flow.draft.json and nodes.draft/ instead of the files the
engine reads, so the pipeline keeps running the published version until
someone publishes. Every save carries the version it was based on: a second
client editing the same flow is refused with 409 and offered the choice
between their version and its own, rather than silently overwriting.
Draft saves no longer rebuild the pipeline; validation and node status for a
draft come from a throwaway build that never touches live state.
Also fixes a latent bug where an empty state backend is falsy, so Pipeline
quietly built itself a second, private state and left message history empty.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The panel is going to sparkline how a value moved, and nothing kept more
than the latest one. Emissions now append to a capped list beside the
value — 120 points, enough to fill a sparkline without making Redis a
time-series store — and `/flows/{flow}/history/{message}` hands it back
oldest first.
Only numbers are recorded, bools included as neither, so a string
message costs nothing at all. The response carries `numeric` so an empty
series reads as "not plottable" rather than "nothing yet". The append is
one pipelined round-trip per emission batch and sits outside the lock,
since the list is append-only.
MemoryState keeps the same window in a deque, so the endpoint answers
without Redis too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkmeRiyeYmVZqJVwuyHq9o
Trigger hooks are mounted unauthenticated because devices cannot present
a JWT. They now take a secret as a trailing path segment, so a device
needs one URL and no header support. The value never enters the
registered route, only a {secret} template, and is compared with
compare_digest; a mismatch is a bare 404 so the endpoint does not
confirm which hooks exist. Pointing the parameter at the encrypted store
keeps the literal out of flow.json. Hooks without a secret keep working
and now raise a validation issue saying so.
Rotating SECRET_KEY made the stored secrets unreadable for good, since
the Fernet key derives from it. scripts/rotate_secret_key.py re-encrypts
with the new key and refuses if the old one does not decrypt. Now that
recovery exists, an unreadable store fails loudly instead of coming back
empty and leaving flows short of credentials with no visible cause.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkmeRiyeYmVZqJVwuyHq9o
- One dot per node now carries the whole story: primary while running, sage
after a good run, red when anything is wrong, with the explanation on hover.
The corner badge is gone, along with the second way of saying the same thing.
- A node that publishes something flashes a ring, so a running flow is legible
without reading the edge values. Nodes that consume but publish nothing stay
quiet, which is why the event carries an output count.
- Flow settings open in the same panel its nodes use, from a pencil in the
dock: the title, the name, and deleting the flow. NodePanel and FlowPanel
share the panel chrome rather than each drawing their own.
- Renaming is a server operation, because a flow's name is the namespace of its
messages: the directory moves and every other flow reading `old.message` is
repointed, instead of being left pointing at a flow that no longer exists.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
- MQTT nodes failed to build: the topic map was renamed to talk in ports, but
__slots__ still declared the old name, so every MQTT node raised
AttributeError. Building one of each node type is now a test, since __slots__
makes this failure invisible until someone places the node.
- Port names offer the messages already in play: everything published is worth
reading, and an input nobody provides yet is worth publishing. A message only
connects when both ends spell it the same way, so choosing beats typing.
- Adding a port focuses its name field.
- Dragging onto an input that already reads something offers the extra port as
well as the replacement — an MQTT or InfluxDB node usually wants both.
- The template's Item model, its routes, screens and table are gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
The browser half of M3. Flows open on a full-bleed canvas with their chrome
floating over it: flow tabs top, dock bottom, node settings in a panel on the
right that leaves the graph visible and running behind it.
- Connections are derived, not stored. A node declares the messages it reads
and publishes; every matching pair draws an edge, so two producers of one
message converge on their consumer. Dragging output to input is shorthand
for pointing that input at the producer's message, and asks before it
replaces an existing one.
- Values land on the edges as they flow, over a websocket that feeds a store
outside React, so a value arriving re-renders its own chip and nothing else.
Clicking an edge shows the last payload and when it arrived.
- Node source is edited in Monaco, loaded only when a panel opens and themed
from the design tokens.
- Edits autosave; identical documents are skipped server-side, so a quiet
canvas writes nothing.
- Validation from the API shows on the node it belongs to and is summarised in
the dock, where each entry pans to its node.
- Works on a phone: touch-connect, 44px dock targets, and the node panel
becomes a full-screen sheet.
Two new tokens (--status-success, --font-mono) are mirrored in the website repo
and recorded in DESIGN-GUIDELINES.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
Makes the flow engine reachable from the API, which is what M3 needs before
any of it can reach the browser.
- app/flow is a package now; the prototype's watch-dir scripts and the
matplotlib/networkx visualiser are gone with their dependencies.
- Messages carry a serializable dtype instead of a live Python type, and a
port name, so the graph can speak qualified names while node functions keep
local arguments. Redis state is JSON, not pickle.
- Message names are namespaced per flow ("heating.temp"); a bare name resolves
to its own flow, a dotted one crosses flows.
- Several nodes may provide the same message: producers are a list, so fan-in
is a real edge instead of a silently dropped one.
- Flows are stored as flow.json plus node sources in a git repository, one
commit per save, with identical saves skipped so autosave stays quiet.
- Node failures are isolated and reported per node; validate() returns cycles
and unconnected inputs instead of raising deep in a run.
- Credentials live in an encrypted store and are referenced as {"$secret": …}.
- Engine events reach websocket clients through a bus, so values, node status
and execution show up live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
