A second bus subscriber folds executions, errors, timings and queue lag
into per-minute rollups, keeps failures with their traceback and an audit
trail of who published what, and records one row per cascade — manual runs
and previews included, under an id of their own that writes no idempotency
markers. Read back through /observability/*, which always answers 200 so a
degraded engine still renders its own health screen.
Also fixes two things found on the way: node-health alerts read `status`
where the engine publishes `health`, so a device dropping never alerted
anyone, and the Redis queue reported `parked: 0` whatever was held.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
User code no longer execs in the engine. A pool of persistent worker
subprocesses speaks one JSON object per line; the controller installs a
proxy as the node's function, so every execution path funnels through it
and the pipeline is untouched. A crash costs one subprocess, a per-node
timeout is a kill, and cancelling from the canvas is that same kill.
The workers run a venv of the user's own on the data volume, filled from
a pip manifest versioned beside the flows. Applying it retires the
workers and rebuilds, so a package lands without restarting the engine.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
Moving a dashboard slider lit up an edge between two nodes that had done
nothing. The canvas pulsed on the message's timestamp alone, and a message
has no idea who published it — so it credited whichever node happened to
be drawn as a producer.
That was never only about dashboards. Two nodes producing one message
pulsed both their edges whichever fired, and a message produced in another
flow changed with nothing on screen to account for it at all.
Values now carry their cause: a node, a dashboard widget, another flow, an
agent or an API caller. An edge pulses only for the producer that actually
published, and the edge inspector says where a value came from when it did
not come from a node.
What is not a node in this flow is now drawn as one — a label rather than
a card, because a dashboard with twenty tiles would otherwise bury the
logic the canvas exists to show. That covers cross-flow wiring too, which
is the link in/out affordance that has been missing.
They are never part of the document. They join at render, after everything
that reads or writes the canvas nodes, so an autosave, an undo or a delete
cannot reach them — with a Playwright test that drags a node and asserts
the stored flow still holds exactly what it did.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Widgets bind to a message name and read it live off the socket the editor
already had — lifted out of the flow editor so a dashboard route gets the
same values, which also gives the home page live data for free.
The input widgets close the loop the other way: a slider publishes into
the graph and whatever consumes that message runs. Verified end to end in
the running app — moving a slider set a flow input, and the stat bound to
what the flow computed from it followed.
View mode is plain CSS grid. A wall panel that only displays should not
download the code that lets someone drag things around, and it now does
not. Editing is a widget picker, a per-widget width control and a
settings card fed by the message catalog.
No new dependencies: the slider is a range input, the gauge is an arc,
and the markdown is a five-line subset. Charts are the one widget still
missing — they need a charting library and the chart tokens the design
guidelines reserved — so they are stored and validated but not offered.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Everything that goes wrong already travelled the event bus, but the only
subscriber was the editor's websocket — so a flow quarantined at three in
the morning was invisible until someone opened the browser.
An alert manager now watches the same bus and forwards failures to ntfy,
email or a webhook. Most of what it does is decline to send: the same
node failing every second is one alert with a count of what followed, a
connection flapping up and down is muted until it settles, and nothing
gets past ten notifications an hour. Verified against a live instance —
six identical failures produced one alert carrying the real traceback
message.
Channels and rules are configured through the API, with a test send so a
channel can be proven before anything depends on it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
Execution was fire-and-forget: an MQTT message or webhook ran a cascade
on a ThreadPoolExecutor built for that one wave, and an engine that died
halfway through simply lost whatever was in flight. Concurrent triggers
each built their own pool, so load meant unbounded threads.
Every external trigger is now journaled to a Redis Streams queue before
anything runs, and acknowledged only once its cascade finishes. A
consumer thread drives cascades on one long-lived pool while node bodies
run on another, so a cascade cannot starve the nodes it is waiting for.
A reaper reclaims what a dead consumer never acknowledged — verified end
to end: work journaled while the engine was stopped runs on restart, and
work abandoned mid-cascade comes back as a second delivery.
At-least-once needs a guard, so nodes that reach outside are marked
non-idempotent and skipped on a redelivery they already completed.
Without Redis the queue degrades to an in-memory one that does not
pretend to be durable, and interactive callers still run inline.
Also fixes two things this turned up: a delay node was sleeping on a
worker thread, where a handful of them could occupy the whole pool, and
webhooks 404'd whenever MCP was enabled because the app mounted at /
answered first for every path.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
A node's subscription, schedule or poll loop was a bare asyncio task: one
that raised outside its own retry handling was simply gone, and the node
went on being listed as running while nothing listened any more.
Those loops now run under a supervisor that restarts them with a growing
delay and quarantines a flow that burns through five restarts in five
minutes — a flow crash-looping every second is worse than one that is
visibly stopped, and the dashboard can now say which.
The MQTT subscription loses its private five-second reconnect in the
process: one backoff policy per socket, and it belongs to the supervisor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
The API image ran four uvicorn workers, and each one built a full flow
controller — four sets of MQTT subscriptions, cron ticks and webhooks.
Runs one worker now; scaling out is the worker split, not more processes.
Adds a loop-lag watchdog and a deep /utils/health/ that fails when the
event loop is wedged or Redis is unreachable, the two failure modes a
process-alive check never sees. Autoheal restarts on that signal, behind
a compose profile because it mounts the Docker socket.
The private user-seeding routes now need an explicit opt-in rather than
just ENVIRONMENT=local, so a deployment that kept the default never
exposes them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
The engine now speaks MCP at /mcp, with a built-in OAuth 2.1 authorization
server in front of it: an agent registers itself, sends a human to the browser
to approve it, and exchanges the resulting code for a token. PKCE is required,
codes are single-use and stored only as hashes, the browser is redirected to
the URI that was registered rather than the one asked for, and refresh tokens
rotate so that replaying a spent one revokes the whole line.
Twenty tools cover reading, building, publishing and running flows, and each
one calls the same REST endpoint the dashboard calls, in-process, carrying the
caller's own token. That keeps one description of what a flow is and what may
be done to it — validation, the draft/publish split, the version check — and
means an agent can do nothing a person could not do in the browser.
Agent tokens are RS256 with a keypair of their own rather than the secret that
signs browser sessions, so deleting the key withdraws every agent without
logging anyone out, and deps.decode_token grew the branch that trusting a
second issuer will need when the hosted login arrives.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Connectors are the device-facing node class third parties write, so the
surface they build against is versioned and documented: ConnectorNode carries
a declared contract version, a polling loop that publishes only what changed
and reports health around it, and parameters whose credential fields are
marked x-secret so the editor offers the secrets store instead of a text box.
They are found through the fluksio.node_types entry point group, with the
package's own metadata as the manifest. docs/connectors/ has the contract and
the authoring guide; connector-skeleton/ is a working one to copy.
The controller no longer knows what any node type is: start, stop and
report_health are protocol methods on Node, and the built-ins were migrated to
them first, so the hooks a connector implements are the ones the engine has
been driving all along.
Marking a node reusable moves its source to _lib/ and points the node at it by
name. Other flows instantiate it with their own ports and settings, one fix
reaches all of them, and a shared source still in use cannot be deleted.
Ports gained an interval: an output publishes, and an input wakes its node, at
most every n seconds. State keeps the latest value, so only the delivery is
skipped, and pressing Run is never throttled.
Also fixes autosave sending no version on its first save of a session, which
made every flow saved more than once conflict with itself.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Flows can now be taken off the engine and put back. Stopped state lives in a
runtime.json beside the flow, not in the flow document: the canvas autosaves
that document, so a stopped flow would otherwise start itself again on the
next edit. A stopped flow gets no subscriptions, schedules or webhooks, its
nodes are skipped by the scheduler, and running it answers 409. Pausing holds
a flow's nodes while its values keep arriving, so the canvas still shows what
is coming in.
Node code is user code and print is how it says things, so stdout is teed
through a contextvar sink active only during a node execution — one event per
execution, capped, so a chatty node cannot outrun the stream. A node that
fails sends its traceback the same way, trimmed to the author's own frames.
The dock gains a logs panel and a pause control; the dashboard replaces its
placeholder with what is running, stopped or failing; the edge inspector can
send the last message again.
Single-stepping is deferred and noted: the scheduler keeps no progress between
calls, so a step button would re-run the same node rather than advance.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Edits autosave to flow.draft.json and nodes.draft/ instead of the files the
engine reads, so the pipeline keeps running the published version until
someone publishes. Every save carries the version it was based on: a second
client editing the same flow is refused with 409 and offered the choice
between their version and its own, rather than silently overwriting.
Draft saves no longer rebuild the pipeline; validation and node status for a
draft come from a throwaway build that never touches live state.
Also fixes a latent bug where an empty state backend is falsy, so Pipeline
quietly built itself a second, private state and left message history empty.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The browser half of M3. Flows open on a full-bleed canvas with their chrome
floating over it: flow tabs top, dock bottom, node settings in a panel on the
right that leaves the graph visible and running behind it.
- Connections are derived, not stored. A node declares the messages it reads
and publishes; every matching pair draws an edge, so two producers of one
message converge on their consumer. Dragging output to input is shorthand
for pointing that input at the producer's message, and asks before it
replaces an existing one.
- Values land on the edges as they flow, over a websocket that feeds a store
outside React, so a value arriving re-renders its own chip and nothing else.
Clicking an edge shows the last payload and when it arrived.
- Node source is edited in Monaco, loaded only when a panel opens and themed
from the design tokens.
- Edits autosave; identical documents are skipped server-side, so a quiet
canvas writes nothing.
- Validation from the API shows on the node it belongs to and is summarised in
the dock, where each entry pans to its node.
- Works on a phone: touch-connect, 44px dock targets, and the node panel
becomes a full-screen sheet.
Two new tokens (--status-success, --font-mono) are mirrored in the website repo
and recorded in DESIGN-GUIDELINES.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
The repo moves to git.stroblme.de:Fluksio/app.git and its sibling to
Fluksio/index.git, so every cross-repo path reference moves with it. The
architecture sources the roadmap cites now live in the docs repo under
architecture/, and the brand marks in the index repo's src/assets.
Also replaces the placeholder README with one that reflects the actual layout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ROADMAP.md was an empty scaffold carried over from another project, pointing
at a docs-private/architecture/ directory that does not exist. Rebuild it
from org/README.md and org/structure.canvas as five phases matching the
intended component split, and seed NOTEPAD.md with the findings from the
architecture survey (flow package not importable, tests outside tests/, CI,
chart tokens).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The submodule collapse was only half applied: .gitmodules was deleted but
backend/ and frontend/ were still recorded as gitlinks, so none of their
files were tracked. Replace the gitlinks with the real trees.
Also untrack .env (it carried placeholder secrets) in favour of a tracked
.env.example, drop the committed __pycache__, and narrow the blanket *.png
ignore that would have swallowed design assets.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
