Commit Graph
50 Commits
Author SHA1 Message Date
stroblmeandClaude Fable 5 774b03953a A node's numbers leave through its ports, not a logging call
The first cut had node code call fluksio.log_metric, which was a second,
undeclared way for data to leave a node: invisible to validation, absent from
the canvas, and stored where the graph could not see it. That is precisely the
MLflow discrepancy this framework exists to avoid, so it is gone.

A node that produces values over time is a generator. Every yield is a dict
keyed by output port, published the instant it happens — same port, same type
check, same place on the canvas as any other value — and what it returns is
its result. A port doing this declares stream: true, and a run keeps every
number one takes, so experiment tracking is a consequence of the graph rather
than an API beside it: a chart binds to a training curve the way it binds to a
temperature. fluksio.emit writes the same ports imperatively, for where a
yield cannot reach — inside a training framework's callback.

In a live flow an emission also wakes what is downstream, as a subscriber
publishing does; in a run it does not, because a run's graph is scheduled once
and mid-node cascades would leave 'finished' with nothing to mean. The
enqueued item carries no payload: the value is already in state, and one
carrying it would re-apply an old emission after the node returned.

Verified on the stack: 30 loss values arrived live on the flow socket during a
run, attributed to the node that produced them, and the same node run on the
remote worker streamed its curve back across the socket.

Also caches remote compile results per worker, so attaching a GPU box does not
put a network round trip in every rebuild.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 20:53:49 +02:00
stroblmeandClaude Fable 5 52d4de62c9 Benchmark the startup claim, and write down what a run is
The milestone is measured on being lighter than Kedro, so make bench-startup
measures it rather than asserting it: 61 ms from submit to result against
1110 ms for kedro run on a pipeline that does the same nothing. The difference
is not orchestration, it is that nothing is booted per run — on a 510-config
sweep that is about nine minutes of pure startup that never happens.

docs/flows/runs.md is the guide: batch flows, sweeps, reporting from inside a
node, artifacts, and the two sanctioned patterns for objects that cannot be
serialized — keep them in one node, or cross at a checkpoint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 18:00:36 +02:00
Melvin Strobl 129c92e3d8 link
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-18 14:22:47 +02:00
stroblme 887e552ce1 gc
Signed-off-by: stroblme <stroblme@posteo.de>
2026-08-17 20:07:56 +02:00
stroblmeandClaude Opus 5 bb90a24b90 Computed flow layout, and mobile written into the design
The canvas lays itself out: a layered graph, left to right on a desktop and
top to bottom on a phone, with room reserved for the value each edge carries.
Nodes cannot be dragged and `NodeDef.position` is gone from the document —
a graph nobody can arrange is one worth keeping small, which is what keeps
flows atomic. Endpoints join the same layout, so their lanes and the
localStorage that remembered where they were dragged go too.

Mobile, per the new Responsive section of DESIGN-GUIDELINES.md: the dock caps
its width and wraps instead of running off the screen, the dashboard stacks
into one column rather than shrinking a wall panel to a fifth of its size, and
Home stops widening its grid track past the viewport. A Playwright project at
a phone's width fails the build when a screen no longer fits.

Along the way: publish is the checkmark that was already there rather than a
button that appears and disappears, with discard beside it on both the flow
and the dashboard; the brain reveals a neuron's name on the first tap; and the
port sparklines get room to breathe.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDSXaRhvqHYNevgDGmNAto
2026-08-17 17:35:14 +02:00
stroblmeandClaude Opus 5 1c09b8209d dashboard: pace a querying chart by its window, and an example to evaluate it
A chart is drawn in buckets, and nothing it can show changes until the
bucket it is drawing closes — so the resolution sets the refresh rather
than a flat five-second floor. A week at quarter-hour buckets now asks
four times an hour instead of sixty, for the same picture. Leaving the
field empty follows the window; a slower rate is still honoured.

`make seed-example` builds the thing to evaluate it with: a flow that
logs a temperature to InfluxDB, a flow that answers a chart's request by
turning the window into Flux and the rows back into a series, and a
dashboard holding the chart. The reading flow declares the request as an
input with a starting value, which is how a flow says a value reaches it
from a panel rather than from a node upstream.

Axis labels keep enough decimals to stay distinct — `si` rounds to three
figures, so every tick of a chart living inside one degree read "19".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:34:32 +02:00
stroblmeandClaude Opus 5 61bfd68f26 docs: record the structured-dtype and querying-chart decisions
Also the one the session settled by hand: database nodes stay transport
and credentials, and the Python nodes either side do the query building
and the answer shaping. That is what a series read mode inside the node
would have prevented, so the read-mode item goes with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:15:28 +02:00
stroblmeandClaude Opus 5 1a589570b1 Brain: dots where a connection meets a neuron, inbound hollow and outbound filled
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 12:52:31 +02:00
stroblmeandClaude Fable 5 02d841e10c Note the database review findings in the NOTEPAD
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 12:40:36 +02:00
stroblmeandClaude Opus 5 9351a86eec Overviews: icon toolbar, dashboard drafts, publish all
Both overviews carried the same toolbar twice, left-aligned, with a search
field permanently taking a row of width. One `OverviewToolbar` now serves
them: the search folds into an icon and expands again on click (Escape puts
it away and hands focus back), create is a `+`, and everything sits right of
the page. Each page keeps its own create dialog — the toolbar only renders
the trigger — so the testids the runtime spec and the capture script drive
stayed where they were.

Dashboards get the flow store's draft/publish split. The editor autosaves
`dashboard.draft.json` beside `dashboard.json`; `/view/{name}`, `bindings_for`
and `history_requirements` keep reading the published file, so a wall panel
sees an edit only once someone publishes it. `POST /dashboards/{name}/publish`
and `/discard` mirror the flow routes down to the version precondition and the
409, `GET /dashboards/{name}?draft=true` is what the editor asks for, and the
dock grows the same Publish button — which flushes a queued save first, so an
autosave in flight is not published around. Creating a dashboard still writes
the published file directly: an empty document on a panel is harmless, and it
keeps the store free of a never-published case.

"Publish all" is a checkmark in the toolbar, live only when something actually
has `has_draft`. A summary carries no version and publish needs the one it is
based on, so each document's detail is read immediately before its publish —
honest against a stale list, and no version-less backend path to maintain.
Failures are counted rather than swallowed: three of five fails says so and
names the three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:57:11 +02:00
stroblmeandClaude Opus 5 683c25b26d Home: one time range across the health block, and failures that follow it
The health screen was fixed at 24 hours everywhere except its lists, which
were fixed at nothing: `failuresQueryOptions` read the newest 100 rows and
`HealthActivity` filtered them client-side, so on a busy engine the failures
list covered whatever few minutes 100 rows happened to span while the chart
beside it spanned a day — and pinning an older minute showed an empty card.

One `RangePicker` now sits on the Health heading and governs the whole block:
the tiles, the flow table, both charts and both lists. Presets are 1h / 6h /
24h / 7d — the collector prunes at `OBS_RETENTION_DAYS` (30), so a week is
behind the last one. The longer windows ask for coarser buckets, since a week
of minute rollups is ten thousand points nobody can see.

Failures get the escape hatch the runs already had: a pinned minute is asked
for with `since`/`until` rather than filtered out of what is held, and the
list itself is bound to the selected range. `RUN_DEPTH`/`EVENT_DEPTH` become
one `LIST_DEPTH`, which now buys coverage of the window on screen instead of
a fixed newest-N — narrowing the range is what makes the same rows reach the
whole of it. The "Failures (24h)" tile counts errors over the selected window
from the rollups the table is drawn from, so tile, column and chart agree.

The node-panel and edge trend curves get no picker. They are a Redis ring of
the last 120 values per message with no window to ask for, so hovering one
reveals what it actually shows — how many readings, and the span they cover.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:51:42 +02:00
stroblmeandClaude Opus 5 69421b857d Brain: hover labels, activity falloff, rim-anchored directed edges
A name under every circle was most of what the graph drew, and none of it was
what someone looks at the brain for. Only the hovered or keyboard-focused
neuron names itself now — a failing one always does, since colour is never the
only carrier of a status. The label is absolute and the collide radius already
held that space, so revealing one moves nothing. Taking it out of flow also
fixed the off-centre edges: the node box was as wide as its widest label, so a
32px circle inside a 140px box sat 54px left of where the layout put it and
every edge aimed at the box rather than at the circle.

Neurons are filled discs rather than `bg-card` outlines, which were a hairline
against `--background` in light and close to nothing in dark. The fill is a
neutral `--muted-foreground` alpha and stays neutral: status still speaks only
through the border and the word beneath. Its lightness carries how recently the
open page saw that neuron publish — brightest just after it fires, decaying over
90s to a floor it never drops below, so a graph left open sorts itself into what
is busy and what is not. Edges do the same on their stroke. The signal is only
what this session has seen on the socket, so a fresh page starts everything at a
neutral middle rather than claiming a history it does not have. Both are CSS
transitions gated on `prefers-reduced-motion`, decayed out of and snapped into,
never the other way round.

`BrainEdge` now carries both circle radii and trims its path back to the rims,
with an arrowhead at the end it flows into. The barbs are two more segments of
the same stroked path rather than an SVG marker, so they dim with the line they
belong to. Neither trim reaches past the midpoint, so neurons closer together
than their radii still get a line pointing the right way.

`edge-pulse` reads its landing colour from `--edge-rest` instead of naming
`--muted-foreground`, so a selected edge — which rests in blue — decays into its
own colour instead of crossing to grey and snapping back at the end.

Obsidian's graph view is the reference for the first three: filled nodes sized
by degree, labels on a fade threshold with hover to recover them, and arrows as
an explicit affordance. The falloff-by-opacity encoding is the standard one in
the dynamic-graph literature.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:31:13 +02:00
stroblmeandClaude Opus 5 78c605bd37 Home: one sparkline style with a left fade, SI values, a full-width activity table, roomier panels
The trend curve was two components: a rich one in the node panel and on edges,
and a bare line in the flow table. It is one `Sparkline` now, taking the colour
token, the height and whether the live dot and the readout show. The flow table
draws its rollup in the chart ramp with no dot — the rollups are polled, so the
right edge is the last completed slice rather than this instant — and keeps its
"nothing yet" state, as the panel keeps its three distinct silences.

All of them fade out to the left, through an SVG mask over the curve and its
area. The dot sits outside the mask: the newest reading is the one thing that
must stay solid.

`si` replaces `compact` and the ad-hoc "k" the uPlot axis carried. It prefixes
k/M/G and m/µ, but only outside 0.01–1000, where the plain number is already
the shortest thing to read and a written unit ("0.4 ms") stays honest. The
sparkline readout asks for four digits, so two neighbouring readings never
collapse into one string. Exact counts and anything the user acts on — a
payload, a form field, the edge inspector's value — are left unrounded.
`src/lib/utils.check.ts` asserts the rounding cases.

The activity table now spans its card: the flow name anchors the left, the four
numbers read down their own centre, and the trend takes the slack on the right.
Panel rhythm steps up one notch, gap-5 to gap-6 outside and gap-2 to gap-3
within a section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:27:54 +02:00
stroblme 5483de13c0 NOTEPAD: components needed for wall-panel parity with the current home dashboard 2026-08-17 09:54:28 +02:00
stroblmeandClaude Fable 5 0832be32a9 Note the run-list cap a busy pinned minute would hit
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-17 00:29:02 +02:00
stroblmeandClaude Fable 5 0b91379914 Let a pinned minute read runs from the whole day
/observability/runs gains since/until, so the throughput chart's pin asks
the server for its minute instead of filtering a fixed recent list. This
engine writes ~60 runs a minute, so any minute but the newest read empty.

since is inclusive and until exclusive, matching the minute buckets the
charts are drawn from. Hover stays the client-side preview it was:
scrubbing a day would otherwise be a request per minute rested on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-17 00:27:40 +02:00
stroblmeandClaude Fable 5 affb0a5f8c Fold the brain and health screens into Home
Home is the one overview now: the brain graph flat across the top, the flow
switches, then the health sections. The /brain and /health routes and their
sidebar entries are gone.

- charts report their cursor and clicks, so hovering one filters the list
  beside it to that minute and a click pins it until Escape or Clear
- chart values round to about three significant digits, the legend mounts
  under the plot so it can wrap without leaving the card, and axis ticks
  shorten past a thousand
- the embedded brain leaves the wheel to the page rather than zooming
- number fields no longer draw their up/down spinner (NOTEPAD)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-17 00:17:14 +02:00
stroblmeandClaude Fable 5 83c30aa1c7 Take the engine off the path node code imports from, and let it stop
The worker script is handed to the interpreter by path, so app/flow was
sys.path[0] for every node: `import queue` got the engine's. It now drops
its own directory before anything else imports, and runs with the
deployment's credentials scrubbed out of its environment.

Also: reload builds off the event loop, the pool wakes what is blocked on
it when it stops, a refused metrics flush is kept for the next one rather
than dropped, and the cascade events are paired through failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 23:46:08 +02:00
stroblmeandClaude Fable 5 b33be3fdd4 Load the engine, then take its pieces away and check what survives
A standalone harness, never part of the test run, that drives a real stack
through the durable path — the webhook, which journals every trigger — and
then stops Redis, kills the engine mid-cascade and restarts the broker
under a live subscription. Latest-value-wins with concurrent cascades means
counter equality is not a promise, so what it asserts is that the queue
drains, that state ends on the last value sent, and that nothing reached
the dead-letter stream.

Every docker verb goes through one helper that checks the compose label
before it acts and refuses anything outside this project, because the
machines this runs on host unrelated services.

Two invariants are deliberately looser than they look. The queue belongs to
the whole stack, so "nothing pending" would be measuring other people's
traffic; the harness waits on the age of the oldest unacknowledged entry
instead, which a stuck item always dominates. And the observability tables
are cleared only after the collector's flush interval has passed, since
deleting a flow publishes an audit event of its own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 23:14:14 +02:00
stroblmeandClaude Fable 5 4bcd38354b Draw every flow as one graph, merged on what it talks to
A node type can now say which outside thing its parameters point at, and
nodes sharing one — a broker topic, a URL, a bucket — are drawn as a single
neuron on a new /brain canvas. That makes the wiring which runs between
flows through a broker visible for the first time; no single flow's canvas
can show it. The key is read off stored parameters, so a credential
reference never reaches an id.

Layout is a d3 force simulation settled once and then frozen, lit by the
socket the editor already listens to: a neuron pulses when any node behind
it publishes, and its connections light as values pass.

Fixes the message pulse while here: interpolating the stroke against the
edge's `color-mix()` resting colour went through oklab and left the gamut,
which turned every pulse on both canvases fluorescent yellow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 22:59:31 +02:00
stroblmeandClaude Fable 5 af3ba51571 Keep the engine's own history, and a screen that reads it
A second bus subscriber folds executions, errors, timings and queue lag
into per-minute rollups, keeps failures with their traceback and an audit
trail of who published what, and records one row per cascade — manual runs
and previews included, under an id of their own that writes no idempotency
markers. Read back through /observability/*, which always answers 200 so a
degraded engine still renders its own health screen.

Also fixes two things found on the way: node-health alerts read `status`
where the engine publishes `health`, so a device dropping never alerted
anyone, and the Redis queue reported `parked: 0` whatever was held.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 22:29:32 +02:00
stroblmeandClaude Fable 5 f300c43f3a Run python nodes out of process, with modules of their own
User code no longer execs in the engine. A pool of persistent worker
subprocesses speaks one JSON object per line; the controller installs a
proxy as the node's function, so every execution path funnels through it
and the pipeline is untouched. A crash costs one subprocess, a per-node
timeout is a kill, and cancelling from the canvas is that same kill.

The workers run a venv of the user's own on the data volume, filled from
a pip manifest versioned beside the flows. Applying it retires the
workers and rebuilds, so a package lands without restarting the engine.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 21:43:36 +02:00
stroblme 979c9d3c1f Note that a node's error status clears before anyone sees it 2026-08-16 19:32:10 +02:00
stroblmeandClaude Fable 5 49a3f3da01 Note the canvas follow-ups the review turned up
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 19:16:06 +02:00
stroblmeandClaude Fable 5 63531fa36e Record what the package sweep landed and what it turned up
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 17:28:55 +02:00
stroblmeandClaude Fable 5 19970b0ad8 Group the notepad into work packages, move two reliability items to M5
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 16:20:48 +02:00
stroblmeandClaude Fable 5 c70359af67 Let the canvas endpoints be moved, and lift them on hover
Three things they were missing: they could not be dragged out of the lane
they were placed in, they stayed the same weight whether or not you were
reaching for one, and the label sat right against its connector dot.

Where one has been dragged to is a view preference, not part of the flow —
writing a position for a dashboard widget into flow.json would be a lie
about what the document holds — so it lives in the browser, keyed by flow.

Also folds the dashboard shot into `make verify` and drops the one-off
scripts that had accumulated beside it; the durable coverage is the
Playwright specs, which all pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 16:07:07 +02:00
stroblmeandClaude Fable 5 75c26ef000 Say what caused a value, and draw what is not a node
Moving a dashboard slider lit up an edge between two nodes that had done
nothing. The canvas pulsed on the message's timestamp alone, and a message
has no idea who published it — so it credited whichever node happened to
be drawn as a producer.

That was never only about dashboards. Two nodes producing one message
pulsed both their edges whichever fired, and a message produced in another
flow changed with nothing on screen to account for it at all.

Values now carry their cause: a node, a dashboard widget, another flow, an
agent or an API caller. An edge pulses only for the producer that actually
published, and the edge inspector says where a value came from when it did
not come from a node.

What is not a node in this flow is now drawn as one — a label rather than
a card, because a dashboard with twenty tiles would otherwise bury the
logic the canvas exists to show. That covers cross-flow wiring too, which
is the link in/out affordance that has been missing.

They are never part of the document. They join at render, after everything
that reads or writes the canvas nodes, so an autosave, an undo or a delete
cannot reach them — with a Playwright test that drags a node and asserts
the stored flow still holds exactly what it did.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 15:51:54 +02:00
stroblmeandClaude Fable 5 4de3cd7c66 Record the connectors and what is still read-only
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 10:00:10 +02:00
stroblmeandClaude Fable 5 be6d4fc13c Build dashboards you can actually look at and press
Widgets bind to a message name and read it live off the socket the editor
already had — lifted out of the flow editor so a dashboard route gets the
same values, which also gives the home page live data for free.

The input widgets close the loop the other way: a slider publishes into
the graph and whatever consumes that message runs. Verified end to end in
the running app — moving a slider set a flow input, and the stat bound to
what the flow computed from it followed.

View mode is plain CSS grid. A wall panel that only displays should not
download the code that lets someone drag things around, and it now does
not. Editing is a widget picker, a per-widget width control and a
settings card fed by the message catalog.

No new dependencies: the slider is a range input, the gauge is an arc,
and the markdown is a five-line subset. Charts are the one widget still
missing — they need a charting library and the chart tokens the design
guidelines reserved — so they are stored and validated but not offered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 09:33:02 +02:00
root 80acf342f4 Merge branch 'main' of git.stroblme.de:Fluksio/app
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
2026-08-16 08:59:41 +02:00
rootandClaude Fable 5 fef0545ae4 Let a function node's own settings be edited
A python node's params reach process() as whatever its author put there,
but there was no way to put anything there: the settings form is built
from a type's declared schema, and a function node declares none.

Node types now say whether they take settings beyond their schema, and
the panel offers a key/value editor for the ones that do — named, typed
as text, number, on/off or JSON, and laid out like the port list beside
it. Rows are keyed by position rather than by name, so renaming a setting
does not remount the row and lose what was being typed into it.

Verified in the running app in both themes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 08:20:17 +02:00
rootandClaude Fable 5 dbdbcc1091 Tell someone when the engine breaks
Everything that goes wrong already travelled the event bus, but the only
subscriber was the editor's websocket — so a flow quarantined at three in
the morning was invisible until someone opened the browser.

An alert manager now watches the same bus and forwards failures to ntfy,
email or a webhook. Most of what it does is decline to send: the same
node failing every second is one alert with a count of what followed, a
connection flapping up and down is muted until it settles, and nothing
gets past ten notifications an hour. Verified against a live instance —
six identical failures produced one alert carrying the real traceback
message.

Channels and rules are configured through the API, with a test send so a
channel can be proven before anything depends on it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 08:01:50 +02:00
rootandClaude Fable 5 04329149b3 Journal work before running it, so a crash stops losing messages
Execution was fire-and-forget: an MQTT message or webhook ran a cascade
on a ThreadPoolExecutor built for that one wave, and an engine that died
halfway through simply lost whatever was in flight. Concurrent triggers
each built their own pool, so load meant unbounded threads.

Every external trigger is now journaled to a Redis Streams queue before
anything runs, and acknowledged only once its cascade finishes. A
consumer thread drives cascades on one long-lived pool while node bodies
run on another, so a cascade cannot starve the nodes it is waiting for.
A reaper reclaims what a dead consumer never acknowledged — verified end
to end: work journaled while the engine was stopped runs on restart, and
work abandoned mid-cascade comes back as a second delivery.

At-least-once needs a guard, so nodes that reach outside are marked
non-idempotent and skipped on a redelivery they already completed.
Without Redis the queue degrades to an in-memory one that does not
pretend to be durable, and interactive callers still run inline.

Also fixes two things this turned up: a delay node was sleeping on a
worker thread, where a handful of them could occupy the whole pool, and
webhooks 404'd whenever MCP was enabled because the app mounted at /
answered first for every path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 07:57:07 +02:00
rootandClaude Fable 5 f8693daad6 Split the node types into a package, and stop reconnecting per message
nodes.py had grown to 2k lines holding every integration behind a single
blanket mypy exemption. It is now a package split by the outside world
each node talks to, so the exemption shrinks to the four integration
modules; base and mlp are type-checked, which turned up a dozen missing
annotations.

The senders opened a fresh connection — and, in the MQTT case, a fresh
thread pool and event loop — for every single message. HTTP senders now
share one pooled client, and a publisher holds one broker connection for
its lifetime, fed from a bounded queue that drops the oldest value when
the broker cannot keep up.

An HTTP sender also no longer trips over a JSON reply that is not an
object: outputs are keyed by port, so a bare scalar is a valid reply with
nothing to publish.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 07:33:10 +02:00
Melvin Strobl f6e6f1b0d5 comments
pre-commit / pre-commit (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 0s
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-16 00:34:45 +02:00
Melvin StroblandClaude Fable 5 8d82d6c4ec Let agents drive the flow API over MCP
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
The engine now speaks MCP at /mcp, with a built-in OAuth 2.1 authorization
server in front of it: an agent registers itself, sends a human to the browser
to approve it, and exchanges the resulting code for a token. PKCE is required,
codes are single-use and stored only as hashes, the browser is redirected to
the URI that was registered rather than the one asked for, and refresh tokens
rotate so that replaying a spent one revokes the whole line.

Twenty tools cover reading, building, publishing and running flows, and each
one calls the same REST endpoint the dashboard calls, in-process, carrying the
caller's own token. That keeps one description of what a flow is and what may
be done to it — validation, the draft/publish split, the version check — and
means an agent can do nothing a person could not do in the browser.

Agent tokens are RS256 with a keypair of their own rather than the secret that
signs browser sessions, so deleting the key withdraws every agent without
logging anyone out, and deps.decode_token grew the branch that trusting a
second issuer will need when the hosted login arrives.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 00:22:41 +02:00
Melvin StroblandClaude Fable 5 3724b68f23 Add the connector contract, reusable nodes and per-port intervals
Connectors are the device-facing node class third parties write, so the
surface they build against is versioned and documented: ConnectorNode carries
a declared contract version, a polling loop that publishes only what changed
and reports health around it, and parameters whose credential fields are
marked x-secret so the editor offers the secrets store instead of a text box.
They are found through the fluksio.node_types entry point group, with the
package's own metadata as the manifest. docs/connectors/ has the contract and
the authoring guide; connector-skeleton/ is a working one to copy.

The controller no longer knows what any node type is: start, stop and
report_health are protocol methods on Node, and the built-ins were migrated to
them first, so the hooks a connector implements are the ones the engine has
been driving all along.

Marking a node reusable moves its source to _lib/ and points the node at it by
name. Other flows instantiate it with their own ports and settings, one fix
reaches all of them, and a shared source still in use cannot be deleted.

Ports gained an interval: an output publishes, and an input wakes its node, at
most every n seconds. State keeps the latest value, so only the delivery is
skipped, and pressing Run is never throttled.

Also fixes autosave sending no version on its first save of a session, which
made every flow saved more than once conflict with itself.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 23:57:44 +02:00
Melvin StroblandClaude Fable 5 7344eac262 Start, stop and pause flows, and show what their nodes print
Flows can now be taken off the engine and put back. Stopped state lives in a
runtime.json beside the flow, not in the flow document: the canvas autosaves
that document, so a stopped flow would otherwise start itself again on the
next edit. A stopped flow gets no subscriptions, schedules or webhooks, its
nodes are skipped by the scheduler, and running it answers 409. Pausing holds
a flow's nodes while its values keep arriving, so the canvas still shows what
is coming in.

Node code is user code and print is how it says things, so stdout is teed
through a contextvar sink active only during a node execution — one event per
execution, capped, so a chatty node cannot outrun the stream. A node that
fails sends its traceback the same way, trimmed to the author's own frames.
The dock gains a logs panel and a pause control; the dashboard replaces its
placeholder with what is running, stopped or failing; the edge inspector can
send the last message again.

Single-stepping is deferred and noted: the scheduler keeps no progress between
calls, so a step button would re-run the same node rather than advance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 23:35:08 +02:00
Melvin StroblandClaude Fable 5 606ab3c423 Separate editing from running with a draft/publish split
Edits autosave to flow.draft.json and nodes.draft/ instead of the files the
engine reads, so the pipeline keeps running the published version until
someone publishes. Every save carries the version it was based on: a second
client editing the same flow is refused with 409 and offered the choice
between their version and its own, rather than silently overwriting.

Draft saves no longer rebuild the pipeline; validation and node status for a
draft come from a throwaway build that never touches live state.

Also fixes a latent bug where an empty state backend is falsy, so Pipeline
quietly built itself a second, private state and left message history empty.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 23:13:15 +02:00
Melvin StroblandClaude Opus 5 36be6f1081 Drop the InfluxDB rotation item, note the orphaned flow state
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
The token does not need rotating. Deleting a flow does leave its value,
timestamp, version and history keys behind in Redis.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkmeRiyeYmVZqJVwuyHq9o
2026-08-15 22:07:31 +02:00
Melvin StroblandClaude Opus 5 881d5c1cf2 Clear the notepad of what this pass closed
Records what the work turned up on the way: no way to reach a stored
secret from the params form, flow-level edits outside the undo stack,
the mobile panel having no test id, and the Playwright specs leaving
their users and flows in the development database.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkmeRiyeYmVZqJVwuyHq9o
2026-08-15 22:00:18 +02:00
Melvin StroblandClaude Fable 5 aa0e760615 Give the pulse its own duration, and let the editor fill the screen
The shared duration tokens had been stretched to slow the pulses down, which
also slowed every panel and dock animation and left `motion.ts` out of step
with the CSS. Both are back to 200/300 ms, and the pulses use a new
`--duration-pulse` (500 ms): a message arriving is a signal, not a state
change, and it is the only thing here that should linger. The edge pulse read
its own 300 ms constant, so it now takes the token through `motion.ts` and the
two sides cannot drift again.

The code editor also expands: the same node panel fills the content region
beside the sidebar, the canvas chrome steps aside rather than floating on top,
and the settings above the editor keep a readable width.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
2026-08-15 20:03:21 +02:00
Melvin StroblandClaude Fable 5 f05831b4a5 Note the motion token drift between CSS and motion.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
2026-08-15 19:47:09 +02:00
Melvin StroblandClaude Fable 5 6fb42bb3ef Note the test-teardown and token findings
Running the backend suite against the development database wipes its users,
and the old InfluxDB token turns out to be in pushed history rather than only
in the working tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
2026-08-15 18:50:46 +02:00
Melvin StroblandClaude Fable 5 8c82549cf6 Add the flow editor: canvas, node panel and live values
The browser half of M3. Flows open on a full-bleed canvas with their chrome
floating over it: flow tabs top, dock bottom, node settings in a panel on the
right that leaves the graph visible and running behind it.

- Connections are derived, not stored. A node declares the messages it reads
  and publishes; every matching pair draws an edge, so two producers of one
  message converge on their consumer. Dragging output to input is shorthand
  for pointing that input at the producer's message, and asks before it
  replaces an existing one.
- Values land on the edges as they flow, over a websocket that feeds a store
  outside React, so a value arriving re-renders its own chip and nothing else.
  Clicking an edge shows the last payload and when it arrived.
- Node source is edited in Monaco, loaded only when a panel opens and themed
  from the design tokens.
- Edits autosave; identical documents are skipped server-side, so a quiet
  canvas writes nothing.
- Validation from the API shows on the node it belongs to and is summarised in
  the dock, where each entry pans to its node.
- Works on a phone: touch-connect, 44px dock targets, and the node panel
  becomes a full-screen sheet.

Two new tokens (--status-success, --font-mono) are mirrored in the website repo
and recorded in DESIGN-GUIDELINES.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
2026-08-15 18:10:50 +02:00
Melvin StroblandClaude Fable 5 06a4506767 Add the flow API: typed messages, git-backed store, REST and live events
Makes the flow engine reachable from the API, which is what M3 needs before
any of it can reach the browser.

- app/flow is a package now; the prototype's watch-dir scripts and the
  matplotlib/networkx visualiser are gone with their dependencies.
- Messages carry a serializable dtype instead of a live Python type, and a
  port name, so the graph can speak qualified names while node functions keep
  local arguments. Redis state is JSON, not pickle.
- Message names are namespaced per flow ("heating.temp"); a bare name resolves
  to its own flow, a dotted one crosses flows.
- Several nodes may provide the same message: producers are a list, so fan-in
  is a real edge instead of a silently dropped one.
- Flows are stored as flow.json plus node sources in a git repository, one
  commit per save, with identical saves skipped so autosave stays quiet.
- Node failures are isolated and reported per node; validate() returns cycles
  and unconnected inputs instead of raising deep in a run.
- Credentials live in an encrypted store and are referenced as {"$secret": …}.
- Engine events reach websocket clients through a bus, so values, node status
  and execution show up live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
2026-08-15 17:23:36 +02:00
Melvin StroblandClaude Opus 5 4aa2705d34 Rename the workspace layout: fluksio -> app, website -> index
The repo moves to git.stroblme.de:Fluksio/app.git and its sibling to
Fluksio/index.git, so every cross-repo path reference moves with it. The
architecture sources the roadmap cites now live in the docs repo under
architecture/, and the brand marks in the index repo's src/assets.

Also replaces the placeholder README with one that reflects the actual layout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 16:16:08 +02:00
Melvin StroblandClaude Opus 5 396d7593eb Rewrite ROADMAP and NOTEPAD from the real requirement sources
ROADMAP.md was an empty scaffold carried over from another project, pointing
at a docs-private/architecture/ directory that does not exist. Rebuild it
from org/README.md and org/structure.canvas as five phases matching the
intended component split, and seed NOTEPAD.md with the findings from the
architecture survey (flow package not importable, tests outside tests/, CI,
chart tokens).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 15:36:47 +02:00
Melvin StroblandClaude Opus 5 1916f7f778 Vendor backend and frontend into the monorepo
The submodule collapse was only half applied: .gitmodules was deleted but
backend/ and frontend/ were still recorded as gitlinks, so none of their
files were tracked. Replace the gitlinks with the real trees.

Also untrack .env (it carried placeholder secrets) in favour of a tracked
.env.example, drop the committed __pycache__, and narrow the blanket *.png
ignore that would have swallowed design assets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 15:14:54 +02:00