Commit Graph
138 Commits
Author SHA1 Message Date
stroblmeandClaude Fable 5 a4dae864e5 Let the worker agent run straight from a checkout
It expects worker_main.py beside it, which is true on a worker host and not in
a clone — so trying a remote worker started with copying files around. It now
looks one directory over as well.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 18:03:28 +02:00
stroblmeandClaude Fable 5 c90b7a4a94 Benchmark the startup claim, and write down what a run is
The milestone is measured on being lighter than Kedro, so make bench-startup
measures it rather than asserting it: 61 ms from submit to result against
1110 ms for kedro run on a pipeline that does the same nothing. The difference
is not orchestration, it is that nothing is booted per run — on a 510-config
sweep that is about nine minutes of pure startup that never happens.

docs/flows/runs.md is the guide: batch flows, sweeps, reporting from inside a
node, artifacts, and the two sanctioned patterns for objects that cannot be
serialized — keep them in one node, or cross at a checkpoint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 18:00:36 +02:00
stroblmeandClaude Fable 5 e302ba1a43 Remote workers: a GPU box dials in and runs the nodes bound to it
The engine runs where the automations are and the GPU is somewhere else,
usually behind a different network — so the worker connects out and the engine
answers over the socket it was given. Nothing has to expose Redis, and the
same connection works through the tunnel the hosted access will use.

What travels is the protocol the local pool already speaks, so a node cannot
tell which kind of worker it is on. A node declares device: gpu and
device_policy, the label is resolved per call (a worker attaching later needs
no rebuild), and a run whose labels nothing carries waits in the queue saying
what it waits for rather than failing — submit from the couch, the GPU box
picks it up when it is switched on.

Two things had to move with it. Compiling now happens on the machine that will
run the node: a node importing torch is correct on the GPU box and a missing
module on the engine, so checking it here failed nodes that were fine. And the
artifact endpoint accepts a worker's own credential, because storing a
checkpoint is exactly what that credential is for — and only that.

Verified against the real split: the training ran on this host (its checkpoint
names the machine and a numpy the engine does not have), streamed 40 metric
points back mid-run, and the evaluate node read the checkpoint on the engine.
Cancel kills the remote training; pulling the worker fails the run in six
seconds instead of waiting out its ten-minute timeout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 17:52:42 +02:00
stroblmeandClaude Fable 5 b1cb8b41bc Artifacts: bytes a node produced, addressed by their content
A checkpoint is not a message. DType.ARTIFACT carries a reference — digest,
size, media type, name — so everything on the wire stays JSON and thirty
megabytes never sit in Redis, which answers the vision's open binary-payload
question by narrowing it: inline codecs would only serve payloads too small to
be worth a round trip, and nothing asks for that.

The store is content-addressed rather than per-run, for three reasons that all
pay later: a sweep whose fifty configs share one preprocessed input stores it
once, a reference stays valid however it is passed around because it names
content instead of a location, and the digest is what a stage cache will
compare — so building it in now is what keeps that from being a change to the
message contract.

Node code calls fluksio.save_artifact/load_artifact and cannot tell whether it
is writing the engine's own directory or putting bytes over HTTP, which is
what will let the same flow run on a remote worker unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 17:09:27 +02:00
stroblmeandClaude Fable 5 3f5bb12ed0 Runs report while they run, and the reader frames lines properly
A training loop has numbers worth keeping thousands of steps before it has a
result. Node code now imports fluksio and calls log_metric/progress, which
sends a line back without ending the call; the engine writes those to
run_metric in batches from the run's own driver rather than folding them off
the event bus, which drops what it cannot keep up with.

Two things fall out. Each event resets the worker deadline, so a node's
timeout measures silence rather than duration — which is what lets a two-hour
training keep a liveness contract instead of racing it. And the worker pool's
_running is now keyed by (run, node), so cancelling one config of a sweep
kills that training and leaves the rest alone.

Fixes a latent framing bug: read_line returned whatever a read had taken,
which was fine while a worker only ever sent one line per request and
unparseable as soon as it sent several. It now keeps the remainder.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 17:02:03 +02:00
stroblmeandClaude Fable 5 2dd9ce28f6 Runs: a flow taken from its inputs to its outputs, once
A cascade has no end worth recording; a run does. Parameters go in, the graph
executes until it drains, and the result is kept — which is what an ML
experiment is and what a CI-style job is, so both are one entity.

Each run gets a state backend namespaced to itself, so two runs of one flow
cannot overwrite each other's messages; that is a constructor argument rather
than a change to the pipeline, because every key the engine keeps already goes
through the state backend. Its record is written by the driver thread rather
than folded off the event bus, which drops what it cannot keep up with. Its
own Redis stream wakes an engine up, and from the claim onwards the database
row is the truth: redelivering hours of training because an acknowledgement
was late is not recovery, so a stale lease is what marks a run whose engine
died.

Flows gain mode: batch, which are built and validated but never activated, and
nodes gain a device label for the worker that must run them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
2026-08-18 16:55:29 +02:00
Melvin Strobl 76dbedd0f7 link
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-18 14:22:47 +02:00
Melvin Strobl a03d17975f analytics
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-18 13:17:07 +02:00
stroblme 449a1472cd gc
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Signed-off-by: stroblme <stroblme@posteo.de>
2026-08-17 20:07:56 +02:00
stroblmeandClaude Opus 5 eb1ee65c3a Cap the long lists, and wrap a wide rank on a phone
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Two things the first pass got wrong.

The Changes and "Given up on" lists had no cap, so they ran as long as the
audit trail did — and worse, a truncating flex row still offers its whole
unwrapped line as a min-content contribution, so the card sized itself to the
longest entry and took the page sideways with it. Both now use the same
capped, scrolling box the runs and failures lists already had, which fixes the
length and the width together.

Running downwards, a rank of eight nodes was two thousand pixels across
because a node box is landscape: siblings cost a rank four times as much
across as they do down it. A rank wider than two now wraps onto the ranks
below, settling over a few passes since wrapping one rank pushes what was
under it up into the room that freed. The value chip on an edge no longer
reserves its width there either — that width is spent across the screen rather
than along the flow, and the rank gap it would widen is already wider than the
chip is tall. The same flow that laid out 2040x216 is now 803x722; a chain and
a diamond are unchanged, and so is every desktop layout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDSXaRhvqHYNevgDGmNAto
2026-08-17 18:22:15 +02:00
stroblmeandClaude Opus 5 39ee0e0aa5 Computed flow layout, and mobile written into the design
The canvas lays itself out: a layered graph, left to right on a desktop and
top to bottom on a phone, with room reserved for the value each edge carries.
Nodes cannot be dragged and `NodeDef.position` is gone from the document —
a graph nobody can arrange is one worth keeping small, which is what keeps
flows atomic. Endpoints join the same layout, so their lanes and the
localStorage that remembered where they were dragged go too.

Mobile, per the new Responsive section of DESIGN-GUIDELINES.md: the dock caps
its width and wraps instead of running off the screen, the dashboard stacks
into one column rather than shrinking a wall panel to a fifth of its size, and
Home stops widening its grid track past the viewport. A Playwright project at
a phone's width fails the build when a screen no longer fits.

Along the way: publish is the checkmark that was already there rather than a
button that appears and disappears, with discard beside it on both the flow
and the dashboard; the brain reveals a neuron's name on the first tap; and the
port sparklines get room to breathe.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VDSXaRhvqHYNevgDGmNAto
2026-08-17 17:35:14 +02:00
stroblmeandClaude Opus 5 e7a1466d7b dashboard: pace a querying chart by its window, and an example to evaluate it
A chart is drawn in buckets, and nothing it can show changes until the
bucket it is drawing closes — so the resolution sets the refresh rather
than a flat five-second floor. A week at quarter-hour buckets now asks
four times an hour instead of sixty, for the same picture. Leaving the
field empty follows the window; a slower rate is still honoured.

`make seed-example` builds the thing to evaluate it with: a flow that
logs a temperature to InfluxDB, a flow that answers a chart's request by
turning the window into Flux and the rows back into a series, and a
dashboard holding the chart. The reading flow declares the request as an
input with a starting value, which is how a flow says a value reaches it
from a panel rather than from a node upstream.

Axis labels keep enough decimals to stay distinct — `si` rounds to three
figures, so every tick of a chart living inside one degree read "19".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:34:32 +02:00
stroblmeandClaude Opus 5 61bfd68f26 docs: record the structured-dtype and querying-chart decisions
Also the one the session settled by hand: database nodes stay transport
and credentials, and the Python nodes either side do the query building
and the answer shaping. That is what a series read mode inside the node
would have prevented, so the read-mode item goes with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:15:28 +02:00
stroblmeandClaude Opus 5 19c196b7e6 dashboard: the agenda reads from the top of its card
The frame centres a single reading, which is right for a gauge and wrong
for a list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:14:00 +02:00
stroblmeandClaude Opus 5 59295b1646 Brain: terracotta for fault, a ring set off from the fill, and a dotted ground
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 15:07:30 +02:00
stroblmeandClaude Opus 5 e19776aed1 dashboard: charts that query
A chart can now ask for what it draws instead of reading the ring the
engine keeps. It publishes a request — the window and the resolution —
exactly as a slider publishes a value, and draws the series a flow
answers with. What serves the request is the flow's business, so the
widget never learns which database was behind it.

The answer says what it was computed for and one computed for another
window is ignored, so two charts on one node cost a duplicate query
rather than overwriting each other's picture. Identical requests still
in flight are asked once per tab, and the refresh has a floor under it.

The panel gains the presentation the document could already hold: the
per-series label, a unit, and a y axis that can be pinned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:07:18 +02:00
stroblmeandClaude Opus 5 413501c6ce flow: structured dtypes, and the widgets that read them
A series, record or list message declares its shape instead of riding
DType.JSON, so a widget binds a shape rather than some JSON and a wrong
binding is refused before anything runs. A list declares its item type,
which is what keeps list[float] expressible for a pipeline.

On top of that: an agenda over a list, a notification over a record, and
a dashboard alert channel that publishes engine faults as one — so a
panel can show what went wrong without a flow wiring it by hand.

Also: only None means a node published nothing, a falsy value of the
wrong shape is now the named error it always should have been; and the
gauge's readout says its size is viewBox geometry rather than type scale.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:06:45 +02:00
stroblmeandClaude Opus 5 18837e8880 Brain: neurons in the mark's blue, ringed in destructive when they cannot run
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 14:50:39 +02:00
stroblmeandClaude Opus 5 a1265450df Health: a flow that cannot run says so, and the brain marks which neurons
A dependency loop is flagged on the canvas and was invisible everywhere else:
/observability/summary answered "ok" with an empty problems list while the
published flow could not run at all. It now reports the flows validation
blocks, and the brain graph carries the reason on each neuron the issue names
so the view built to find broken wiring can show it.

Node errors stay counted once, as the nodes that failed to load, and an
advisory like an unauthenticated webhook marks nothing — it is worth saying,
but the flow still runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 14:39:07 +02:00
stroblmeandClaude Opus 5 3b5241bb9a Overviews: create before publish-all; Home: the trend curves keep their dot
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
The Home table drew the one sparkline without its live dot, on the grounds
that a polled rollup's right edge is the last complete slice rather than this
instant. Identical curves reading differently is the worse trade, so the dot
is back everywhere and the cell keeps room for the half that hangs past the
curve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 12:58:35 +02:00
stroblmeandClaude Opus 5 1a589570b1 Brain: dots where a connection meets a neuron, inbound hollow and outbound filled
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 12:52:31 +02:00
stroblmeandClaude Fable 5 02d841e10c Note the database review findings in the NOTEPAD
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 12:40:36 +02:00
stroblmeandClaude Opus 5 aabe7bb147 Let the Home capture wait for the sections that load on their own
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
networkidle now falls between Home's independent fetches, so make verify was
photographing the skeletons: no name, no Admin nav, "No flows yet" and zeroed
tiles on an instance with three flows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 12:08:32 +02:00
stroblmeandClaude Opus 5 2ba56f57b2 Publish the dashboard the endpoint test asserts on
A dashboard edit is a draft now, and the flow canvas draws the controls the
panels actually carry, so the fixture has to publish it the same way it
already publishes the flow. The assertion is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 12:03:54 +02:00
stroblmeandClaude Opus 5 9351a86eec Overviews: icon toolbar, dashboard drafts, publish all
Both overviews carried the same toolbar twice, left-aligned, with a search
field permanently taking a row of width. One `OverviewToolbar` now serves
them: the search folds into an icon and expands again on click (Escape puts
it away and hands focus back), create is a `+`, and everything sits right of
the page. Each page keeps its own create dialog — the toolbar only renders
the trigger — so the testids the runtime spec and the capture script drive
stayed where they were.

Dashboards get the flow store's draft/publish split. The editor autosaves
`dashboard.draft.json` beside `dashboard.json`; `/view/{name}`, `bindings_for`
and `history_requirements` keep reading the published file, so a wall panel
sees an edit only once someone publishes it. `POST /dashboards/{name}/publish`
and `/discard` mirror the flow routes down to the version precondition and the
409, `GET /dashboards/{name}?draft=true` is what the editor asks for, and the
dock grows the same Publish button — which flushes a queued save first, so an
autosave in flight is not published around. Creating a dashboard still writes
the published file directly: an empty document on a panel is harmless, and it
keeps the store free of a never-published case.

"Publish all" is a checkmark in the toolbar, live only when something actually
has `has_draft`. A summary carries no version and publish needs the one it is
based on, so each document's detail is read immediately before its publish —
honest against a stale list, and no version-less backend path to maintain.
Failures are counted rather than swallowed: three of five fails says so and
names the three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:57:11 +02:00
stroblmeandClaude Opus 5 683c25b26d Home: one time range across the health block, and failures that follow it
The health screen was fixed at 24 hours everywhere except its lists, which
were fixed at nothing: `failuresQueryOptions` read the newest 100 rows and
`HealthActivity` filtered them client-side, so on a busy engine the failures
list covered whatever few minutes 100 rows happened to span while the chart
beside it spanned a day — and pinning an older minute showed an empty card.

One `RangePicker` now sits on the Health heading and governs the whole block:
the tiles, the flow table, both charts and both lists. Presets are 1h / 6h /
24h / 7d — the collector prunes at `OBS_RETENTION_DAYS` (30), so a week is
behind the last one. The longer windows ask for coarser buckets, since a week
of minute rollups is ten thousand points nobody can see.

Failures get the escape hatch the runs already had: a pinned minute is asked
for with `since`/`until` rather than filtered out of what is held, and the
list itself is bound to the selected range. `RUN_DEPTH`/`EVENT_DEPTH` become
one `LIST_DEPTH`, which now buys coverage of the window on screen instead of
a fixed newest-N — narrowing the range is what makes the same rows reach the
whole of it. The "Failures (24h)" tile counts errors over the selected window
from the rollups the table is drawn from, so tile, column and chart agree.

The node-panel and edge trend curves get no picker. They are a Redis ring of
the last 120 values per message with no window to ask for, so hovering one
reveals what it actually shows — how many readings, and the span they cover.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:51:42 +02:00
stroblmeandClaude Opus 5 3d73e42313 Roadmap: tick the CI workflows and the broker/InfluxDB dev services
Both were already in the tree — the four workflows under .gitea/workflows and
the mosquitto/influxdb services in docker/compose.dev.yml. The CI line says
the wiring is unproven, since nothing here has watched it run on a push.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:49:07 +02:00
stroblmeandClaude Opus 5 0cff521fd2 Roadmap: tick what M5 already delivered
Only the soak harness was still open here: `backend/scripts/soak.py` and
`make soak` exist with all four scenarios, so the Phase 0 item is done —
with the two caveats from NOTEPAD on the line, since the redis scenario
stops a shared container and the engine kill barely stresses redelivery.
The other M5 items (brain graph, modules, out-of-process nodes, engine
history and the health screens) were already ticked and match the code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:47:41 +02:00
stroblmeandClaude Opus 5 b9e67fa436 Home: spread the activity table's slack, shorten its counts, shorten the fade
The trend column carried `w-full`, which made it swallow every spare pixel:
the five columns beside it huddled in the left 40% with a dead gap before the
curve. It takes a bounded third now, so the auto layout spreads the rest over
the columns that have content to show, and a long flow name still widens its
own column into the card's scroll.

`si` reaches the rollup counts it was written for — 179240 executions reads as
"179k", and the failure badge and the failures tile follow. The plain window
leaves the small ones alone: 615 is still 615. The standing counts (nodes,
flows running, queue depth) stay exact, as does everything acted on digit by
digit.

The fade ran a full 40 of the 100-unit box. In a table cell three times the
width of a panel that washed out readings still worth seeing, so it is 25 —
a hint at the left end rather than a quarter of the series.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:38:07 +02:00
stroblmeandClaude Opus 5 64f57b732b Regenerate the SDK for the events time window
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:35:03 +02:00
stroblmeandClaude Opus 5 2554488a73 Backend: real alert test results, state cleanup on delete/rename, node trigger errors, queue and collector fixes
`AlertManager.send` swallowed every delivery failure, so the alerts screen's
Test button answered 200 whatever happened — the one thing it exists for. It
takes `raise_on_error` now, which only the test route passes; the per-channel
loop keeps the swallow, because one dead channel must not stop the others
hearing about the same fault. A refused delivery answers 502 with whatever the
sender said.

Renaming a flow left its values under the old name for good: the delete path
already swept them, the rename path never did. It calls the same `forget_flow`,
which covers the messages and the `__ts__`/`__version__`/`__history__`
bookkeeping keyed by message name. Cleanup, not migration — they repopulate
under the new name on the next run.

Triggering a node by hand ran `Node.__call__` with nothing catching it, so a
node that raised produced a 500 and a stack trace in the server log, and
nothing at all on the canvas. `Pipeline.publish_error` is the reporting half of
`_execute_node` lifted out; both paths go through it, so a manual failure now
reads the same on the canvas and in the metrics as a queued one. The route
answers 400 with the node's error.

`MemoryWorkQueue.stats()` counts claimed-but-unacknowledged work rather than
reporting zero, so the health tile means something without Redis. The metrics
collector's held tracebacks are capped at `DETAIL_CAP` and swept on the same
`RUN_STALE_S` cutoff the open runs use, instead of one untruncated traceback
per node kept for the life of the process — a traceback still survives the
flush between the log and the failure it belongs to.

`GET /observability/events` takes `since`/`until`, the window `/runs` already
took, so a failures list can cover the span the charts beside it are drawn from.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:33:25 +02:00
stroblmeandClaude Opus 5 69421b857d Brain: hover labels, activity falloff, rim-anchored directed edges
A name under every circle was most of what the graph drew, and none of it was
what someone looks at the brain for. Only the hovered or keyboard-focused
neuron names itself now — a failing one always does, since colour is never the
only carrier of a status. The label is absolute and the collide radius already
held that space, so revealing one moves nothing. Taking it out of flow also
fixed the off-centre edges: the node box was as wide as its widest label, so a
32px circle inside a 140px box sat 54px left of where the layout put it and
every edge aimed at the box rather than at the circle.

Neurons are filled discs rather than `bg-card` outlines, which were a hairline
against `--background` in light and close to nothing in dark. The fill is a
neutral `--muted-foreground` alpha and stays neutral: status still speaks only
through the border and the word beneath. Its lightness carries how recently the
open page saw that neuron publish — brightest just after it fires, decaying over
90s to a floor it never drops below, so a graph left open sorts itself into what
is busy and what is not. Edges do the same on their stroke. The signal is only
what this session has seen on the socket, so a fresh page starts everything at a
neutral middle rather than claiming a history it does not have. Both are CSS
transitions gated on `prefers-reduced-motion`, decayed out of and snapped into,
never the other way round.

`BrainEdge` now carries both circle radii and trims its path back to the rims,
with an arrowhead at the end it flows into. The barbs are two more segments of
the same stroked path rather than an SVG marker, so they dim with the line they
belong to. Neither trim reaches past the midpoint, so neurons closer together
than their radii still get a line pointing the right way.

`edge-pulse` reads its landing colour from `--edge-rest` instead of naming
`--muted-foreground`, so a selected edge — which rests in blue — decays into its
own colour instead of crossing to grey and snapping back at the end.

Obsidian's graph view is the reference for the first three: filled nodes sized
by degree, labels on a fade threshold with hover to recover them, and arrows as
an explicit affordance. The falloff-by-opacity encoding is the standard one in
the dynamic-graph literature.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:31:13 +02:00
stroblmeandClaude Opus 5 78c605bd37 Home: one sparkline style with a left fade, SI values, a full-width activity table, roomier panels
The trend curve was two components: a rich one in the node panel and on edges,
and a bare line in the flow table. It is one `Sparkline` now, taking the colour
token, the height and whether the live dot and the readout show. The flow table
draws its rollup in the chart ramp with no dot — the rollups are polled, so the
right edge is the last completed slice rather than this instant — and keeps its
"nothing yet" state, as the panel keeps its three distinct silences.

All of them fade out to the left, through an SVG mask over the curve and its
area. The dot sits outside the mask: the newest reading is the one thing that
must stay solid.

`si` replaces `compact` and the ad-hoc "k" the uPlot axis carried. It prefixes
k/M/G and m/µ, but only outside 0.01–1000, where the plain number is already
the shortest thing to read and a written unit ("0.4 ms") stays honest. The
sparkline readout asks for four digits, so two neighbouring readings never
collapse into one string. Exact counts and anything the user acts on — a
payload, a form field, the edge inspector's value — are left unrounded.
`src/lib/utils.check.ts` asserts the rounding cases.

The activity table now spans its card: the flow name anchors the left, the four
numbers read down their own centre, and the trend takes the slack on the right.
Panel rhythm steps up one notch, gap-5 to gap-6 outside and gap-2 to gap-3
within a section.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
2026-08-17 11:27:54 +02:00
stroblme 5483de13c0 NOTEPAD: components needed for wall-panel parity with the current home dashboard 2026-08-17 09:54:28 +02:00
stroblmeandClaude Opus 5 5fc7bbee34 Home: widen the brain layout, fix its entrance, add a clear button to pinned charts
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
The force layout settled roughly square, so in a wide band the graph sat as a
small island. Vertical centring now pulls nearly three times as hard as
horizontal and the seed sits on an ellipse, which settles at about 2.2:1
without stretching the circles or the edges.

The graph also slid in on load: the rebuild fit ran with a 300ms duration, so
React Flow animated the pan from its default viewport to the content. The fit
is instant now, and the canvas grows in from the centre with `scaleIn` once it
has a layout.

The pinned chart header said "Esc clears", which a phone cannot act on. It
carries the same "× Clear" button as the list header instead; Escape still
works.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-17 08:33:27 +02:00
stroblmeandClaude Fable 5 0832be32a9 Note the run-list cap a busy pinned minute would hit
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-17 00:29:02 +02:00
stroblmeandClaude Fable 5 0b91379914 Let a pinned minute read runs from the whole day
/observability/runs gains since/until, so the throughput chart's pin asks
the server for its minute instead of filtering a fixed recent list. This
engine writes ~60 runs a minute, so any minute but the newest read empty.

since is inclusive and until exclusive, matching the minute buckets the
charts are drawn from. Hover stays the client-side preview it was:
scrubbing a day would otherwise be a request per minute rested on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-17 00:27:40 +02:00
stroblmeandClaude Fable 5 affb0a5f8c Fold the brain and health screens into Home
Home is the one overview now: the brain graph flat across the top, the flow
switches, then the health sections. The /brain and /health routes and their
sidebar entries are gone.

- charts report their cursor and clicks, so hovering one filters the list
  beside it to that minute and a click pins it until Escape or Clear
- chart values round to about three significant digits, the legend mounts
  under the plot so it can wrap without leaving the card, and axis ticks
  shorten past a thousand
- the embedded brain leaves the wheel to the page rather than zooming
- number fields no longer draw their up/down spinner (NOTEPAD)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-17 00:17:14 +02:00
stroblme fe346ea7a3 up
Signed-off-by: stroblme <stroblme@posteo.de>
2026-08-17 00:04:22 +02:00
stroblmeandClaude Fable 5 83c30aa1c7 Take the engine off the path node code imports from, and let it stop
The worker script is handed to the interpreter by path, so app/flow was
sys.path[0] for every node: `import queue` got the engine's. It now drops
its own directory before anything else imports, and runs with the
deployment's credentials scrubbed out of its environment.

Also: reload builds off the event loop, the pool wakes what is blocked on
it when it stops, a refused metrics flush is kept for the next one rather
than dropped, and the cascade events are paired through failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 23:46:08 +02:00
stroblmeandClaude Fable 5 b33be3fdd4 Load the engine, then take its pieces away and check what survives
A standalone harness, never part of the test run, that drives a real stack
through the durable path — the webhook, which journals every trigger — and
then stops Redis, kills the engine mid-cascade and restarts the broker
under a live subscription. Latest-value-wins with concurrent cascades means
counter equality is not a promise, so what it asserts is that the queue
drains, that state ends on the last value sent, and that nothing reached
the dead-letter stream.

Every docker verb goes through one helper that checks the compose label
before it acts and refuses anything outside this project, because the
machines this runs on host unrelated services.

Two invariants are deliberately looser than they look. The queue belongs to
the whole stack, so "nothing pending" would be measuring other people's
traffic; the harness waits on the age of the oldest unacknowledged entry
instead, which a stuck item always dominates. And the observability tables
are cleared only after the collector's flush interval has passed, since
deleting a flow publishes an audit event of its own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 23:14:14 +02:00
stroblmeandClaude Fable 5 4bcd38354b Draw every flow as one graph, merged on what it talks to
A node type can now say which outside thing its parameters point at, and
nodes sharing one — a broker topic, a URL, a bucket — are drawn as a single
neuron on a new /brain canvas. That makes the wiring which runs between
flows through a broker visible for the first time; no single flow's canvas
can show it. The key is read off stored parameters, so a credential
reference never reaches an id.

Layout is a d3 force simulation settled once and then frozen, lit by the
socket the editor already listens to: a neuron pulses when any node behind
it publishes, and its connections light as values pass.

Fixes the message pulse while here: interpolating the stroke against the
edge's `color-mix()` resting colour went through oklab and left the gamut,
which turned every pulse on both canvases fluorescent yellow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 22:59:31 +02:00
stroblmeandClaude Fable 5 af3ba51571 Keep the engine's own history, and a screen that reads it
A second bus subscriber folds executions, errors, timings and queue lag
into per-minute rollups, keeps failures with their traceback and an audit
trail of who published what, and records one row per cascade — manual runs
and previews included, under an id of their own that writes no idempotency
markers. Read back through /observability/*, which always answers 200 so a
degraded engine still renders its own health screen.

Also fixes two things found on the way: node-health alerts read `status`
where the engine publishes `health`, so a device dropping never alerted
anyone, and the Redis queue reported `parked: 0` whatever was held.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 22:29:32 +02:00
stroblmeandClaude Fable 5 f300c43f3a Run python nodes out of process, with modules of their own
User code no longer execs in the engine. A pool of persistent worker
subprocesses speaks one JSON object per line; the controller installs a
proxy as the node's function, so every execution path funnels through it
and the pipeline is untouched. A crash costs one subprocess, a per-node
timeout is a kill, and cancelling from the canvas is that same kill.

The workers run a venv of the user's own on the data volume, filled from
a pip manifest versioned beside the flows. Applying it retires the
workers and rebuilds, so a package lands without restarting the engine.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
2026-08-16 21:43:36 +02:00
stroblme 979c9d3c1f Note that a node's error status clears before anyone sees it 2026-08-16 19:32:10 +02:00
stroblmeandClaude Fable 5 49a3f3da01 Note the canvas follow-ups the review turned up
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 19:16:06 +02:00
stroblmeandClaude Fable 5 949fc0bf7e Ask for the name in a dialog instead of a field that is always there
Create in both overviews now opens a dialog, so the toolbar is a search box
and one button. The specs and the capture script open it first.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 19:08:22 +02:00
stroblmeandClaude Fable 5 c2321fe942 Give a dashboard the panel's own size, and put the dots on its grid
A dashboard now carries the canvas it is drawn for (canvas_width /
canvas_height, presets plus two numbers in the settings panel). Editor and
wall panel render that surface at its true pixel size and scale it to fit,
so a side panel opening changes only the scale — never the arrangement
being made. With the width and column count known the dot pitch is exact,
(width + gap) / columns by row height + gap, so a dot sits where every
widget corner snaps. The wall panel shows no dots.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 19:08:09 +02:00
stroblmeandClaude Fable 5 fcd43e9ad9 Keep an endpoint's measurement so its drag stops relighting the edges
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 18:47:17 +02:00
stroblmeandClaude Fable 5 63531fa36e Record what the package sweep landed and what it turned up
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 17:28:55 +02:00