Commit Graph
381 Commits
Author SHA1 Message Date
stroblmeandClaude Opus 5 e2f97d36e4 Document the scheduler
Docs / docs (push) Successful in 32s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m56s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m0s
pre-commit / pre-commit (push) Failing after 3m8s
Test Backend / test-backend (push) Successful in 2m45s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Successful in 1m15s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
2026-08-27 09:36:39 +02:00
stroblmeandClaude Opus 5 7d87dfffdd Put resources where somebody would look for them
`resources` was reachable from the SDK and the API and nowhere else, and
`node_queued` was published to nothing at all — so a node waiting for a machine
looked exactly like a node that had hung, which is the failure the event was
added for.

The node panel gets a Resources section: a size by name, the numbers for a node
that wants its own, and how long it is expected to take. Picking a flavor drops
the numbers, because saying both is two answers and the engine refuses it.

A queued node draws a neutral dot rather than one of the three status colours —
it is not running, it did not go well, and it did not go wrong; it is idle with
a reason, which the tooltip gives.

And a Workers screen, which is the first UI for any of this: every machine the
engine can reach with what is free of each, what is attached, what a cluster
has been asked for, and the sizes, editable. `fluksio status` grew a line of
the same.

The demo's training node already preferred a GPU worker, which was exactly the
declaration that used to be dropped, so it now says how much of that machine it
takes as well.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
2026-08-27 09:32:08 +02:00
stroblmeandClaude Opus 5 40f8ad378d Ask a cluster for a machine when nothing here will do
Slurm is not a machine that attaches and stays; it is a queue somebody else
owns. So nothing here submits a node to it. It submits a job whose payload is an
ordinary worker dialling back in, and everything downstream — the protocol, the
artifacts, cancellation, the books — already worked and did not have to learn
what Slurm is.

The alternative, which Covalent takes, is to stage a serialized call and a
runner onto the login node, poll squeue and copy the result back: a second way
of running a node beside the one that exists. The cost of not doing that is one
assumption, that a compute node can open a connection outward. Where that is
false, _payload is the single method a staged variant would replace.

Clusters are configured in provisioners.json beside the alerts, since this is
infrastructure an operator writes rather than anything a flow says. The script
is generated with the system ssh and no new dependency, and prerun owns the
environment — deliberately no pip install, because what is on a cluster is
somebody's decision.

One outstanding request per profile, cancelled if it never attaches and on the
way out. Nothing autoscales.

The run gate needed the same hook: a run held before it starts never reaches the
placer's own wait, so it would have queued forever on a machine nothing had
asked for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
2026-08-27 09:09:44 +02:00
stroblmeandClaude Opus 5 a82f88cf0a Name the sizes a node can ask for
Raw cpus and gpus are a property of the machines an installation has, so a node
written against a cluster quietly stops meaning anything when the cluster is
replaced. A node says "gpu-small" instead, and what that is stored here —
editable, and read again every time the node is built, so changing the flavor
changes what the next run gets.

Memory joins the schema properly (`ram`, in MB, accepting "2G"), along with
`duration_s` for how long a node is expected to take. That one is recorded and
shown and nothing else yet: a statement for whoever is planning around the node,
not a limit — the limit is still `timeout`.

A flavor and a number for the same thing is refused, compared by value so an
editor writing the whole object back with its defaults still round-trips. A name
nothing stores is refused at the save, which covers the canvas and `fluksio
sync` at once, and deleting one a node still asks for says which node.

Four sizes are seeded on an installation that has none, and never re-seeded:
re-adding one somebody deliberately removed is an argument nobody wins.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
2026-08-27 08:59:10 +02:00
stroblmeandClaude Opus 5 6ff56533f5 Schedule a node across every machine, not just this one
The engine answered "where does this node run" twice, in two ways that could
not see each other: a device sent it to a worker carrying that label, and
resources were counted against the engine's own cores. Declaring both meant the
second answer won and nothing was counted at all — which the data-science
getting-started page and the worked example both do.

One question now, in flow/placement.py: of every machine attached, which could
grant what this node asked for, and which of those has it free. The books move
onto each machine — one accountant per worker, built from the inventory it
reported — and the waiting moves above them, where one condition variable can
be woken by a release anywhere or by a worker attaching. Locks go one way:
placer, then a machine's books, never back.

So a node asking for a card now finds the box that has one, rather than being
clamped down to none and run here. When nothing can grant the ask at all it is
still cut down and run — a flow written on a cluster has to work on a laptop —
but the ceiling is one real machine now, since taking the largest of each
dimension separately can describe a machine nobody has.

Two things fixed on the way. A device on a connector node held every batch run
of its flow forever, waiting for a worker that could never run an entry point.
And `prefer` falling back to the engine skipped the books, so the fallback held
nothing.

The bench flow's node has taken a `params` argument that with_settings has not
forwarded for some time, so the benchmark could not run at all: 62 ms median
submit-to-result with this, against the 61 ms on record.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
2026-08-27 08:49:36 +02:00
stroblmeandClaude Opus 5 1a9753fa9d Let a worker say what machine it is
A worker reported its labels and nothing about the machine behind them, so the
engine could route a node to a GPU box but not tell whether that box had a GPU
free. Inventory — cores, GPUs, memory — now arrives with the hello frame, and
the run frame carries back what the engine allocated for that call.

Which is protocol 2 on both ends. GPUs are never probed: asking a vendor tool
would make the one dependency two, so a GPU is what the batch job says it was
given or what --gpus says. A worker that reports nothing still attaches and is
scheduled by its label alone.

Two things a job scheduler needs: --max-idle stops a worker started for one job
rather than letting it hold its allocation to the walltime, and a refusal is now
fatal instead of a reconnect loop that reads as a hang in a job's log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
2026-08-27 08:29:35 +02:00
stroblmeandClaude Opus 5 0ffcabfdb9 Media dtypes: image, audio and video as narrowed artifact references
Docs / docs (push) Successful in 30s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m7s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m54s
pre-commit / pre-commit (push) Failing after 4m24s
Test Backend / test-backend (push) Successful in 3m8s
Compose Smoke Test / test-compose (push) Successful in 40s
Playwright Tests / merge-reports (push) Successful in 1m33s
A port may now declare `image`, `audio` or `video`. Each is the artifact
reference the engine already had, narrowed by the `media_type` on it, so a
speech recogniser declares what it eats rather than taking any bytes at all and
finding out. Bytes still never travel as a message and nothing on the wire
stops being JSON: a camera publishes one reference per frame, a microphone one
per chunk, and a reference may carry a `meta` dict nothing here interprets.

Streaming media is therefore an ordinary streaming port — with one change to
what that means. An emission used to journal an item with no payload, so
downstream read whatever was current when the item was claimed; a consumer
slower than its producer saw only the newest chunk and the ones between were
lost. That is right for a training curve and wrong for a second of speech, so
an emission now journals a `kind="emission"` item carrying its values, and the
executor hands them to the nodes reading that message instead of writing them
to state again. The value in state stays the latest, which is what everything
else reads, and the wave is filtered by what actually changed rather than
walking everything reachable. No queue serialization change — the existing
`outputs` field carries it.

Continuous media makes the store's missing GC a real problem, so this closes
it: `sweep_artifacts` runs hourly, keeps every digest a `run_artifact` row
records or a live message holds, spares anything written in the last hour, and
stands aside entirely while a run is in flight, since a node may store a
checkpoint long before it returns the reference to it. That also collects the
orphans a deleted flow has always left behind. `ARTIFACT_GC_INTERVAL_S=0` turns
it off.

Around the edges: `GET /artifacts/{digest}` serves the media type the caller
passes and answers ranged requests, so a browser plays a clip rather than
downloading it; `PUT` spools to disk instead of holding the whole body in
memory, as does `save_artifact` given a path; a Media widget draws whatever its
message points at, and a wall panel may fetch the bytes its own tiles are
showing and nothing else; and a connector gets `save_artifact`, for a device
whose readings are bytes.

What this cannot do is live video: a frame every second or two is a glance, and
the honest answer above that is the camera's own stream, which the widget takes
as a URL and the browser plays from source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 23:44:55 +02:00
stroblmeandClaude Opus 5 8be7e424ba Cover the sink reaching a node the controller built
Docs / docs (push) Successful in 30s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m59s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m9s
pre-commit / pre-commit (push) Failing after 3m16s
Test Backend / test-backend (push) Successful in 2m48s
Compose Smoke Test / test-compose (push) Successful in 39s
Playwright Tests / merge-reports (push) Successful in 1m12s
The pieces were tested one at a time, so removing the one line in
`_build_node` that wraps a worker-backed function with its sink broke nothing
visible — while losing every generator node's last yield and returning None
in its place.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 23:01:15 +02:00
stroblmeandClaude Opus 5 8f71b638b6 Make the example's __main__ actually run
It claimed to run the pipeline with no engine involved, and could not: the
node bodies it called save and load artifacts, which raise outside a node by
design. Each node is now a thin wrapper over a plain function — make_rows,
train_curve, score — and __main__ calls those, which is the split the sandbox
already demonstrates and the one worth copying.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:57:36 +02:00
stroblmeandClaude Opus 5 4f3eaf950c Report a published node that has no code of its own
The store answers the new-node template when nothing was ever written for a
node, so such a node ran — returning {} on every call, reporting active and
ok, and saying nothing anywhere. Unreachable through `fluksio sync`, which
writes every body before it publishes; the editor end was open.

A run of a flow holding one is now refused, and the flow carries a
missing_source issue so it is visible before anybody runs it. A draft is
exempt: a node being written legitimately has no published body yet.

The generated client is regenerated for the new issue code, which also
catches up the drift left by earlier backend work (resources, code_digest,
idempotency_key).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:55:11 +02:00
stroblmeandClaude Opus 5 69fc0ceeb5 Format the generator test's sink helper
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:53:41 +02:00
stroblmeandClaude Opus 5 2e82367926 Keep the file name a node gave an artifact
A run_artifact row is keyed by the message the bytes left on, and that was
also the only name it could answer with — so an `@run:` reference resolved
through the row was the same bytes under a name its producer never chose.
The row now records the file name beside the message name; rows written
before the column answer as they always did.

The fallback also checks the bytes are still in the store, which the bare
digest spelling beside it has always done. A missing blob now fails at
submit rather than in the middle of the run that wanted it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:45:11 +02:00
stroblmeandClaude Opus 5 001ec7b282 Key the stage cache by the node's input names, not the flow's
A cache key held qualified input names, so the same node reading the same
values through two flows keyed differently and only a node with no inputs
could ever hit across one. The fingerprint beside the key already says what
the node is, and it has been flow-agnostic since it moved ahead of
assign_flow — the names were the last thing tying an entry to one flow.

Inputs now reduce by the node's own name for them; a name belonging to
another flow keeps its prefix, since reading it is part of what the
execution is. Every stored entry misses once and is re-run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:41:35 +02:00
stroblmeandClaude Opus 5 647644ebbd Check a generator's yield at the yield that produced it
The worker held each yield one behind, because the last one is the node's
result when the generator returns nothing of its own. Only the engine knows
what ports a node declared, so the check happened when the *next* yield
arrived — a pass late, which for a training loop is however long one epoch
takes.

The worker now sends every yield as it happens and returns whatever its
generator returned; EmitSink holds the last one back and decides at the end
of the call what it was. Old "emit" frames are still handled, so a remote
agent that has not been restarted keeps working.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:39:22 +02:00
stroblmeandClaude Opus 5 f2dd1ffc34 Show resources on the example's training node
The example's `fit` is what a real training node looks like, so it is where
`resources={"cpus": 2}` belongs -- beside the `device="gpu"` it already
carries, since the two answer different questions about the same node.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 21:46:27 +02:00
stroblme 15a3c2ce62 Document how a connector handles a device that decides its own ports
An undeclared key fails the whole reading, so a connector whose ports vary by
model has to narrow what it publishes to what was bound. Names the idiom and
the once-not-per-poll rule for saying a declared port is missing.
2026-08-26 21:43:59 +02:00
stroblmeandClaude Opus 5 608d30d884 Let a node say how much of the machine it takes
Five concurrent training nodes, each sizing its thread pool to every core,
left the engine's own event loop unscheduled: the API stopped answering
within 10 s and every client died. The same shape on a GPU deadlocked a run
for 21 minutes at 0% utilisation with nothing failing and nothing to read --
it just sat in `running`.

@node(resources={"cpus": 2}) is the declaration. The engine holds that much
for the length of the execution, so more of them than the machine has room
for wait their turn rather than oversubscribing it, and a `gpus` node holds
its card exclusively. FLOW_CPUS defaults to every core but two, and those two
are what keeps the engine answering.

Because a thread cap is read when the process imports the library, a warm
worker cannot be told a different one -- so an environment gets a pool of its
own and nodes deriving the same one share it, rather than paying a cold start
per call on exactly the nodes whose imports are slowest. XLA_FLAGS is never
derived: it is a composed, version-dependent string, so it travels in
resources.env where it is visible.

A node that declares nothing is not accounted for and behaves as it always
did -- it just gets FLOW_CPUS/FLOW_MAX_WORKERS as a thread cap, which is the
half of this that fixes the reported incident without anybody declaring
anything. An operator who set OMP_NUM_THREADS themselves still wins.

Resources are claimed strictly before a worker slot, so the two blocking
waits cannot deadlock. A node queued for them publishes node_queued and shows
on GET /workers/resources, because waiting and hanging looked identical.

Accounted, not enforced: no cgroups, no rlimits. Scheduling across machines,
flavours and enforcement are the next steps.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 21:36:59 +02:00
stroblmeandClaude Opus 5 4a38c6ed31 Name the code a run ran, and let an interrupted sync finish
Three faults with one root: the stored body of a code-defined node is an
import shim, and nothing that mattered was ever read from the code itself.

- The run stamp could not identify what ran. The shim imports whatever is on
  disk when the worker starts, and an uncommitted tree stamps <commit>-dirty
  for every run it ever produces. Run.code_digest hashes the repository's .py
  files, memoized on their stat state, and it is read again when the run is
  actually claimed -- so a sweep queued for hours records the code each of its
  runs executed, not the code that was there when it was submitted.
- The stage cache adopted code that was too new. The fingerprint hashed the
  shim, which is invariant under any edit to the imported function or anything
  it calls into, so a re-run was served from cache and answered without the
  outputs the edit added. It now carries the repo digest and the node's
  declared ports. Every fingerprint changes once, which invalidates the
  existing cache; a canvas flow has no repository and keys as before.
- An interrupted sync looked like a hand-edited canvas. The engine answers a
  new-node template for a node with no stored body, and the template carries
  no marker, so the drift check read "somebody edited this" and demanded
  --force -- for the one state that re-running the sync is the fix for.
  NodeSource.missing states the fact, and sync skips those and reuses the
  bodies it read instead of asking for each one twice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 21:27:10 +02:00
stroblmeandClaude Opus 5 1f7c6646f1 Survive a busy engine: retry, idempotent submit, resilient waiting
A driver script died of one slow answer: httpx.ReadTimeout out of
RunHandle.refresh() with a 30 s read timeout and no retry anywhere, which
cost a sweep 78 of its 84 runs.

- Split the timeout (5 s connect, 120 s read): a wrong URL fails at once,
  and a busy engine gets longer than the slowest thing it does on purpose
  (a 60 s compile, a 15 s rebuild wait).
- Retry idempotent calls three times on a transport error or 502/503/504.
  503 is the engine's own "ask again" — it is what RebuildBusy answers.
- Submit carries a key the engine stores with the run, so a retry after a
  timeout returns that run instead of starting a second. A sweep keys every
  entry, so a half-created one recreates only what is missing.
- wait() and --follow tolerate five failed polls in a row; a 404 still stops
  at once, because that is an answer rather than a gap.
- CLI says "engine not answering" and names the run still on the engine,
  instead of printing a traceback.
- runs: clamp the params column to 80 characters; events() takes the
  flow/since/until the endpoint already had; RunHandle.failures answers
  "what killed this run" from the run's own node rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 21:19:09 +02:00
stroblme 2cf45e4f39 Release v0.1.3
Docs / docs (push) Successful in 20s
Playwright Tests / test-playwright (1, 2) (push) Successful in 2m38s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m40s
pre-commit / pre-commit (push) Failing after 2m44s
Test Backend / test-backend (push) Successful in 2m15s
Compose Smoke Test / test-compose (push) Successful in 31s
Publish / publish (push) Successful in 50s
Playwright Tests / merge-reports (push) Successful in 1m31s
v0.1.3
2026-08-26 16:05:04 +02:00
stroblmeandClaude Opus 5 a71e0b3375 Lay flows out so edges stop crossing where they need not
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Successful in 4m17s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m43s
Playwright Tests / merge-reports (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 18m52s
Test Backend / test-backend (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
The canvas placed nodes with dagre, which orders nodes but knows nothing
about ports: an edge leaves a particular handle and lands on a particular
one, so two producers feeding a consumer in the other order crossed just
before they landed however well the nodes were arranged. The graph was not
a multigraph either, so two messages between one pair of nodes collapsed
into one before crossings were counted, and the endpoint labels were laid
out as 220-wide cards with every handle stacked on a single pixel.

ELK's layered algorithm replaces it. Every declared port is handed to it as
a fixed point on the node's rim, at the fraction `portFraction` puts the
handle at — which the node components now render from the same function —
so what the crossing count is minimised over is what ends up on the screen.
Measured on this installation's flows, with the labels included: `home`
goes from 51 crossings to 6 across and 70 to 29 down, `demo_training` 27 to
5 and 59 to 32, and every graph is the same size or smaller.

The engine answers asynchronously and is a chunk of its own, so positions
became state: the canvas draws nothing until the first layout lands, and an
edit keeps the arrangement it had rather than flashing through the corner.
The entry chunk is untouched and the flow route's own chunk came down 167
to 121 kB, since dagre used to be in it.

Two things the crossings made worse come with it. A hovered edge resolves
to full strength so one line can be followed through a busy rank, and a
feedback edge — one whose target the layout put behind its source — swings
out into a lane beside the graph instead of being drawn through everything
between its ends, which is what "House history" was reported for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U6vyQvfX249hsv9YPmsYYg
2026-08-26 15:37:36 +02:00
stroblmeandClaude Opus 5 b80b92aad7 Local test targets read the running stack, not the deployment's domain
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Successful in 1m57s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m43s
pre-commit / pre-commit (push) Failing after 3m32s
Test Backend / test-backend (push) Successful in 2m18s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Successful in 1m27s
`make test-frontend` built PLAYWRIGHT_BASE_URL from DOMAIN in .env, which in a
checkout configured for a deployment is that deployment's domain — so the suite
that creates and deletes flows, dashboards and users was pointed at
app.fluksio.com, held local only by --add-host and tests/guard.ts.

The hostname now comes off the running stack (the frontend container's own
Traefik rule), so a name no local container answers to cannot be reached at
all, and the local targets default to *.localhost instead of reading .env.
Target-specific on purpose: an exported DOMAIN outranks --env-file in compose
interpolation and would put the production targets on localhost.

`rebuild-frontend` replaces the raw compose line CLAUDE.md spelled out, taking
the same domain so the baked VITE_API_URL cannot disagree with what Traefik
serves. Also: a coverage HTML report that cannot be written no longer fails
test-backend after a green suite, and both artifact actions in playwright.yml
drop to @v3, which is the only version without the github.com-only guard that
failed every run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 13:48:05 +02:00
stroblmeandClaude Opus 5 bda67d1c0d Let the slider keep the drag the browser wanted to take
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m38s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m41s
pre-commit / pre-commit (push) Failing after 2m47s
Test Backend / test-backend (push) Successful in 2m19s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Failing after 1m4s
Dragging the brightness slider on the wall panel set the handle and
published nothing; tapping a point on the track worked. The control is a
native range input laid transparent over the drawn track, and only the
release publishes — but the input never said the drag was its own. On a
touch panel a sideways swipe is a pan, or a back-navigation, so the
browser took the pointer over mid-drag and ended it in `pointercancel`.
The value had followed the finger and was never sent.

`touch-action: none`, as the colour disk beside it has always had. The
release also answers `pointercancel` and `lostpointercapture` now, which
covers a mouse let go outside the input and leaves no way for a draft to
sit there unpublished.

Two checks: a drag across the track reaches the engine, and the input
still owns its gesture. The second fails against a build without the
CSS, which is what makes it worth having.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012n6CehUsHYYUXJ48rxaD18
2026-08-26 13:32:22 +02:00
stroblme 95aae7b95f Show what differs between the runs being compared
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m38s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m40s
pre-commit / pre-commit (push) Failing after 2m44s
Test Backend / test-backend (push) Successful in 2m17s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Failing after 1m3s
2026-08-26 13:18:16 +02:00
stroblme 0578a30a3a Name the comparison's X and Y beside their pickers, not in every option
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m47s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m44s
pre-commit / pre-commit (push) Failing after 3m14s
Test Backend / test-backend (push) Successful in 2m24s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Failing after 1m7s
2026-08-26 12:20:38 +02:00
stroblmeandClaude Opus 5 266acd1661 Document the cascade guarantee and the cascade ceiling
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m46s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m42s
pre-commit / pre-commit (push) Failing after 2m55s
Test Backend / test-backend (push) Successful in 2m23s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Failing after 1m3s
A node runs when something it reads was published, not merely when something
upstream of it ran — worth saying now that it is true, since it is what a
reader relies on when a rate-limited port goes quiet. Plus `FLOW_MAX_CASCADES`
in the engine's configuration table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf
2026-08-26 10:27:23 +02:00
stroblmeandClaude Opus 5 1430c4060b Give the engine bench a make target
And stop its drain loop asking for the whole of `stats()` every twenty
milliseconds — four round trips a poll, competing with the engine for the
connection it is measuring.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf
2026-08-26 10:16:04 +02:00
stroblmeandClaude Opus 5 180da3d640 Stop paying five Redis round trips and a global lock per message
The engine was I/O-bound on its own state backend. `RedisState.lock()` is one
key — `pipeline:_lock` — for the whole process, taken five times a message at
two round trips each, and every cascade and every node read queued behind it.
Inside it, reading a node's inputs was three round trips per input (an EXISTS
for `in`, then EXISTS and GET for the value), writing was two updates that a
single transaction already gives, and the version counters went one INCR at a
time.

Replaced with the atomic command that was always available: `get_present` is
one MGET and tells a missing key from one holding null, so the lock it used to
be read under bought nothing; value and timestamp land in one `update`, which
is a MULTI/EXEC; `increment_multi` pipelines the counters. `values()` — what
every websocket snapshot calls — is two reads whatever the message count
instead of two per message.

Beside that: every webhook did its blocking XADD on the asyncio event loop
(MQTT already used `to_thread`); the per-execution `NodeOutcome` was built and
validated even with no run watching; `_minute` built a tz-aware datetime per
event on the loop thread to key a dict, and now keys on an int; `move_due`
promoted delayed items one round trip each, every second; `FLOW_MAX_CASCADES`
makes the in-flight ceiling a setting rather than a constant.

`orjson` replaces stdlib json where a message pays for it — state, the
journal, the engine side of the worker pipe. `fluksio-worker` stays
dependency-free, and the run-cache digest stays on stdlib so no stored key is
invalidated. A non-finite number now stores as `null` rather than the bare
`NaN` that was never JSON.

Measured with `scripts/bench_engine.py` against a real Redis, 200 messages:
a five-node chain went from 43.9 to 103.1 msg/s with p50 latency 2110ms →
782ms and p95 3913ms → 1439ms; one source into twenty consumers went from 5.4
to 33.7 msg/s. In memory, twenty consumers went from 187 to 448 msg/s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf
2026-08-26 10:12:25 +02:00
stroblmeandClaude Opus 5 a9136c7811 A rate limit now thins the work, not only the messages
The limit was applied in `apply_outputs`, which the executor reaches after the
item is off the queue — so a subscriber told to publish every 15s still cost a
queue entry, a `cascade_started`, a run record and a walk of everything
reachable from it per inbound message. Seven relay nodes behind one inverter
ran 192 times a minute to publish six.

Two halves, matching the two shapes it takes:

`trigger()` now keeps a value whose every port is inside its window and
journals nothing at all. The window split came out of `_throttled` as a
read-only `_window_split`, so the question is asked the same way in both
places and the exact split is still made once, at claim time.

A cascade carries the names it actually published, and the wave runs only the
nodes something in that set feeds. A node whose triggering inputs were all
held back is completed without running, which frees its own consumers to be
judged the same way — the case where a node re-published 619 messages a minute
off inputs that changed six times. Redeliveries and emissions carry no such
set and still walk everything, since one has a half-finished wave to finish
and the other is the value already being in state.

Skipping a node can make one ready that the scheduling pass has already walked
past, so `submit_ready` runs to a fixpoint. That also closes the same latent
hole on the replay path, where a done-marker skip could strand a join with no
future outstanding to come back for it.

Measured with the new `scripts/bench_engine.py`, 500 messages through the
house's shape: a limited source went from 500 cascades / 3500 node runs /
5009 events to 1 / 7 / 19, publishing the same 8 values; an unlimited source
into limited relays took the node reading them from 500 runs to 1.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf
2026-08-26 09:55:56 +02:00
stroblmeandClaude Opus 5 5726c80948 Say how much work is waiting, not just how much is running
`RedisWorkQueue.stats` read XPENDING, which counts entries delivered to a
consumer and not yet acknowledged — work in progress. Entries sitting in the
stream undelivered were counted nowhere, so an engine hours behind reported
itself idle: on the house, `pending: 4` while the group's lag was 1554.

The group's own `lag` is the missing number. `backlog` now carries it on both
queues (`len(_items)` in memory), leads the health tile, and a sustained one
publishes `engine_degraded` from the timer thread — named with the flow most
of the waiting work belongs to, sampled from the undelivered tail, since that
is the actionable half. It is a summary problem rather than a /utils/health
503: a backlog should not restart the container.

Also drops the keyspace `scan_iter` `stats()` did per poll to count parked
items — it walked every state and idempotency key twice per ten seconds — for
a set the park/unpark path maintains.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf
2026-08-26 09:43:23 +02:00
stroblmeandClaude Opus 5 dd7db026e1 Announce an acknowledged node failure on the bus
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m38s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m38s
pre-commit / pre-commit (push) Failing after 2m46s
Test Backend / test-backend (push) Successful in 2m17s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Failing after 1m2s
Clearing a node's last error on the engine published nothing, so a second
browser kept the marker until its next snapshot. One event carries the
qualified node; the receiving client drops the marker without refetching.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 22:33:09 +02:00
stroblmeandClaude Opus 5 5ba379490a Dash the transmit ring so it is not the selection ring
Both were a 2px inset --primary ring, so a widget mid-publish and one
picked in the editor looked the same. Status has three colours and none
of them means "picked", so the stroke tells them apart instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 22:17:15 +02:00
stroblmeandClaude Opus 5 f567c98de4 Say a dashboard is read-only in the editor's view mode too
The lock notice was on `PanelSurface` alone, so /dashboards/{name} showed
disabled controls with nothing naming the state. Same pill, not a second
one; edit mode keeps its controls live, so it stays out of there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 22:15:09 +02:00
stroblmeandClaude Opus 5 9ad44fd3f6 Draw the edge sparkline only for a value it can plot
The popover showed a non-numeric value twice: only numbers are recorded,
so MessageSparkline fell through to a collapsed ValuePreview and the
inspector rendered its own expanded one below it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 22:13:32 +02:00
stroblmeandClaude Opus 5 1148e54c9e A run says whether the dashboard, the CLI or a script asked for it
`POST /runs/flows/{name}` hardcoded `cause: "api"`, so every row in the
history claimed the same origin. The body now carries an optional `cause`,
closed to the values the column knows — the dashboard sends nothing and stays
"api", `fluksio run` says "cli", and the SDK client says "sdk".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 22:06:29 +02:00
stroblmeandClaude Opus 5 b194071ca5 Pick the run this test made, not the first one running
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m56s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m48s
pre-commit / pre-commit (push) Failing after 3m2s
Test Backend / test-backend (push) Successful in 2m26s
Compose Smoke Test / test-compose (push) Successful in 30s
Playwright Tests / merge-reports (push) Failing after 2m0s
The suite runs in a random order and a run left marked running by an
earlier module lands in the same list, so asserting on data[0] failed on
whichever seed put that module first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 21:52:48 +02:00
stroblmeandClaude Opus 5 3af5342a2b Advisory issues read as advice rather than failure
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m37s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 35s
Playwright Tests / merge-reports (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
`ADVISORY_ISSUES` moves next to `ValidationIssue` in pipeline.py, and the
model derives an `advisory` flag from its own code, so the distinction the
engine already made ships to the client instead of being re-guessed there.

The dock keeps its summary in `--destructive` only when a real fault is
among the issues and paints an advisory row `--muted-foreground`; the
canvas leaves advisories off a node's dot and border entirely, since node
status has three colours and no warning tier.

biome checks the generated `openapi.json`, which nothing formats since the
SDK script dropped its format pass — ignore it like the other generated
files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 21:46:40 +02:00
stroblmeandClaude Opus 5 4a2337f4de Drop the unreachable 404 from save_dashboard
Docs / docs (push) Successful in 31s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m39s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m40s
pre-commit / pre-commit (push) Failing after 2m54s
Playwright Tests / merge-reports (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 1m10s
Compose Smoke Test / test-compose (push) Canceled after 0s
write_draft creates a first draft for a name that does not exist yet, so
it raises only StaleVersion — the DashboardNotFound arm never ran.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 21:36:02 +02:00
stroblmeandClaude Opus 5 49e6bc6c4e Stop the SDK generator reformatting the whole frontend
biome.json excludes src/client, so the trailing format pass never
touched a generated file — it only rewrote unrelated sources on every
backend commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 21:33:11 +02:00
stroblmeandClaude Opus 5 d3a0b4c3a4 Panel spec keeps the installation's other panels
Saving panels replaces the whole list, so the spec's setup deleted every
panel it had not created — and with it the credential of the screen on
it. It now appends to the stored list and puts the original back in
teardown, which also removes its own panel.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 21:30:35 +02:00
stroblmeandClaude Opus 5 65f859c04d Land a paired panel on its own screen instead of a new code
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
2026-08-25 21:24:41 +02:00
stroblmeandClaude Opus 5 a3a234756c A batch flow's input is a run parameter, not a missing value
Docs / docs (push) Successful in 20s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m59s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m40s
pre-commit / pre-commit (push) Failing after 2m49s
Test Backend / test-backend (push) Successful in 2m20s
Compose Smoke Test / test-compose (push) Successful in 30s
Playwright Tests / merge-reports (push) Failing after 1m6s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UytviPMJbXzD8P84nLvXcq
2026-08-25 18:51:54 +02:00
stroblmeandClaude Opus 5 3503512d05 Report how long a run has been going, not just how long it took
Docs / docs (push) Successful in 20s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m39s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m39s
pre-commit / pre-commit (push) Failing after 2m51s
Playwright Tests / merge-reports (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 1m4s
Compose Smoke Test / test-compose (push) Canceled after 0s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UytviPMJbXzD8P84nLvXcq
2026-08-25 18:33:28 +02:00
stroblmeandClaude Opus 5 055f4d7d97 Filter the icon grid, and ask before removing a panel
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UytviPMJbXzD8P84nLvXcq
2026-08-25 18:33:28 +02:00
stroblmeandClaude Opus 5 476dc6a1e9 Make the brand CTA visible and fix two more v3 variable classes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UytviPMJbXzD8P84nLvXcq
2026-08-25 18:33:28 +02:00
stroblmeandClaude Opus 5 df22475f54 Claim only what the cascade pool can run
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UytviPMJbXzD8P84nLvXcq
2026-08-25 18:33:28 +02:00
stroblmeandClaude Opus 5 68711e2ee7 Start the docs site with the integrated local stack
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UytviPMJbXzD8P84nLvXcq
2026-08-25 18:33:28 +02:00
stroblmeandClaude Opus 5 8679b0bcf7 Draw a wall panel at the size it was drawn for
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m41s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m40s
pre-commit / pre-commit (push) Failing after 3m16s
Test Backend / test-backend (push) Successful in 2m18s
Compose Smoke Test / test-compose (push) Successful in 30s
Playwright Tests / merge-reports (push) Canceled after 0s
A dashboard whose canvas matches the screen came out at 95% of it: the
panel and view routes inset the canvas by a further 16px, on top of the
margin the grid already keeps from its own edges. Two margins, one of
them costing the scale. The padding now applies only to the stacked
phone layout, which has no canvas to shrink.

A slider's tick labels are the third row of a control that has three,
and the first thing a short tile runs out of; `ticks: false` drops them
and keeps the reading and the track. The house's two sliders set it.

The 1024x600 preset is named for what it is rather than for a diagonal
it may not have — the same resolution is sold as a seven-inch panel and
as a ten-inch one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012n6CehUsHYYUXJ48rxaD18
2026-08-25 17:30:37 +02:00
stroblme 4350916bd8 Scroll the runs table, pick a range, and choose what a comparison plots against
Docs / docs (push) Successful in 35s
Playwright Tests / test-playwright (1, 2) (push) Failing after 3m14s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m44s
pre-commit / pre-commit (push) Failing after 3m54s
Test Backend / test-backend (push) Successful in 3m12s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Failing after 1m28s
2026-08-25 16:32:34 +02:00
stroblmeandClaude Opus 5 3c15964364 A cached node keeps its curve, and any input can name a run's output
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m42s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m41s
pre-commit / pre-commit (push) Failing after 2m53s
Test Backend / test-backend (push) Successful in 2m21s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Failing after 1m6s
A cache hit still replays no emissions — those values were the story of an
execution that is not happening — but the run they were recorded in is now
written on the row (`run_node.cached_from`), and the metrics endpoints read the
series back from there. So a reused run answers `run.metrics("train.loss")`
with the same points the run that trained did, rather than looking like a run
that produced no numbers at all. Pointed at rather than copied: a sweep of 500
reusing one frozen node would otherwise duplicate its curve 500 times.

That needed the cross-flow restore fixed first. The cache key has no flow in
it while the stored outputs are named for the flow that produced them, so
`quick.prepare` getting a hit from `train` wrote `train.dataset` into `quick`'s
state and the next node was called without its argument. One rule now covers
both halves: `requalify` reads a name owned by one flow as the same name in
another, applied to the restored outputs, to the node id behind the pointer,
and to the series names on the way out. Reuse across flows is kept.

Also: `@run:<id>.<output>` and a bare `sha256:` digest resolve on every input,
not only artifacts. Chaining a run's json config into the next one from a shell
meant pasting the whole object inline, and the CLI could not even send the
spelling — `_coerce` died in `json.loads` before the engine saw it. Both
spellings are reserved on every input now, `str` included, and `_from_run`
returns whatever the run's result holds rather than only a reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dp9L6gakMVro1K2C5zdtBE
2026-08-25 15:12:28 +02:00