Commit Graph
100 Commits
Author SHA1 Message Date
Melvin Strobl e0f5ddc3a4 Devices may publish to their own out ports' messages (a level set on the device itself)
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 36s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m13s
Test Backend / test-backend (3.12) (push) Successful in 4m32s
Test Backend / test-backend (3.14) (push) Successful in 4m18s
Compose Smoke Test / test-compose (push) Failing after 26s
Playwright Tests / merge-reports (push) Failing after 2m16s
2026-10-29 21:58:38 +01:00
Melvin Strobl d2e8c01785 Getting started: own settings go in compose.override.yml, uv waits 120 s for a download 2026-10-29 21:35:50 +01:00
Melvin Strobl 00cda9d2f1 connector: wanted(port) says whether anyone watches or reads a port 2026-10-27 15:20:24 +01:00
Melvin Strobl 7b453af3fe wren node: calibration setting, passed to wren's training as the installation's calibration records 2026-10-26 22:17:48 +01:00
Melvin Strobl ba75c78820 wren node: install hints and docs pin the public v2.0.0 tag 2026-10-25 17:17:17 +01:00
Melvin Strobl b443240f92 up 2026-10-25 14:56:25 +01:00
Melvin Strobl 82c64a10b6 Make documentation standalone and generalize examples 2026-10-25 14:43:20 +01:00
Melvin Strobl 96ba512d40 wren node: untrained targets left out of the prompt, a read beside a held-back set goes on whole 2026-10-25 12:09:37 +01:00
Melvin Strobl aa49972087 docs: no wren model makes brightness calls yet 2026-10-23 12:21:43 +02:00
Melvin Strobl 4708265c85 Bump certifi to 2026.10.19 (eMudhra CA distrust) 2026-10-23 08:57:14 +02:00
Melvin Strobl a34d47b24f wren node writes the house brightness from lights.set/dim/brighten (wren s11) 2026-10-22 22:43:33 +02:00
Melvin Strobl 613eb8a744 wren node takes switches (plug, socket, appliances), refused when the model lacks the kind 2026-10-22 17:48:41 +02:00
Melvin Strobl 6b575a4989 wren applies all/every commands itself, answers temperatures, reads aliases and number words; tts says units and clock times
wren's model answers "turn off all lights" with one call, and at 0.98 a
living room light switched on. The node now applies such a command to every
wired target of the kind. Number words become digits ("in five minutes"
lost its delay), aliases put the house's words and known mishearings in the
model's, and sensors.temperature is answered from <area>_temperature or
outdoor. tts writes out units and clock times before speaking.
2026-10-22 12:34:40 +02:00
Melvin Strobl f1452f3119 wren sees every area and reads the command plain
Shown only the wired areas, the model took "turn on the bathroom light"
for the living room's light and "the office light" for the kitchen's,
both above 0.93: 5 of the 28 recorded 7B commands switched a wrong light.
Every area is shown now, with a light each, so such a call names an
unwired device and goes on to `unhandled`.

With every area shown, a transcript's "Turn on the kitchen light." fell
to 0.80. The command is passed lower case without sentence marks, as the
model was trained, which brings it to 0.99.
2026-10-20 08:34:07 +02:00
Melvin Strobl 1a8f6875c4 Assistant confirms a command from the values it set, without a second round
run_turn takes answer_tools: false ends the turn once the tools have
answered. The assistant node uses it and says what was set ("Kitchen light
off, living room light off."), which saves one model round, about 1.1 s on
the P100. A turn without a call still streams the model's own words.

rebuild-frontend keeps the running stack's FRONTEND_HOST, so the token
issuer no longer flips between the LAN address and app.localhost.
2026-10-20 08:26:14 +02:00
Melvin Strobl a286cf4ceb Assistant node, and the Control tab as one conversation (openpanel V5)
run_turn takes its tools, system prompt and tool caller; the chat passes the
MCP tools as before. The assistant node offers one tool per wired target,
reads its other inputs as context, streams its reply a sentence at a time,
warms the provider on `listening` and says so when the provider is not there.
Completions share the per-server HTTP client. wren replies when nothing reads
`unhandled`, stt shows a note when nothing was heard. /ai/control lists one
conversation per entry node with the messages to watch.
2026-10-19 23:23:25 +02:00
Melvin Strobl 4df7bdb057 Speech calls keep one connection per server, opened when listening starts
transcribe and speak share an httpx client per speech server with a 300 s
keep-alive, and run on the engine's loop. stt warms it with GET /v1/models
when an utterance opens, so a reply's first sentence finds the connection
open: a new one over a lossy link cost up to a second (openpanel V4).
2026-10-19 22:06:08 +02:00
Melvin Strobl f50d8b9570 tts speaks MP3: a seventh of WAV's bytes over a slow link to the server
A sentence's length for `append` comes from the Xing header LAME writes into
the first frame. The cache key carries the format, so no WAV is reused.
2026-10-19 21:55:17 +02:00
Melvin Strobl 16383ea6e2 Voice nodes (openpanel V4): stt, tts, wren reply/unhandled, socket publish
- `stt`: one realtime transcription session per utterance on the speech
  server's url, opened at `listening` true or the first chunk, committed at
  false; a whole clip goes to /v1/audio/transcriptions. Returns at once and
  publishes `partial` and `text` when the server answers.
- `tts`: one /v1/audio/speech call per sentence, cached by voice and text in
  node state; a reference is marked `append` while earlier ones still play.
- wren: `reply` says what was applied, `unhandled` carries the prompt when
  nothing was. The install hint names the git pin (PyPI's `wren` is another
  package).
- ai/client: transcribe, speak and transcribe_stream beside stream_completion.
- A device socket publishes scalar `in` ports as {"type": "publish"} text,
  in order with its binary frames.
- Pipeline.publish walks only what its values reach: a node downstream of a
  consumer that published nothing no longer runs again (wren re-acted on
  every microphone chunk).
2026-10-19 21:34:02 +02:00
Melvin Strobl 85dad39973 Service node: a server kept running on a worker (openpanel V2)
- `service` node type: command, port, worker label, health path, env and
  weights (an artifact digest the worker fetches as {weights}). Its flow
  starts it; it restarts with backoff, reads degraded while starting or
  without a worker and down after three starts that never got healthy, and
  publishes http://<worker>:<port> once /health answers 200, "" when it ends.
- Worker protocol 3: a `serve` call, every call in its own process group,
  killed when the engine ends the call, the socket drops or the agent gets
  SIGTERM. `--service-host` overrides the address the engine sees.
- Services and compiles take no `--parallel` slot, and a flow's detail is
  built off the event loop, so a save no longer stalls the engine.
2026-10-19 18:43:04 +02:00
Melvin Strobl 80e0201612 Binary uplink takes audio chunks beside image frames (openpanel VS4) 2026-10-19 16:04:39 +02:00
Melvin Strobl d18ddad789 Docs: a camera's snapshot_min_s, hold_s at least 2, person on connect 2026-10-17 10:44:14 +02:00
Melvin Strobl 72c1d73e96 Camera permission and review fixes (openpanel C3, C4)
C4: a device's `viewers` (superusers by default, or members) covers the
image and video messages its ports publish. Anyone else gets none of their
values on the socket, in the catalogue or a flow's state, 403 for their
history, for the frames they hold (the last 8 included) and for TURN
credentials, no bytes when asking for them, and `bye: forbidden` for a live
view. A device credential stays bounded by its dashboards. The device page
sets it in its Camera section, and a tile the viewer may not see says
"No access to this camera" and asks the camera for nothing.

C3:
- A volatile upload is held to MAX_FRAME_BYTES (4 MiB, 413 above it, read
  without extra copies), and so is a frame on a device's socket.
- A frame the engine holds in memory is written to the ring directory
  before a local worker is handed it, so load_artifact finds it.
- A socket the bus dropped events for catches up with a snapshot, demand
  included, and a device's credential is checked again per frame, cached by
  the flow store revision and its nonce, so a lost event never keeps an
  unpaired device's frames coming.
- A report is at most 64 KB once its named config_schemas are put in place.
- A device away for 60 s has its true bool messages set to false.
- Refused frames are logged by fixed keys, a frame is named after its
  message, `fluksio serve` waits 5 s for open connections, a tile with a
  stream URL creates no demand, the 8-name cap keeps the newest names.
- Every container's log rotates (json-file, 3 x 10 MB).
2026-10-17 09:20:04 +02:00
Melvin Strobl 6a98dcd23f Live view over WebRTC (openpanel L1, server and browser)
A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.

A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.
2026-10-17 08:37:39 +02:00
Melvin Strobl efd883d092 Access log: the pairing poll's ?secret= and a socket's ?ticket= are masked too 2026-10-16 21:57:35 +02:00
Melvin Strobl 464490bc37 Docs: keeping a camera's snapshot 2026-10-16 21:15:15 +02:00
Melvin Strobl 312f35be3e Dev: a reload waits at most five seconds for open connections 2026-10-16 20:46:42 +02:00
Melvin Strobl ecc4bee4bb Cameras on devices (openpanel C1): frames in memory, a binary uplink, demand
- Volatile frames are held in memory with a byte-bounded queue instead of
  the /dev/shm ring scanned on every put; workers keep writing to the
  directory, which is trimmed on a timer. A volatile upload no longer spools
  to disk. ARTIFACT_VOLATILE_BYTES=0 turns the ring off.
- A device sends camera frames as binary messages on its socket, held and
  published in one step and credited to its device node; refused frames are
  logged and dropped. Its snapshot carries demand and the uplink (binary, or
  http through the portal).
- One demand registry for every socket's wanted media, the tunnel's
  included, plus what running flows read, mapped onto on-demand ports and
  published as device_demand; at most 8 media names per socket.
- Per-message deflate off in serve and dev; a device may fetch the last 8
  digests of a message; published dashboards cached per store revision; a
  revoked device socket is closed with 1008.
- Media tiles want bytes only while on screen; the Devices tab previews a
  wired camera; the device node warns when a flow reads an on-demand port.
2026-10-16 20:43:14 +02:00
Melvin Strobl d79203a5f7 Pairing limits: no shared bucket for portal calls, a cap per side
Every call through the portal arrives off the tunnel, so the per-address limit
skips those (the hub limits them by the real address), and the cap of 50 counts
and evicts portal and local requests apart, so a portal flood never pushes out
a device waiting on this network.
2026-10-16 19:33:33 +02:00
Melvin Strobl c604cfdf3b Device security and robustness (openpanel C0): check code, pnc, lenient report
- POST /devices/pair answers a 2-character `check`, listed by /devices/pending
  and shown beside each waiting entry; five codes a minute per address, and at
  50 waiting the oldest gives way instead of a 429
- portal-minted device tokens are asked for with the device's nonce (`pnc`) and
  held against it, so unpair and delete revoke them; one without `pnc` stands
  only while the nonce is 0
- a device credential acts as the device (`device_principal`), not as the
  approving superuser, so deleting that account unpairs nothing
- the report is validated port by port: unknown enum values read as `unknown`,
  unreadable ports and sections are left out and logged, `config` may name an
  entry of `config_schemas`; 422 only for no schema 1 or no object
- devices.json is parsed once per change (cached by stat), delete drops the
  report and keeps the nonce, a busy recheck runs in the background
- scope and io only from enabled flows (start/stop tells the devices), a port
  bound twice gives one io row
- the Devices tab's saves cancel and hold back device refetches until settled
2026-10-16 19:26:05 +02:00
Melvin Strobl 13d23f6baf Restart safety: transient credential checks are 1013/503, not a refusal
- A device credential that cannot be checked now (device store not running,
  nonces unreadable) raises DeviceUnavailable: 503 over HTTP, and the
  /flows/ws handshake and the tunnel stream accept and close with 1013.
  1008 (a 403 before accepting) stays for a revoked or invalid one
- Device files are written through a temporary file, so a read mid-save no
  longer looks like a deleted device
- The first report turns Touch friendly on only for a device paired since
  pairings are noted (DeviceState.paired_at), so devices paired before keep
  the switch they were given
- NodeDef reads `device`/`device_policy` from older clients as the worker
  label, for a release
- Tests: transient vs revoked socket, the touch default, the old worker keys,
  and a restart with a device node and a node_health rule sending no push
2026-10-16 18:18:42 +02:00
Melvin Strobl 4286acadfd Capability report: PUT /devices/{id}/report, a Capabilities section, offline devices degraded
A device reports its board, screen, camera and ports from its own
credential (16 KB, schema 1, replacing the last report), and the flows
are validated against it again. The node panel turns a reported port's
config schema into a form, a reported screen is a canvas preset and a
touch screen's first report turns Touch friendly on. An offline device
node is degraded: amber on the canvas, a warning on Home, never a fault.
2026-10-16 17:37:49 +02:00
Melvin Strobl 1dc891aa1b A node's worker label is worker, so a device is only a paired one
- NodeDef `device`/`device_policy` become `worker`/`worker_policy`, the
  issue `ignored_device` becomes `ignored_worker`, and the SDK's `@node`
  takes `worker=` and `worker_policy=`
- FlowStore._migrate renames the keys in published and draft documents
  once, bumping their version; a queued run's `needs` reads the old key
- placement passes the label as `label`, client regenerated
- docs: worker placement wording, MCP issuer without DOMAIN, device rows in
  the CLI data directory and the API table, device links in compose comments
2026-10-16 16:56:05 +02:00
Melvin Strobl 29d3ec467f Device node: a device's ports are wired in flows
- flow/nodes/device.py: `device` node with params `device` and `config`;
  provides the device's `in` ports, requires its `out` ports without
  waking anything, never run (Node.passive), down "Offline" from presence
- DeviceDef.io and DevicePort go; GET /devices/{id} computes `io` and
  `bindings` from the device nodes of published flows, in a fixed order,
  cached by FlowStore.revision; the credential's scope comes from them
- issues `unknown_device` and `device_port` (wired twice; against a report
  the wrong direction or dtype, or unlisted as advisory), rechecked when a
  device is created or deleted; ValidationIssue.advisory is a field
- a device publishing on a bound port is credited to its node by the
  server; the panel endpoint kind is gone from `_endpoints` and the canvas
- a rebuild that moves a device's bindings sends `dashboard_changed` with
  an empty name, which the firmware refetches on
- startup moves each device's `io` rows into device nodes in the flows
  declaring their messages, draft and published, one commit; rows no
  flow declares are logged and left out
- frontend: Device node settings (device, Publishes, Follows, a port
  picker from a report), palette shortcuts per reported port, DeviceDetail
  lists bindings with "Use in flow" in place of the I/O editor
- docs: devices, flow editor, node types, issues, API
2026-10-16 16:40:31 +02:00
Melvin Strobl 91e3b703f9 Merge branch 'main' of git.stroblme.de:Fluksio/app 2026-10-16 15:56:28 +02:00
Melvin Strobl 28a565e501 Panels become devices: per-device documents in the flow repo, a Devices tab
- flow/devices.py: DeviceDef in FLOWS_DIR/_devices/<id>.json, committed under
  the flow store's write lock, 409 on a stale version; nonces in DEVICES_FILE
  outside git; in-memory presence announced as device_status
- api/routes/devices.py: CRUD, pairing with an optional {name, board,
  firmware}, a pending list adopted without a code from the LAN only, unpair;
  /panels/pair, /panels/pair/{code} and GET /panels/{id} kept as aliases,
  PUT /panels/ gone
- _device_may replaces _panel_may, JWT claims unchanged
- panels.json moves into devices once at startup, then .migrated
- MCP list_devices, get_device; search category device
- Devices tab (list, detail with presence, pairing, ordered dashboards,
  touch, I/O), PanelsDialog removed, serialised optimistic saves
2026-10-16 15:48:36 +02:00
Melvin Strobl bfab6e1d2e Counter wrap stays on the step grid: it counts in span = top - lo + the finest step rather than the period, so one counter of step 5 up to 100 wraps 100 + 5 to 0 and 0 - 5 to 100; the time picker is unchanged. Check cases and docs formula 2026-10-16 09:00:36 +02:00
Melvin Strobl 21c7961d49 Counter widget: one to three linked counters are the digits of one number, published on one message in the finest counter's unit (hours and minutes send 07:30 as 450, 07:55 + 5 min carries to 08:00); wraps or stops at the coarsest's min/max. The arithmetic is one pure module, ui/core/counter.ts, with counter.check.ts. A tile steps each counter with a press up and down and publishes once 300 ms after the last; a round face turns the selected counter on the ring (wheel, keys, a drag counted in steps), its arc where that counter stands in its turn, and a press on a counter's digits selects it. Default config is the time picker. Editor rows, widgetIssue, server type and dtype, docs 2026-10-16 08:58:42 +02:00
Melvin Strobl 4ce50b5c3a LAN stack gives out its LAN address: dev-lan sets FRONTEND_HOST to the published port on the host's default-route address, so the Panels dialog links a screen somewhere it can reach instead of app.localhost 2026-10-15 22:22:58 +02:00
Melvin Strobl f600f46924 Merge remote-tracking branch 'origin/main' 2026-10-15 21:17:33 +02:00
Melvin Strobl e5a953c11c Name pill held its full 1000 ms once faded in, as the docs and openpanel count it 2026-10-15 10:04:07 +02:00
Melvin Strobl 300d9017fd Merge remote-tracking branch 'origin/main' 2026-10-15 07:47:21 +02:00
Melvin Strobl d33c18af55 Dashboard name pill opaque: the look's card colour with a hairline border, no blur 2026-10-14 22:19:34 +02:00
Melvin Strobl 09ce4798c3 Dashboard name pill on a round panel: 56 high, icon and title at 24, at most 280 wide 2026-10-14 22:16:00 +02:00
Melvin Strobl 210023b7e2 Round panel: the dashboards either side drawn in advance and live, on the face they open on, and dragged in up and down as faces are sideways; the name pill only on landing 2026-10-14 17:20:22 +02:00
Melvin Strobl d8e9fbda26 Round paging follows the finger: faces and dashboards on a track, swiped one to one with the next beside it, turned or sent back on release, rubber-banded at the ends; keys and dots slide the same track; the reading dot a smaller companion of the handle 2026-10-14 17:08:51 +02:00
Melvin Strobl c294789d85 Round faces: a bar as concentric arcs with a legend in the middle; larger title, gap line and face text; Read-only in the ring's gap; a round dashboard unpadded on a phone; reduced-motion ring assertion 2026-10-14 16:52:46 +02:00
Melvin Strobl bae4b15505 Marks as dots inside the track: ticks, a selector's options and a slider's reading on the track's midline and the ring's centre line, coloured by the fill or the handle over them; the selector ring fills to its option 2026-10-14 16:42:56 +02:00
Melvin Strobl 42c752c94e Dashboard motion: round dashboards page up and down, the ring eases to a notch or a reading, the slider tile to a value set elsewhere; reduced motion honoured by every tween; motion spec in the docs 2026-10-14 16:34:10 +02:00
Melvin Strobl 66e6c4b6ec Slider: the set value to its step's decimals; the reading marked on the track and on the ring; mark geometry in the docs 2026-10-14 16:00:53 +02:00
Melvin Strobl d05d3c4665 Slider reading in a tile: a line under the value in every look, row and fader; Beside it on every dashboard; tile geometry in the docs 2026-10-14 15:51:04 +02:00
Melvin Strobl 2008150692 Round dashboards: one face at a time, a face per widget type, paged by swipe, dots and keys; slider reading; face row in the editor 2026-10-14 15:24:25 +02:00
Melvin Strobl be3446cb16 Round dashboards: shape and ring, the ring drawn and turned in every look, Knob preset and face guide in the editor 2026-10-14 14:23:55 +02:00
Melvin Strobl 9bed078c41 Gauge: no value arc at the minimum
At 0 the value arc is a zero-length dash (pathLength 0), and its round
caps still painted a primary dot on the dial in every look. The arc is
now transparent while its fraction is not above 0. The widget spec puts
a gauge at 0 on its panel and checks it in each look; its tile count,
stale since the buttons widget, is 11.
2026-10-14 12:29:07 +02:00
Melvin Strobl 13e885ee4b Docs: panels without a browser (openpanel), device-interpreted I/O ports 2026-10-14 11:33:27 +02:00
Melvin Strobl 13a0c69edc Colour widget: commit the value being released
The brightness column changes and commits in one handler, so the commit
read the render's draft (still null) and sent nothing; the queued draft then
stuck and rode along with the next release. The pending colour lives in a ref
now. color-drag.spec gains a column drag.
2026-10-14 08:44:38 +02:00
Melvin Strobl a5afd88a5e LAN proxy: no 1 MB upload cap 2026-10-14 08:14:19 +02:00
Melvin Strobl 0264310d4f Defer ffmpeg in the image; record the decision 2026-10-14 07:33:59 +02:00
Melvin Strobl 1964c5b2e8 Clip transcoding: ffmpeg in the backend image, panel recipe in the node docs 2026-10-14 07:30:36 +02:00
Melvin Strobl 659799d1e5 Panel I/O: PanelDef.io ports, scoped publish and upload, I/O section in the Panels dialog 2026-10-13 21:10:54 +02:00
Melvin Strobl 0e42fbfdb2 link
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-18 14:22:47 +02:00
Melvin Strobl eb8f703396 analytics
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-18 13:17:07 +02:00
Melvin Strobl e0fdf0751b comments
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-16 00:34:45 +02:00
Melvin Strobl 6d2e25c2ca Let agents drive the flow API over MCP
The engine now speaks MCP at /mcp, with a built-in OAuth 2.1 authorization
server in front of it: an agent registers itself, sends a human to the browser
to approve it, and exchanges the resulting code for a token. PKCE is required,
codes are single-use and stored only as hashes, the browser is redirected to
the URI that was registered rather than the one asked for, and refresh tokens
rotate so that replaying a spent one revokes the whole line.

Twenty tools cover reading, building, publishing and running flows, and each
one calls the same REST endpoint the dashboard calls, in-process, carrying the
caller's own token. That keeps one description of what a flow is and what may
be done to it — validation, the draft/publish split, the version check — and
means an agent can do nothing a person could not do in the browser.

Agent tokens are RS256 with a keypair of their own rather than the secret that
signs browser sessions, so deleting the key withdraws every agent without
logging anyone out, and deps.decode_token grew the branch that trusting a
second issuer will need when the hosted login arrives.
2026-08-16 00:22:41 +02:00
Melvin Strobl cb5e8f0ecc Add the connector contract, reusable nodes and per-port intervals
Connectors are the device-facing node class third parties write, so the
surface they build against is versioned and documented: ConnectorNode carries
a declared contract version, a polling loop that publishes only what changed
and reports health around it, and parameters whose credential fields are
marked x-secret so the editor offers the secrets store instead of a text box.
They are found through the fluksio.node_types entry point group, with the
package's own metadata as the manifest. docs/connectors/ has the contract and
the authoring guide; connector-skeleton/ is a working one to copy.

The controller no longer knows what any node type is: start, stop and
report_health are protocol methods on Node, and the built-ins were migrated to
them first, so the hooks a connector implements are the ones the engine has
been driving all along.

Marking a node reusable moves its source to _lib/ and points the node at it by
name. Other flows instantiate it with their own ports and settings, one fix
reaches all of them, and a shared source still in use cannot be deleted.

Ports gained an interval: an output publishes, and an input wakes its node, at
most every n seconds. State keeps the latest value, so only the delivery is
skipped, and pressing Run is never throttled.

Also fixes autosave sending no version on its first save of a session, which
made every flow saved more than once conflict with itself.
2026-08-15 23:57:44 +02:00
Melvin Strobl 05dccdf88d Start, stop and pause flows, and show what their nodes print
Flows can now be taken off the engine and put back. Stopped state lives in a
runtime.json beside the flow, not in the flow document: the canvas autosaves
that document, so a stopped flow would otherwise start itself again on the
next edit. A stopped flow gets no subscriptions, schedules or webhooks, its
nodes are skipped by the scheduler, and running it answers 409. Pausing holds
a flow's nodes while its values keep arriving, so the canvas still shows what
is coming in.

Node code is user code and print is how it says things, so stdout is teed
through a contextvar sink active only during a node execution — one event per
execution, capped, so a chatty node cannot outrun the stream. A node that
fails sends its traceback the same way, trimmed to the author's own frames.
The dock gains a logs panel and a pause control; the dashboard replaces its
placeholder with what is running, stopped or failing; the edge inspector can
send the last message again.

Single-stepping is deferred and noted: the scheduler keeps no progress between
calls, so a step button would re-run the same node rather than advance.
2026-08-15 23:35:08 +02:00
Melvin Strobl 7698289776 Separate editing from running with a draft/publish split
Edits autosave to flow.draft.json and nodes.draft/ instead of the files the
engine reads, so the pipeline keeps running the published version until
someone publishes. Every save carries the version it was based on: a second
client editing the same flow is refused with 409 and offered the choice
between their version and its own, rather than silently overwriting.

Draft saves no longer rebuild the pipeline; validation and node status for a
draft come from a throwaway build that never touches live state.

Also fixes a latent bug where an empty state backend is falsy, so Pipeline
quietly built itself a second, private state and left message history empty.
2026-08-15 23:13:15 +02:00
Melvin Strobl 97b496c9fa Drop the InfluxDB rotation item, note the orphaned flow state
The token does not need rotating. Deleting a flow does leave its value,
timestamp, version and history keys behind in Redis.
2026-08-15 22:07:31 +02:00
Melvin Strobl 2e93c40717 Clear the notepad of what this pass closed
Records what the work turned up on the way: no way to reach a stored
secret from the params form, flow-level edits outside the undo stack,
the mobile panel having no test id, and the Playwright specs leaving
their users and flows in the development database.
2026-08-15 22:00:18 +02:00
Melvin Strobl 506b6fd50b Sparkline every message a node names
A number in the panel says where a value is, not where it has been. Each
named port in Consumes and Provides now carries a sparkline under it,
fetched once on open and extended from the socket as values arrive, held
to the same 120-point window the server keeps.

Both lists render through the shared port row, so a sink node shows what
flows through it rather than nothing. A flat series draws its mid line
and drops the redundant range label, a lone reading is just its dot, and
a series spanning decades switches to a log scale with the labels left
in real units.
2026-08-15 21:59:30 +02:00
Melvin Strobl c9e369fa39 Regenerate the frontend SDK
Picks up the message history endpoint.
2026-08-15 21:40:09 +02:00
Melvin Strobl 10041416b5 Give the panel the canvas, and open a flow already fitted
Opening a node or the flow settings now takes the flowbar and the dock
away, leaving the graph and the panel; they come back on close, leaving
by the same slideUp they arrive with. The panel header names the flow
before the node, `/heating/`, which is how messages are qualified
everywhere else.

Opening a flow already fitted, and pressing Fit to screen, disagreed:
one used a 0.25 padding capped at 1x, the other xyflow's defaults, so
the button visibly jumped and a small flow opened at 100% in an empty
canvas. Both read one constant now.
2026-08-15 21:40:09 +02:00
Melvin Strobl e23dc179de Say what went wrong on a node in one line
The bubble opened with the loader's own frame — `exec(compile(...))`
inside `_load_function` — before anything about the user's code, and
then said all of it twice, because a load failure arrives both as the
node's status and as a node_error issue carrying the same text. It is
now `Type: message (line N)`, deduplicated, with the line taken from the
node's own file. Full tracebacks still go to the server log.

Long messages grow the bubble instead of stretching it, and scroll only
past 240px. The tooltip arrow is gone: it read as a rotated square notch
under the bubble, and no better anywhere else it appeared.
2026-08-15 21:39:59 +02:00
Melvin Strobl 10d7024f85 Record a short value history per message
The panel is going to sparkline how a value moved, and nothing kept more
than the latest one. Emissions now append to a capped list beside the
value — 120 points, enough to fill a sparkline without making Redis a
time-series store — and `/flows/{flow}/history/{message}` hands it back
oldest first.

Only numbers are recorded, bools included as neither, so a string
message costs nothing at all. The response carries `numeric` so an empty
series reads as "not plottable" rather than "nothing yet". The append is
one pipelined round-trip per emission batch and sits outside the lock,
since the list is append-only.

MemoryState keeps the same window in a deque, so the endpoint answers
without Redis too.
2026-08-15 21:39:59 +02:00
Melvin Strobl 1819cefb60 Regenerate the frontend SDK
Picks up the unauthenticated_hook validation issue.
2026-08-15 21:20:05 +02:00
Melvin Strobl 78b9f94cab Add the Gitea Actions workflows
Ports n3xd's four: pre-commit, backend tests, sharded Playwright and a
compose smoke test. Everything comes from the committed .env.example, so
no repository secrets are needed; Actions still has to be enabled per
repository and a runner registered with the ubuntu-latest label.

container_name, published ports and image tags are all daemon-global, so
two shards on one runner would fight over them. compose.ci.yml resets
them and tags per project. test-backend keeps the fixed names because it
reaches Postgres from the runner host.
2026-08-15 21:20:05 +02:00
Melvin Strobl 235dca5915 Add undo on the canvas, and sharpen the flow chrome
Deleting a node cost its source with no way back. Every mutation already
funnels through one commit, so undo/redo is a bounded stack of node
snapshots replayed through the same debounced save. Deleting a node only
drops it from flow.json — the source file survives — so restoring the id
restores the code. Renaming a message now offers to follow the rename
across every node still bound to the old name, as one undoable step.

Autosave never fired: flush depended on the whole mutation object, which
react-query rebuilds every render, so the effect re-ran and its cleanup
cancelled the pending timer. Unmounting now flushes rather than drops.

The canvas looked blurry zoomed out because Background scales the dot
radius by zoom, leaving quarter-pixel dots on a drifting tile; radius
and spacing now divide the zoom back out, spacing in octaves so the grid
halves. Edge value labels are opaque, the edge popover fits its summary
on one row with a trash icon and scrolls names that overflow, and flow
settings moved to the flowbar. The flowbar was sized against the
viewport rather than the canvas, so its buttons left the screen once
enough flows were open.
2026-08-15 21:19:53 +02:00
Melvin Strobl e600fbfcd2 Put settings in the sidebar and let the sidebar frost composite
The footer was a user card with the theme picker and log out buried in
its dropdown. It is now two plain entries, Settings and Log Out, and the
theme moved into the settings page as its own tab.

The dark sidebar never matched the canvas because _canvas.tsx painted
bg-card across the whole viewport underneath it, so the floating
panel's bg-card/80 resolved to card over card and its gutter was card
too. Nothing there should paint a surface; the panel now reads the same
as the other frosted chrome.

The superuser guard sliced the first three tabs, which silently changed
meaning when a fourth arrived.
2026-08-15 21:19:41 +02:00
Melvin Strobl 01ffd82955 Guard webhooks with a per-hook secret, and allow rotating the app key
Trigger hooks are mounted unauthenticated because devices cannot present
a JWT. They now take a secret as a trailing path segment, so a device
needs one URL and no header support. The value never enters the
registered route, only a {secret} template, and is compared with
compare_digest; a mismatch is a bare 404 so the endpoint does not
confirm which hooks exist. Pointing the parameter at the encrypted store
keeps the literal out of flow.json. Hooks without a secret keep working
and now raise a validation issue saying so.

Rotating SECRET_KEY made the stored secrets unreadable for good, since
the Fernet key derives from it. scripts/rotate_secret_key.py re-encrypts
with the new key and refuses if the old one does not decrypt. Now that
recovery exists, an unreadable store fails loudly instead of coming back
empty and leaving flows short of credentials with no visible cause.
2026-08-15 21:19:32 +02:00
Melvin Strobl 52fe9c4c0b Keep flow state across a restart, and tighten the flow typing
The engine fell back to MemoryState because there was no Redis to talk
to, so every value was lost on restart. Adds a redis service with
append-only persistence, reachable only from the backend, and points
REDIS_HOST at it.

state.py and pipeline.py leave the strict-mypy exclusion. redis-py types
its calls as sync/async unions; the casts narrow that at the points of
use rather than switching to redis.asyncio, which would make a
deliberately synchronous backend asynchronous. nodes.py stays excluded.

The .env.example CORS entry belongs with the CI work that follows.
2026-08-15 21:19:21 +02:00
Melvin Strobl bca9ce4058 Give the tests their own database and drop the template specs
pytest was deleting every user on teardown against the development
database. The engine is built at import time, so tests/__init__.py pins
POSTGRES_DB before app.core.config loads; the fixture creates the schema
and drops the database again, guarded against a name that is not _test.

Playwright now defaults at the integrated stack, which is the origin the
API allows, so a bare `bunx playwright test` works without a dev server.
The four template specs that asserted copy we no longer ship are gone,
along with the helpers they were the last callers of. The admin edit
assertion was page-wide and only ever passed on a fresh database.
2026-08-15 21:19:12 +02:00
Melvin Strobl 037df4c75f Give the pulse its own duration, and let the editor fill the screen
The shared duration tokens had been stretched to slow the pulses down, which
also slowed every panel and dock animation and left `motion.ts` out of step
with the CSS. Both are back to 200/300 ms, and the pulses use a new
`--duration-pulse` (500 ms): a message arriving is a signal, not a state
change, and it is the only thing here that should linger. The edge pulse read
its own 300 ms constant, so it now takes the token through `motion.ts` and the
two sides cannot drift again.

The code editor also expands: the same node panel fills the content region
beside the sidebar, the canvas chrome steps aside rather than floating on top,
and the settings above the editor keep a readable width.
2026-08-15 20:03:21 +02:00
Melvin Strobl df6b60363e Note the motion token drift between CSS and motion.ts 2026-08-15 19:47:09 +02:00
Melvin Strobl bd3b9ac5c5 Add flow settings, pulse emitting nodes, and simplify node state
- One dot per node now carries the whole story: primary while running, sage
  after a good run, red when anything is wrong, with the explanation on hover.
  The corner badge is gone, along with the second way of saying the same thing.
- A node that publishes something flashes a ring, so a running flow is legible
  without reading the edge values. Nodes that consume but publish nothing stay
  quiet, which is why the event carries an output count.
- Flow settings open in the same panel its nodes use, from a pencil in the
  dock: the title, the name, and deleting the flow. NodePanel and FlowPanel
  share the panel chrome rather than each drawing their own.
- Renaming is a server operation, because a flow's name is the namespace of its
  messages: the directory moves and every other flow reading `old.message` is
  repointed, instead of being left pointing at a flow that no longer exists.
2026-08-15 19:46:23 +02:00
Melvin Strobl d3325b3f6e Recover from a dead session, and report a node problem once
An expired or orphaned session left the app on a half-rendered page instead of
the login screen: a properly signed token naming a user who no longer exists
answered 404, which the client does not treat as an authentication failure. All
failures in get_current_user are 401 now, and the client stops retrying them,
so a dead session goes straight back to the login screen.

- Clicking an edge names the two nodes it runs between, not just the message.
- A node with a problem shows one badge carrying the whole explanation, rather
  than a corner badge and a status dot saying the same thing twice. The dot is
  back to what it is good at: whether the node ran.
- The sidebar's collapse control sits in the sidebar, where a phone still finds
  one in the chrome because there is no sidebar on screen to hold it.
2026-08-15 19:19:57 +02:00
Melvin Strobl 11dab5ae30 Note the test-teardown and token findings
Running the backend suite against the development database wipes its users,
and the old InfluxDB token turns out to be in pushed history rather than only
in the working tree.
2026-08-15 18:50:46 +02:00
Melvin Strobl 39f3b35356 Fix the MQTT node, suggest message names, and drop Items
- MQTT nodes failed to build: the topic map was renamed to talk in ports, but
  __slots__ still declared the old name, so every MQTT node raised
  AttributeError. Building one of each node type is now a test, since __slots__
  makes this failure invisible until someone places the node.
- Port names offer the messages already in play: everything published is worth
  reading, and an input nobody provides yet is worth publishing. A message only
  connects when both ends spell it the same way, so choosing beats typing.
- Adding a port focuses its name field.
- Dragging onto an input that already reads something offers the extra port as
  well as the replacement — an MQTT or InfluxDB node usually wants both.
- The template's Item model, its routes, screens and table are gone.
2026-08-15 18:43:17 +02:00
Melvin Strobl 1d6f39a5de Add the flow editor: canvas, node panel and live values
The browser half of M3. Flows open on a full-bleed canvas with their chrome
floating over it: flow tabs top, dock bottom, node settings in a panel on the
right that leaves the graph visible and running behind it.

- Connections are derived, not stored. A node declares the messages it reads
  and publishes; every matching pair draws an edge, so two producers of one
  message converge on their consumer. Dragging output to input is shorthand
  for pointing that input at the producer's message, and asks before it
  replaces an existing one.
- Values land on the edges as they flow, over a websocket that feeds a store
  outside React, so a value arriving re-renders its own chip and nothing else.
  Clicking an edge shows the last payload and when it arrived.
- Node source is edited in Monaco, loaded only when a panel opens and themed
  from the design tokens.
- Edits autosave; identical documents are skipped server-side, so a quiet
  canvas writes nothing.
- Validation from the API shows on the node it belongs to and is summarised in
  the dock, where each entry pans to its node.
- Works on a phone: touch-connect, 44px dock targets, and the node panel
  becomes a full-screen sheet.

Two new tokens (--status-success, --font-mono) are mirrored in the website repo
and recorded in DESIGN-GUIDELINES.md.
2026-08-15 18:10:50 +02:00
Melvin Strobl e8895faa47 Add the flow API: typed messages, git-backed store, REST and live events
Makes the flow engine reachable from the API, which is what M3 needs before
any of it can reach the browser.

- app/flow is a package now; the prototype's watch-dir scripts and the
  matplotlib/networkx visualiser are gone with their dependencies.
- Messages carry a serializable dtype instead of a live Python type, and a
  port name, so the graph can speak qualified names while node functions keep
  local arguments. Redis state is JSON, not pickle.
- Message names are namespaced per flow ("heating.temp"); a bare name resolves
  to its own flow, a dotted one crosses flows.
- Several nodes may provide the same message: producers are a list, so fan-in
  is a real edge instead of a silently dropped one.
- Flows are stored as flow.json plus node sources in a git repository, one
  commit per save, with identical saves skipped so autosave stays quiet.
- Node failures are isolated and reported per node; validate() returns cycles
  and unconnected inputs instead of raising deep in a run.
- Credentials live in an encrypted store and are referenced as {"$secret": …}.
- Engine events reach websocket clients through a bus, so values, node status
  and execution show up live.
2026-08-15 17:23:36 +02:00
Melvin Strobl ec19e2beda Pull the connector node contract forward in the management phase 2026-08-15 15:41:47 +02:00
Melvin Strobl bd9a49a0ef up
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-15 14:48:16 +02:00
Melvin Strobl 552167c7d4 Add parallel stateless-node invocation to the processing phase 2026-08-15 14:42:08 +02:00
Melvin Strobl 539d53fc7c Fold typed-message and UI-managed-secrets decisions into the roadmap 2026-08-15 14:33:02 +02:00
Melvin Strobl e3f595ee60 Point ROADMAP at the milestone-level master in the workspace 2026-08-15 14:20:35 +02:00
Melvin Strobl 4261dfe163 Rename the workspace layout: fluksio -> app, website -> index
The repo moves to git.stroblme.de:Fluksio/app.git and its sibling to
Fluksio/index.git, so every cross-repo path reference moves with it. The
architecture sources the roadmap cites now live in the docs repo under
architecture/, and the brand marks in the index repo's src/assets.

Also replaces the placeholder README with one that reflects the actual layout.
2026-08-09 16:16:08 +02:00
Melvin Strobl 1481fab1cf Green the lint gate and drop the local-stack host ports
- exclude app/flow from ruff and mypy: it is not importable as a package and
  nothing in the API reaches it, so 405 of the 412 findings were about code
  scheduled for restructuring (see NOTEPAD.md)
- fix the 7 real findings outside it: col(...) around created_at for the
  .desc() ordering, and a type: ignore that is no longer needed
- compose.local.yml resets the host ports compose.dev.yml publishes; the
  integrated stack is served entirely through Traefik, and a port already
  taken on the host used to fail the whole stack
2026-08-09 15:52:57 +02:00
Melvin Strobl 3ea599e78d Rewrite ROADMAP and NOTEPAD from the real requirement sources
ROADMAP.md was an empty scaffold carried over from another project, pointing
at a docs-private/architecture/ directory that does not exist. Rebuild it
from org/README.md and org/structure.canvas as five phases matching the
intended component split, and seed NOTEPAD.md with the findings from the
architecture survey (flow package not importable, tests outside tests/, CI,
chart tokens).
2026-08-09 15:36:47 +02:00
Melvin Strobl ba73bbe253 De-template the frontend and apply the design system
- FastAPI branding replaced throughout: title, favicon, logo (now the org/
  brand SVGs), footer links, route titles; workspace package renamed
- shadcn primitives restyled to the component radius/shadow map: pill
  controls, rounded-lg surfaces, shadow-e1/e2/e3
- floating frosted AppSidebar over a bg-card content region
- theme storage key aligned with the pre-paint script, MotionConfig added
- scripts/capture-screenshots.mjs backs the root 'make verify'
2026-08-09 15:33:56 +02:00
Melvin Strobl 41eadce966 Adopt the shared design token contract
Replace the stock shadcn OKLCH neutral palette (whose teal --primary had no
relation to the brand) with the fluksio token block duplicated verbatim from
the root DESIGN-GUIDELINES.md, add the self-hosted Inter variable face, the
shared lib/motion.ts presets, a DESIGN.md pointer stub and the
no-arbitrary-font-size hook.
2026-08-09 15:24:50 +02:00
Melvin Strobl e1a783c120 Restructure docker into layered compose files, add a Makefile
Move compose.yml/compose.override.yml/compose.traefik.yml into docker/ and
split into the explicit prod -> dev -> local layering; compose.override.yml
had to be renamed because docker compose auto-loads that filename, which
defeats the layering.

- external network traefik-public -> proxy (shared with the website stack)
- frontend host dashboard.${DOMAIN} -> app.${DOMAIN}
- stable container_names, security_opt no-new-privileges on prod services
- adminer bound to 127.0.0.1 in dev instead of all interfaces
- .env.example replaces the committed .env
- pre-commit biome hook ran npm in a bun repo
2026-08-09 15:21:34 +02:00