Commit Graph
46 Commits
Author SHA1 Message Date
Melvin StroblandClaude Opus 5.5 f679264b3a Camera permission and review fixes (openpanel C3, C4)
C4: a device's `viewers` (superusers by default, or members) covers the
image and video messages its ports publish. Anyone else gets none of their
values on the socket, in the catalogue or a flow's state, 403 for their
history, for the frames they hold (the last 8 included) and for TURN
credentials, no bytes when asking for them, and `bye: forbidden` for a live
view. A device credential stays bounded by its dashboards. The device page
sets it in its Camera section, and a tile the viewer may not see says
"No access to this camera" and asks the camera for nothing.

C3:
- A volatile upload is held to MAX_FRAME_BYTES (4 MiB, 413 above it, read
  without extra copies), and so is a frame on a device's socket.
- A frame the engine holds in memory is written to the ring directory
  before a local worker is handed it, so load_artifact finds it.
- A socket the bus dropped events for catches up with a snapshot, demand
  included, and a device's credential is checked again per frame, cached by
  the flow store revision and its nonce, so a lost event never keeps an
  unpaired device's frames coming.
- A report is at most 64 KB once its named config_schemas are put in place.
- A device away for 60 s has its true bool messages set to false.
- Refused frames are logged by fixed keys, a frame is named after its
  message, `fluksio serve` waits 5 s for open connections, a tile with a
  stream URL creates no demand, the 8-name cap keeps the newest names.
- Every container's log rotates (json-file, 3 x 10 MB).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138N7k5PT4dcR1iEqXaaQ9k
2026-10-17 09:20:04 +02:00
Melvin StroblandClaude Opus 5.5 02b736c9e8 Live view over WebRTC (openpanel L1, server and browser)
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 4m37s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m19s
pre-commit / pre-commit (push) Failing after 2m25s
Test Backend / test-backend (3.12) (push) Successful in 4m57s
Test Backend / test-backend (3.14) (push) Successful in 4m41s
Compose Smoke Test / test-compose (push) Successful in 28s
Playwright Tests / merge-reports (push) Successful in 1m38s
A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.

A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138N7k5PT4dcR1iEqXaaQ9k
2026-10-17 08:37:39 +02:00
Melvin StroblandClaude Opus 5.5 50e6e22da4 Dev: a reload waits at most five seconds for open connections
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138N7k5PT4dcR1iEqXaaQ9k
2026-10-16 20:46:42 +02:00
Melvin StroblandClaude Opus 5.5 62c0b915f3 Cameras on devices (openpanel C1): frames in memory, a binary uplink, demand
- Volatile frames are held in memory with a byte-bounded queue instead of
  the /dev/shm ring scanned on every put; workers keep writing to the
  directory, which is trimmed on a timer. A volatile upload no longer spools
  to disk. ARTIFACT_VOLATILE_BYTES=0 turns the ring off.
- A device sends camera frames as binary messages on its socket, held and
  published in one step and credited to its device node; refused frames are
  logged and dropped. Its snapshot carries demand and the uplink (binary, or
  http through the portal).
- One demand registry for every socket's wanted media, the tunnel's
  included, plus what running flows read, mapped onto on-demand ports and
  published as device_demand; at most 8 media names per socket.
- Per-message deflate off in serve and dev; a device may fetch the last 8
  digests of a message; published dashboards cached per store revision; a
  revoked device socket is closed with 1008.
- Media tiles want bytes only while on screen; the Devices tab previews a
  wired camera; the device node warns when a flow reads an on-demand port.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138N7k5PT4dcR1iEqXaaQ9k
2026-10-16 20:43:14 +02:00
Melvin StroblandClaude Opus 5.5 a9d453c9d9 A node's worker label is worker, so a device is only a paired one
- NodeDef `device`/`device_policy` become `worker`/`worker_policy`, the
  issue `ignored_device` becomes `ignored_worker`, and the SDK's `@node`
  takes `worker=` and `worker_policy=`
- FlowStore._migrate renames the keys in published and draft documents
  once, bumping their version; a queued run's `needs` reads the old key
- placement passes the label as `label`, client regenerated
- docs: worker placement wording, MCP issuer without DOMAIN, device rows in
  the CLI data directory and the API table, device links in compose comments

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 16:56:05 +02:00
Melvin StroblandClaude Opus 5.5 ed06bdec6b Panels become devices: per-device documents in the flow repo, a Devices tab
- flow/devices.py: DeviceDef in FLOWS_DIR/_devices/<id>.json, committed under
  the flow store's write lock, 409 on a stale version; nonces in DEVICES_FILE
  outside git; in-memory presence announced as device_status
- api/routes/devices.py: CRUD, pairing with an optional {name, board,
  firmware}, a pending list adopted without a code from the LAN only, unpair;
  /panels/pair, /panels/pair/{code} and GET /panels/{id} kept as aliases,
  PUT /panels/ gone
- _device_may replaces _panel_may, JWT claims unchanged
- panels.json moves into devices once at startup, then .migrated
- MCP list_devices, get_device; search category device
- Devices tab (list, detail with presence, pairing, ordered dashboards,
  touch, I/O), PanelsDialog removed, serialised optimistic saves

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 15:48:36 +02:00
stroblmeandClaude Opus 5.5 48aac3855b One image for the whole app: engine, API and dashboard on one port
The backend image now builds the SPA and serves it on the API's own origin
(fluksio/spa.py), so an instance is a single container with no proxy, DNS,
port 80 or .env: `docker compose up -d` with the new compose.yml runs it on
localhost:8080. It starts on fluksio serve (persisted secret key, first
admin printed once) and brings its own Redis when REDIS_HOST is unset.

Slimmer: python:3.13-slim, no dev group, uv once, fastapi without the cloud
CLI. Optional features are per-feature extras (mqtt, influx, email, sentry,
ml, server for all) picked with the EXTRAS build argument; numpy moved to
`ml`. ffmpeg left the image for imageio-ffmpeg via the Modules screen.

The operator stack runs the same image with app. and api. routed to it; the
nginx frontend service, the LAN nginx proxy and DOMAIN in the backend are
gone, and the MCP issuer is FRONTEND_HOST.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:40:02 +02:00
Melvin StroblandClaude Opus 5.5 0c7f8bde82 LAN stack gives out its LAN address: dev-lan sets FRONTEND_HOST to the published port on the host's default-route address, so the Panels dialog links a screen somewhere it can reach instead of app.localhost
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m51s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m5s
pre-commit / pre-commit (push) Failing after 2m24s
Test Backend / test-backend (3.12) (push) Successful in 4m9s
Test Backend / test-backend (3.14) (push) Successful in 4m14s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Successful in 1m45s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016ezWcLmBCtTVKFPxfdK2MK
2026-10-15 22:22:58 +02:00
stroblmeandClaude Opus 5.5 cdded499b1 LAN nginx proxies to the container name fluksio-api, which the test stack's backend does not carry
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgBfkm4rCmtd1T4vHWjEPx
2026-10-15 17:07:22 +02:00
stroblmeandClaude Opus 5.5 0ac78764c8 make test-frontend runs against a stack of its own
The suite creates and deletes flows, dashboards and users, and ran against
whatever stack was up, which is the dev instance's data. make test-stack starts
the compose project fluksio-test (compose.test.yml over the local stack's
files) with its own containers, volumes and image tags, served by the local
Traefik on app.test.localhost and api.test.localhost, and leaves it running.
The hostname is fixed, so it no longer comes off the running dev stack, and a
LAN stack makes no difference. The suite's default origin is the test stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgBfkm4rCmtd1T4vHWjEPx
2026-10-15 16:15:32 +02:00
stroblmeandClaude Opus 5.5 e903b1ba1d Pin Traefik to 3.6.25 in the dev and production compose files
A floating 3.6 tag left a host on an older cached image that refuses
--accesslog.fields.queryparameters, so the proxy exited on start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgBfkm4rCmtd1T4vHWjEPx
2026-10-15 15:56:14 +02:00
stroblmeandClaude Opus 5.5 c5c12376fc Opt-in WITH_FFMPEG build arg installs Debian's ffmpeg in the backend image
Off by default, since it adds about 400 MB. Documented with the clip recipe
and in the configuration reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgBfkm4rCmtd1T4vHWjEPx
2026-10-15 15:55:02 +02:00
stroblmeandClaude Opus 5.5 f8d14be8e6 Dev backend reloads on worker edits too: fastapi dev with --reload-dir over backend and worker, worker in develop.watch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgBfkm4rCmtd1T4vHWjEPx
2026-10-14 23:34:07 +02:00
stroblmeandClaude Opus 5.5 0b4eb70452 LAN proxy: serve /hooks next to /api, so webhooks and served sockets reach the backend instead of the SPA
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WgBfkm4rCmtd1T4vHWjEPx
2026-10-14 23:24:32 +02:00
Melvin StroblandClaude Opus 5.5 340b4f3447 LAN proxy: no 1 MB upload cap
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:14:19 +02:00
stroblmeandClaude Opus 5.5 71b0f986e9 Keep query-string tokens out of the uvicorn, nginx and Traefik access logs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013GAeg4Jo6xjZzjwy9QDWMQ
2026-10-06 17:32:51 +02:00
stroblmeandClaude Opus 5.5 b0c601c6ed Enrol over http only to loopback; serve the API docs to this machine
The enrolment exemption covered any *.local host, which mDNS answers with
whoever claims it; it is now localhost, 127.0.0.0/8 and ::1. /docs, /redoc
and openapi.json stay closed in production and otherwise answer only a
loopback caller unless API_DOCS_ENABLED is set, which the dev compose does,
so `fluksio serve --host 0.0.0.0` no longer hands the schema to the network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013GAeg4Jo6xjZzjwy9QDWMQ
2026-10-06 12:20:13 +02:00
stroblmeandClaude Opus 5.5 8cfe5c4e6e Count per-address limits on the hop a trusted proxy added; limit signup
The limits read the leftmost X-Forwarded-For entry, which the client writes,
and the LAN nginx appended to it. The address now comes from uvicorn's
proxy-headers middleware (rightmost hop not in FORWARDED_ALLOW_IPS), applied
in the app too so tunnel-replayed calls resolve the same way; nginx sets the
header to $remote_addr and the compose backend trusts the Docker networks.
/users/signup gets a per-address limit of 10 an hour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013GAeg4Jo6xjZzjwy9QDWMQ
2026-10-06 12:15:58 +02:00
stroblmeandClaude Opus 5 d471614e6a Push a frame instead of storing and fetching it
The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.

`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.

The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.

Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
2026-09-02 10:15:14 +02:00
stroblmeandClaude Opus 5 d01a8dad37 Rename Installation to Instance
Follows the portal: the noun is "instance" everywhere the app says it —
UI strings, CLI output, error details, docs and comments. The wire keys
(`instance_id`, `instance_token`) and the hub route this calls move with it.

An existing cloud.json is adopted rather than refused: without the key
alias the dataclass fails to parse, which the caller swallows and reads as
"never enrolled" instead of "reconnect".

`instance_key` on a node type becomes `target_key`. It means the outside
thing a node points at, which is a different sense of the word, and keeping
both would put two meanings of "instance" in one codebase.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YrQnKV3bnQd4K342y8tKj
2026-08-31 10:12:01 +02:00
stroblmeandClaude Opus 5 45cc7504e1 Notify a phone that has this installation installed
A `webpush` alert channel, and the PWA it needs to arrive. The payload is
encrypted to the subscription (RFC 8291) and the request signed with this
installation's own keypair (RFC 8292), both over `http-ece` — `pywebpush`
does the same in one call but brings `requests` and `aiohttp` with it, two
HTTP stacks beside httpx on a machine that may be a Raspberry Pi.

The manifest and the worker are hand-written rather than `vite-plugin-pwa`:
there is nothing worth precaching when the page carrying the credential is
`no-store`, so the worker handles `push` and `notificationclick` and nothing
else. `registration.scope` is the app's root in both places it runs, which is
why the payload carries no URL.

A run finishing in error is the first event worth waking someone for; `ok`
and `cancelled` describe to nothing, so a nightly batch that works stays
quiet. The events were already on the bus — only the filter changed.

`WEBPUSH_FILE` is a derived path, so the keypair lands on the data volume
with the alerts beside it. Off it, a rebuild would silently stop every phone
being notified: the key they subscribed against would be gone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014EbeFPm6WNC3YD9vrqqT3a
2026-08-30 12:12:37 +02:00
stroblme 93e45274d0 Let a teardown's cancellation through, and reap the workers it leaves
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m12s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m49s
pre-commit / pre-commit (push) Failing after 2m16s
Test Backend / test-backend (push) Successful in 2m38s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Successful in 1m15s
Node stop paths cancelled their background task and then caught
CancelledError around the await, which swallows a cancellation aimed at
the caller — the trap Supervisor._cancel already documents. One shared
Node._cancel_task now waits the way the supervisor does; mqtt's publisher
and subscription and delay's cron call it.

The api container also collected zombie python workers: orphaned when
--reload replaces the process holding their handle, they reparent onto a
PID 1 that reaps nothing but its own. `init: true` on the backend service.
2026-08-28 11:27:01 +02:00
stroblmeandClaude Opus 5 c33fa404a4 Keep the dev bind mounts out of CI, where they land nowhere
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m4s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m36s
pre-commit / pre-commit (push) Successful in 3m0s
Test Backend / test-backend (push) Successful in 2m6s
Compose Smoke Test / test-compose (push) Successful in 30s
Playwright Tests / merge-reports (push) Failing after 1m2s
compose.dev.yml mounts the source over the image's own copy so `--reload`
means something in a detached local stack, and mounts the Playwright report
out to the host. Under act_runner the daemon is a sibling of the job rather
than its host: it resolves those paths against its own filesystem, finds
nothing, and mounts empty directories. So the backend reported "Path does not
exist fluksio/main.py" for a file the image plainly ships, and every shard
uploaded nothing — "No files were found with the provided path:
frontend/blob-report".

compose.ci.yml now puts backend's volumes back to what compose.yml declares
and drops Playwright's entirely; the shard copies its report out of the
container before the teardown instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 21:42:04 +02:00
stroblmeandClaude Opus 5 775d151307 Mount the worker source in dev so it cannot fall behind the engine
The engine and the worker speak a version-matched protocol, but compose.dev.yml
mounted only backend/fluksio: a reloaded engine talked to the worker baked into
the image. Adding the request id to that protocol therefore left every python
node failing with "the worker answered request None", because the old worker
echoed no id at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StpRc2C6au1WJ1EUU7fsfu
2026-08-23 19:15:06 +02:00
stroblmeandClaude Opus 5 ce465e7b0e Stop the flow store leaking git zombies, honour VITE_API_URL from .env
`FlowStore._git` now passes `-c gc.auto=0`, so `git commit` no longer forks a
background `gc --auto` that reparents onto PID 1 and stays there unreaped. With
auto-gc off nothing packs on its own, so `_commit` runs a foreground `git gc`
every 500 commits — the trigger a long-lived seeding session actually reaches.

`docker/compose.yml` takes the frontend's `VITE_API_URL` build arg from the
environment, keeping `https://api.${DOMAIN}` only as the fallback. `setup.sh`
already derives the scheme from `ENVIRONMENT`, so an `up --build` that does not
layer `compose.local.yml` stops shipping a bundle that calls `https://api.localhost`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01StpRc2C6au1WJ1EUU7fsfu
2026-08-23 16:49:46 +02:00
stroblmeandClaude Opus 5 ba707c8051 nodes: what porting the house needed from the vocabulary
Four small things, each with a device behind it.

An MQTT filter now routes what it subscribed to. `+` and `#` reached the
broker and were then looked up in an exact-match dict, so every message a
wildcard subscription received was dropped in silence.

`json_key` lifts a value out of the object a device wraps it in — Victron
publishes `{"value": 47}` on every path, which was otherwise a Python node
per port.

The trigger node learned `passthrough` and `wait_port`, because how long to
wait can be a value rather than a constant: a rollershutter takes 26 seconds
up and 28 down. A wait of zero sends nothing afterwards and still cancels
what the last message scheduled, which is how a stop is commanded once
instead of forever.

The HTTP sender takes fixed `query` parameters, so an API key is a secret
reference rather than a message on the canvas, and `send_inputs` off for a
request whose inputs are only a trigger.

Also: `delay` accepts fractional seconds, and `TZ` reaches the container, so
a cron expression means local time. Left unset it is UTC, as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 14:16:08 +02:00
stroblme 1fe712e7fc Add make dev-lan: the app on a host port, API proxied on the same origin 2026-08-22 09:25:02 +02:00
stroblmeandClaude Opus 5 11e032386b Publish the documentation site: docs.fluksio.com
A zensical site under docs/, served by a new `docs` compose service behind
Traefik, built with --strict in CI. Same pattern the sibling n3xd workspace
uses.

Getting started splits the way the landing page does — one path is
`pip install fluksio` and a training script, the other is a Docker stack and
an afternoon in the browser — because the two audiences will not spend the same
amount of time. Everything after that is shared: the concepts, the web
interface (app and portal), the CLI and the API, and a reference for node types,
payload types and configuration.

The three flow guides move here from the docs submodule rather than being
copied, so there is one version of them.

Styling mirrors DESIGN-GUIDELINES.md: the app's token palette remapped onto
Material's variables in both schemes, Inter, the 16px panel radius, and the one
terracotta accent spent on the facility lane of the audience split.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M7Xv3cJEW5c8AXxn2hoojV
2026-08-22 05:55:34 +02:00
stroblmeandClaude Opus 5 961a8f881d Keep the engine's state in SQLite, not Postgres
One process owns this database — the image has run a single uvicorn
worker for that reason since the four-engines bug — so a file beside the
flows is the honest shape for it, and it is what lets `fluksio serve`
need no infrastructure at all. Live values, node execution and the work
queue never came here anyway; what does is a rollup a minute at a time,
a row per cascade and the run history, and WAL keeps the readers going
while that one writer works.

DATA_DIR is now the one setting that moves everything an installation
keeps; the rest derive from it and the images still spell theirs out.
The schema is prepared in-process at startup, so the prestart service is
gone, and the ten Postgres-only revisions collapse into one portable
baseline.

Three things only worked because psycopg was casting for us: a token's
subject arriving as a string where the column is a UUID, `greatest`, and
`date_bin`. The timestamps needed a column type of their own — SQLite
stores no offset, and a naive datetime read back either raises against an
aware `now` or serialises as local time.

Postgres stays in the stack only for Umami, behind the analytics profile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 22:19:45 +02:00
stroblmeandClaude Opus 5 60d7ec81c0 Rename the import package app to fluksio
A wheel whose top-level module is `app` collides with anything else in a
user's venv, so the package that is about to be published takes the name
it is published under. Only the Python package moves; the repo, the
Docker WORKDIR and the compose project keep theirs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-21 21:48:05 +02:00
stroblmeandClaude Opus 5 c912388ed6 Make lint report instead of rewriting, and stop verify guessing the domain
`make lint-frontend` was `biome check --write --unsafe ./` — a lint target that
reformatted the whole tree rather than checking it, which is why every parallel
change in this repo has had to work around it. `lint` checks now and a new
`format` writes. The pre-commit hook and CI needed no edit at all: both call
`bun run lint`, so they became checks the moment its meaning changed.

`app/Makefile` assigned DOMAIN from .env, and a plain assignment beats an
inherited environment variable and is not exported — so `cd app && make
dev-local` served localhost while the same checkout's tests targeted the
configured domain. `export DOMAIN ?=` gives the lattice that was intended:
command line, then environment, then .env.

Alongside: the backend's htmlcov bind mount created that directory as root, so
`make test-backend` died on the coverage step after every test had passed,
which reads like a test failure and is not one. The alerts screen's copy of
ALERTING_EVENTS is now checked by a test rather than trusted. And the shard
comment claimed two spec files where there are nine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
2026-08-21 14:34:14 +02:00
stroblmeandClaude Opus 5 a8065ad91c Give the work queue its own Redis root, and pin what the suite runs as
The queue shared the `pipeline:` prefix with flow state, so `RedisState.clear()`
could DEL the queue stream and `keys()` enumerated queue entries — only callers
filtering `__`-prefixed names kept it safe. It moves to `queue:` without a
migration: whatever is in flight at the upgrade is dropped once, documented in
DEPLOY.md rather than papered over.

Alongside it: `pool_pre_ping`, so a connection idle across a Postgres restart
costs a round trip instead of a failed request; the test suite pins
ENVIRONMENT=local and DOMAIN=localhost itself rather than inheriting a
deployment's .env; and `depth` leaves the queue stats, where it reported the
capped journal length as if it were a backlog.

ALERTS_FILE and PANELS_FILE now point at /data. They defaulted to a path on no
volume, so alert routing and every wall-panel pairing were living in the
container's writable layer and vanishing on each rebuild. Carrying the existing
files across is a manual step; DEPLOY.md has it.

development.md was still the upstream template — compose.override.yml,
localhost.tiangolo.com, `docker compose watch` as the dev flow — and said
nothing about the Playwright suite. Rewritten against what the Makefiles
actually do. deployment.md was template text too, duplicating the root
DEPLOY.md, and is gone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
2026-08-21 10:10:57 +02:00
stroblmeandClaude Opus 5 41b2c28b2b The e2e suite names its own origins, and refuses a live instance
`tests/utils/api.ts` took the API origin from `VITE_API_URL`, which
`tests/config.ts` loads out of `app/.env`. In a checkout configured for a
deployment that names the deployment — so the browser went to the local stack
while every setup and teardown call, `deleteAll` included, went to the live
one. `privateApi.ts` had the same reading, and it creates users.

Both origins now come from one place: `PLAYWRIGHT_BASE_URL`, with the API
derived from it (`app.<domain>` → `api.<domain>`) or named outright by
`PLAYWRIGHT_API_URL`, which is what CI and the compose service set. Nothing in
the suite reads `VITE_API_URL` any more.

Belt and braces, since a stack served under a real domain answers to the same
names its production instance does: a global setup resolves both origins and
refuses anything that is not loopback or a private range, before a test runs.
`PLAYWRIGHT_ALLOW_PUBLIC=1` says you meant it.

`make test-frontend` is now that safe run — the Playwright image on the proxy
network with both names mapped onto Traefik by address, as the host user so it
does not leave root-owned results behind.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NUb8YpL2s3gmN9WTACTt4q
2026-08-20 20:46:04 +02:00
stroblme 5955dd082e up
Signed-off-by: stroblme <stroblme@posteo.de>
2026-08-20 10:30:12 +02:00
stroblmeandClaude Opus 5 0e7c0038c0 Compose: keep the portal enrolment on the data volume
CLOUD_CONFIG_FILE defaulted to a path relative to the working directory, so the
enrolment — this installation's credential and the portal key it pinned — sat
inside the container and was lost on every rebuild, silently detaching a
connected installation. It belongs beside the OAuth key and the flow secrets,
for exactly the same reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtBzdDyLsmDaF1W7DLYtYM
2026-08-19 22:13:07 +02:00
Melvin Strobl aa40ccb58b analytics
Signed-off-by: Melvin Strobl <lc3267@kit.edu>
2026-08-18 13:17:07 +02:00
stroblmeandClaude Fable 5 2613fbead4 Allow the standalone Vite origin through CORS
`make dev-frontend` serves the SPA from localhost:5173 and talks to the
stack's backend, which refused it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 16:46:35 +02:00
stroblmeandClaude Fable 5 82df345bf0 Give the dev stack a broker, a time-series database, and live backend source
The mqtt and influx node types had nothing to talk to locally; mosquitto and
influxdb fill that in, on the compose network only — no host port, since 1883
usually belongs to a real broker on the machine.

The same file's --reload only ever reloaded the source baked into the image:
develop.watch needs `docker compose watch`, and the dev stack starts detached.
A bind mount is what the standard flow actually reads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 16:46:35 +02:00
stroblmeandClaude Fable 5 b4d5aa6186 Let two backend CI runs share a runner host
The job kept the fixed container names and the fixed 5432 that compose.ci.yml
resets for every other job, so a second run could not start. Layering that
file leaves the db on a port Docker assigns, which the suite is told about,
and the project name is now per run so one teardown cannot take another run's
database with it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7LwYgJfpkbLCTeiAf8U4A
2026-08-16 16:46:25 +02:00
rootandClaude Fable 5 5462842b8a Supervise the engine's host: deep health, loop watchdog, one worker
The API image ran four uvicorn workers, and each one built a full flow
controller — four sets of MQTT subscriptions, cron ticks and webhooks.
Runs one worker now; scaling out is the worker split, not more processes.

Adds a loop-lag watchdog and a deep /utils/health/ that fails when the
event loop is wedged or Redis is unreachable, the two failure modes a
process-alive check never sees. Autoheal restarts on that signal, behind
a compose profile because it mounts the Docker socket.

The private user-seeding routes now need an explicit opt-in rather than
just ENVIRONMENT=local, so a deployment that kept the default never
exposes them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
2026-08-16 07:14:28 +02:00
Melvin StroblandClaude Fable 5 8d82d6c4ec Let agents drive the flow API over MCP
Playwright Tests / test-playwright (1, 2) (push) Canceled after 0s
Playwright Tests / test-playwright (2, 2) (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 0s
Test Backend / test-backend (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Playwright Tests / merge-reports (push) Canceled after 0s
The engine now speaks MCP at /mcp, with a built-in OAuth 2.1 authorization
server in front of it: an agent registers itself, sends a human to the browser
to approve it, and exchanges the resulting code for a token. PKCE is required,
codes are single-use and stored only as hashes, the browser is redirected to
the URI that was registered rather than the one asked for, and refresh tokens
rotate so that replaying a spent one revokes the whole line.

Twenty tools cover reading, building, publishing and running flows, and each
one calls the same REST endpoint the dashboard calls, in-process, carrying the
caller's own token. That keeps one description of what a flow is and what may
be done to it — validation, the draft/publish split, the version check — and
means an agent can do nothing a person could not do in the browser.

Agent tokens are RS256 with a keypair of their own rather than the secret that
signs browser sessions, so deleting the key withdraws every agent without
logging anyone out, and deps.decode_token grew the branch that trusting a
second issuer will need when the hosted login arrives.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 00:22:41 +02:00
Melvin StroblandClaude Opus 5 8a221bfe50 Add the Gitea Actions workflows
Ports n3xd's four: pre-commit, backend tests, sharded Playwright and a
compose smoke test. Everything comes from the committed .env.example, so
no repository secrets are needed; Actions still has to be enabled per
repository and a runner registered with the ubuntu-latest label.

container_name, published ports and image tags are all daemon-global, so
two shards on one runner would fight over them. compose.ci.yml resets
them and tags per project. test-backend keeps the fixed names because it
reaches Postgres from the runner host.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkmeRiyeYmVZqJVwuyHq9o
2026-08-15 21:20:05 +02:00
Melvin StroblandClaude Opus 5 11d491dae0 Keep flow state across a restart, and tighten the flow typing
The engine fell back to MemoryState because there was no Redis to talk
to, so every value was lost on restart. Adds a redis service with
append-only persistence, reachable only from the backend, and points
REDIS_HOST at it.

state.py and pipeline.py leave the strict-mypy exclusion. redis-py types
its calls as sync/async unions; the casts narrow that at the points of
use rather than switching to redis.asyncio, which would make a
deliberately synchronous backend asynchronous. nodes.py stays excluded.

The .env.example CORS entry belongs with the CI work that follows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkmeRiyeYmVZqJVwuyHq9o
2026-08-15 21:19:21 +02:00
Melvin StroblandClaude Fable 5 8c82549cf6 Add the flow editor: canvas, node panel and live values
The browser half of M3. Flows open on a full-bleed canvas with their chrome
floating over it: flow tabs top, dock bottom, node settings in a panel on the
right that leaves the graph visible and running behind it.

- Connections are derived, not stored. A node declares the messages it reads
  and publishes; every matching pair draws an edge, so two producers of one
  message converge on their consumer. Dragging output to input is shorthand
  for pointing that input at the producer's message, and asks before it
  replaces an existing one.
- Values land on the edges as they flow, over a websocket that feeds a store
  outside React, so a value arriving re-renders its own chip and nothing else.
  Clicking an edge shows the last payload and when it arrived.
- Node source is edited in Monaco, loaded only when a panel opens and themed
  from the design tokens.
- Edits autosave; identical documents are skipped server-side, so a quiet
  canvas writes nothing.
- Validation from the API shows on the node it belongs to and is summarised in
  the dock, where each entry pans to its node.
- Works on a phone: touch-connect, 44px dock targets, and the node panel
  becomes a full-screen sheet.

Two new tokens (--status-success, --font-mono) are mirrored in the website repo
and recorded in DESIGN-GUIDELINES.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
2026-08-15 18:10:50 +02:00
Melvin StroblandClaude Opus 5 0c3e6081f2 Green the lint gate and drop the local-stack host ports
- exclude app/flow from ruff and mypy: it is not importable as a package and
  nothing in the API reaches it, so 405 of the 412 findings were about code
  scheduled for restructuring (see NOTEPAD.md)
- fix the 7 real findings outside it: col(...) around created_at for the
  .desc() ordering, and a type: ignore that is no longer needed
- compose.local.yml resets the host ports compose.dev.yml publishes; the
  integrated stack is served entirely through Traefik, and a port already
  taken on the host used to fail the whole stack

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 15:52:57 +02:00
Melvin StroblandClaude Opus 5 529b5f9ed5 Restructure docker into layered compose files, add a Makefile
Move compose.yml/compose.override.yml/compose.traefik.yml into docker/ and
split into the explicit prod -> dev -> local layering; compose.override.yml
had to be renamed because docker compose auto-loads that filename, which
defeats the layering.

- external network traefik-public -> proxy (shared with the website stack)
- frontend host dashboard.${DOMAIN} -> app.${DOMAIN}
- stable container_names, security_opt no-new-privileges on prod services
- adminer bound to 127.0.0.1 in dev instead of all interfaces
- .env.example replaces the committed .env
- pre-commit biome hook ran npm in a bun repo

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 15:21:34 +02:00