113 Commits
Author SHA1 Message Date
stroblme d028bf326a A batch run skips the nodes waiting on a failed one and ends error; a run whose nodes cannot load fails with the load error before anything executes; cancelling a run stops its pipeline without announcing a pause of the live flow; batch_issues refuses a delay node's interval; the client accepts an engine URL with or without /api/v1; RedisState's compare-and-swap records the names it writes 2026-11-04 21:38:21 +01:00
stroblme b7f1d21712 Docs for key renames: ids and keys in the flow concepts, the rename routes, events and 503s in the API reference, hooks during a rename, sync and run metrics after one; a pinned restore refused for its recorded key names that key in the 409
Docs / docs (push) Successful in 3m11s
Playwright Tests / test-playwright (1, 2) (push) Successful in 11m42s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m22s
pre-commit / pre-commit (push) Successful in 3m12s
Test Backend / test-backend (3.12) (push) Successful in 6m19s
Test Backend / test-backend (3.14) (push) Successful in 5m59s
Compose Smoke Test / test-compose (push) Successful in 37s
Playwright Tests / merge-reports (push) Successful in 1m12s
2026-11-03 08:09:35 +01:00
stroblme 53916d869b Clients take a flow's id or key: one resolver in the SDK, id first, explicit id:/key:, a warning for reads and a refusal for changes when a name means two flows, deleted flows found by their last key; sync creates by POST, retries a create race through the drift checks and reports the id; MCP gets create_flow and resolves every flow argument with the caller's token; displays show flow_name; an engine from before ids is named as such 2026-11-02 19:39:14 +01:00
stroblme d4a1b1db47 History is kept by flow id: runs, cascades, rollups and events file the id in their flow column and nodes as <flow id>.<node>, the API answers flow plus flow_name and shows and takes nodes as <key>.<node>; events carry flow_id from the moment they are published; run queue items, the stage cache and cached curves keep ids and keys apart; endpoints, the brain graph, device bindings and search name the id to open 2026-11-02 19:01:19 +01:00
stroblme 08a6376836 Flows and dashboards are created by POST under an issued or superuser-pinned id; PUT only updates; deleted flows leave a tombstone with their key; the controller stores by id and runs by key, hooks live under /hooks/<flow id>; a key with state or queued work behind it is refused until cleared 2026-11-02 17:41:23 +01:00
stroblme fae85f5ecb Flows and dashboards get ids: the store is addressed by id with a key index, the first start gives every document its key as id, fluksio store migrate --dry-run and ls; the rename routes are gone 2026-11-02 16:54:11 +01:00
stroblme 7919cf5670 fluksio requires fluksio-worker 0.2.x: the 0.1 range left the 0.2.0 wheels unresolvable, and publish stopped on it
Publish / publish (push) Successful in 47s
2026-11-01 13:47:01 +01:00
Melvin Strobl d2e8c01785 Getting started: own settings go in compose.override.yml, uv waits 120 s for a download 2026-10-29 21:35:50 +01:00
stroblme 5d0ea480bf Alerts: a Telegram channel, one chat or group each, connected by a one-time link
The channel sends through the Bot API with httpx only: HTML with escaped text,
cut to fit a message, silent for a warning, a short 429 waited out once, and
errors that say what to do about a blocked bot, a lost chat or a bad token and
never quote the request URL. The bot token is a stored secret: saving a
channel refuses a literal one, the card's "Store token" writes the secret and
a reference, and no endpoint returns it.

A chat id is never typed. Connect starts a ten minute code, and the chat that
sends `/start <code>` back through the bot is the one connected, so it is a
chat somebody opened a link for. The engine fetches updates only while a code
is waiting (no standing poll, no webhook, no inbound address) and ignores
everything else without a reply. Connect needs a saved form, and the form
takes the connected chat in without touching what was typed since.

The token is part of every Bot API URL, so the httpx logger gets the same
filter the request lines have, which now also masks an httpx.URL argument.
2026-10-28 13:06:58 +01:00
stroblme 2dc20911a3 Alerts: configuration, test and search entries are a superuser's; subscribing a browser stays open
Any signed-in account could save an ntfy channel with its own `server` and a
`{"$secret": ...}` token and press Test, which sent the stored secret to that
host. The config GET/PUT, the test route and the global search's alert entries
now answer to superusers, like /secrets. The webpush key and subscribe routes
stay open to every account, and the Alerts page shows a non-superuser only the
button for its own browser. The events table in the docs gains run_finished.
2026-10-28 12:40:44 +01:00
stroblme c04e4883c2 MCP: mcp 2.3 (MCPServer, transport arguments on streamable_http_app, bearer read in middleware); tests for the 2026-07-28 revision and the resource check 2026-10-28 09:21:01 +01:00
stroblme 5c7a8773ad SDK: sync and run import the declaring module in a child process
The command line no longer imports a research module, so import-time GPU work
ends with a short-lived child instead of holding a CUDA context for the length
of a followed run. fluksio.sdk.declare starts the child, which writes each
flow's document and shims as JSON; Declared stands in for Flow in sync().
Errors keep their type and message, a child's traceback reaches the user.
2026-10-28 08:11:14 +01:00
stroblme 34243e6902 SDK: artifacts work outside a node, run --follow prints status and queue waits, fluksio cancel
save_artifact/load_artifact outside a node use the worker's own store code in
$FLUKSIO_ARTIFACT_DIR or <tmp>/fluksio-artifacts, so a node can be called by
hand. --follow prints status changes and each node's resource wait; the
placer's waiting snapshot now carries the run. fluksio cancel <run>...
2026-10-28 08:07:35 +01:00
stroblme 4cb028de41 Chat: one phone header button, quick-prompt chips, double-click to reuse a message
The phone header carries a single speech bubble: Control on Home, the
assistant elsewhere. Chips above the input offer the user's most sent
messages per tab, then starters; a tap or a double-click on a sent or
heard message fills the input without sending.
2026-10-27 17:39:36 +01:00
Melvin Strobl 82c64a10b6 Make documentation standalone and generalize examples 2026-10-25 14:43:20 +01:00
stroblme 07fa5a2dc2 wren node: targets from the wiring, on wren 2
Every output is a target and every input besides prompt a reading. A
targets setting gives descriptions, values, ranges and per-target
guards, model takes a trained pack's artifact digest, min_confidence
defaults to the model's threshold. A command is applied whole or not at
all, and targets the model was not trained on read degraded. The home
path (area/kind ports, groups, brightness, temperature answers) is gone.
2026-10-24 21:28:42 +02:00
stroblme 6892651e76 Artifact sweep keeps digests named in node settings
A Service's weights or a wren node's model held only in a setting were
swept once the run that produced them was deleted.
2026-10-24 21:28:35 +02:00
Melvin Strobl a286cf4ceb Assistant node, and the Control tab as one conversation (openpanel V5)
run_turn takes its tools, system prompt and tool caller; the chat passes the
MCP tools as before. The assistant node offers one tool per wired target,
reads its other inputs as context, streams its reply a sentence at a time,
warms the provider on `listening` and says so when the provider is not there.
Completions share the per-server HTTP client. wren replies when nothing reads
`unhandled`, stt shows a note when nothing was heard. /ai/control lists one
conversation per entry node with the messages to watch.
2026-10-19 23:23:25 +02:00
Melvin Strobl 16383ea6e2 Voice nodes (openpanel V4): stt, tts, wren reply/unhandled, socket publish
- `stt`: one realtime transcription session per utterance on the speech
  server's url, opened at `listening` true or the first chunk, committed at
  false; a whole clip goes to /v1/audio/transcriptions. Returns at once and
  publishes `partial` and `text` when the server answers.
- `tts`: one /v1/audio/speech call per sentence, cached by voice and text in
  node state; a reference is marked `append` while earlier ones still play.
- wren: `reply` says what was applied, `unhandled` carries the prompt when
  nothing was. The install hint names the git pin (PyPI's `wren` is another
  package).
- ai/client: transcribe, speak and transcribe_stream beside stream_completion.
- A device socket publishes scalar `in` ports as {"type": "publish"} text,
  in order with its binary frames.
- Pipeline.publish walks only what its values reach: a node downstream of a
  consumer that published nothing no longer runs again (wren re-acted on
  every microphone chunk).
2026-10-19 21:34:02 +02:00
Melvin Strobl 85dad39973 Service node: a server kept running on a worker (openpanel V2)
- `service` node type: command, port, worker label, health path, env and
  weights (an artifact digest the worker fetches as {weights}). Its flow
  starts it; it restarts with backoff, reads degraded while starting or
  without a worker and down after three starts that never got healthy, and
  publishes http://<worker>:<port> once /health answers 200, "" when it ends.
- Worker protocol 3: a `serve` call, every call in its own process group,
  killed when the engine ends the call, the socket drops or the agent gets
  SIGTERM. `--service-host` overrides the address the engine sees.
- Services and compiles take no `--parallel` slot, and a flow's detail is
  built off the event loop, so a save no longer stalls the engine.
2026-10-19 18:43:04 +02:00
Melvin Strobl 80e0201612 Binary uplink takes audio chunks beside image frames (openpanel VS4) 2026-10-19 16:04:39 +02:00
Melvin Strobl 72c1d73e96 Camera permission and review fixes (openpanel C3, C4)
C4: a device's `viewers` (superusers by default, or members) covers the
image and video messages its ports publish. Anyone else gets none of their
values on the socket, in the catalogue or a flow's state, 403 for their
history, for the frames they hold (the last 8 included) and for TURN
credentials, no bytes when asking for them, and `bye: forbidden` for a live
view. A device credential stays bounded by its dashboards. The device page
sets it in its Camera section, and a tile the viewer may not see says
"No access to this camera" and asks the camera for nothing.

C3:
- A volatile upload is held to MAX_FRAME_BYTES (4 MiB, 413 above it, read
  without extra copies), and so is a frame on a device's socket.
- A frame the engine holds in memory is written to the ring directory
  before a local worker is handed it, so load_artifact finds it.
- A socket the bus dropped events for catches up with a snapshot, demand
  included, and a device's credential is checked again per frame, cached by
  the flow store revision and its nonce, so a lost event never keeps an
  unpaired device's frames coming.
- A report is at most 64 KB once its named config_schemas are put in place.
- A device away for 60 s has its true bool messages set to false.
- Refused frames are logged by fixed keys, a frame is named after its
  message, `fluksio serve` waits 5 s for open connections, a tile with a
  stream URL creates no demand, the 8-name cap keeps the newest names.
- Every container's log rotates (json-file, 3 x 10 MB).
2026-10-17 09:20:04 +02:00
Melvin Strobl 6a98dcd23f Live view over WebRTC (openpanel L1, server and browser)
A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.

A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.
2026-10-17 08:37:39 +02:00
Melvin Strobl ecc4bee4bb Cameras on devices (openpanel C1): frames in memory, a binary uplink, demand
- Volatile frames are held in memory with a byte-bounded queue instead of
  the /dev/shm ring scanned on every put; workers keep writing to the
  directory, which is trimmed on a timer. A volatile upload no longer spools
  to disk. ARTIFACT_VOLATILE_BYTES=0 turns the ring off.
- A device sends camera frames as binary messages on its socket, held and
  published in one step and credited to its device node; refused frames are
  logged and dropped. Its snapshot carries demand and the uplink (binary, or
  http through the portal).
- One demand registry for every socket's wanted media, the tunnel's
  included, plus what running flows read, mapped onto on-demand ports and
  published as device_demand; at most 8 media names per socket.
- Per-message deflate off in serve and dev; a device may fetch the last 8
  digests of a message; published dashboards cached per store revision; a
  revoked device socket is closed with 1008.
- Media tiles want bytes only while on screen; the Devices tab previews a
  wired camera; the device node warns when a flow reads an on-demand port.
2026-10-16 20:43:14 +02:00
Melvin Strobl d79203a5f7 Pairing limits: no shared bucket for portal calls, a cap per side
Every call through the portal arrives off the tunnel, so the per-address limit
skips those (the hub limits them by the real address), and the cap of 50 counts
and evicts portal and local requests apart, so a portal flood never pushes out
a device waiting on this network.
2026-10-16 19:33:33 +02:00
Melvin Strobl c604cfdf3b Device security and robustness (openpanel C0): check code, pnc, lenient report
- POST /devices/pair answers a 2-character `check`, listed by /devices/pending
  and shown beside each waiting entry; five codes a minute per address, and at
  50 waiting the oldest gives way instead of a 429
- portal-minted device tokens are asked for with the device's nonce (`pnc`) and
  held against it, so unpair and delete revoke them; one without `pnc` stands
  only while the nonce is 0
- a device credential acts as the device (`device_principal`), not as the
  approving superuser, so deleting that account unpairs nothing
- the report is validated port by port: unknown enum values read as `unknown`,
  unreadable ports and sections are left out and logged, `config` may name an
  entry of `config_schemas`; 422 only for no schema 1 or no object
- devices.json is parsed once per change (cached by stat), delete drops the
  report and keeps the nonce, a busy recheck runs in the background
- scope and io only from enabled flows (start/stop tells the devices), a port
  bound twice gives one io row
- the Devices tab's saves cancel and hold back device refetches until settled
2026-10-16 19:26:05 +02:00
Melvin Strobl 4286acadfd Capability report: PUT /devices/{id}/report, a Capabilities section, offline devices degraded
A device reports its board, screen, camera and ports from its own
credential (16 KB, schema 1, replacing the last report), and the flows
are validated against it again. The node panel turns a reported port's
config schema into a form, a reported screen is a canvas preset and a
touch screen's first report turns Touch friendly on. An offline device
node is degraded: amber on the canvas, a warning on Home, never a fault.
2026-10-16 17:37:49 +02:00
Melvin Strobl 1dc891aa1b A node's worker label is worker, so a device is only a paired one
- NodeDef `device`/`device_policy` become `worker`/`worker_policy`, the
  issue `ignored_device` becomes `ignored_worker`, and the SDK's `@node`
  takes `worker=` and `worker_policy=`
- FlowStore._migrate renames the keys in published and draft documents
  once, bumping their version; a queued run's `needs` reads the old key
- placement passes the label as `label`, client regenerated
- docs: worker placement wording, MCP issuer without DOMAIN, device rows in
  the CLI data directory and the API table, device links in compose comments
2026-10-16 16:56:05 +02:00
Melvin Strobl 29d3ec467f Device node: a device's ports are wired in flows
- flow/nodes/device.py: `device` node with params `device` and `config`;
  provides the device's `in` ports, requires its `out` ports without
  waking anything, never run (Node.passive), down "Offline" from presence
- DeviceDef.io and DevicePort go; GET /devices/{id} computes `io` and
  `bindings` from the device nodes of published flows, in a fixed order,
  cached by FlowStore.revision; the credential's scope comes from them
- issues `unknown_device` and `device_port` (wired twice; against a report
  the wrong direction or dtype, or unlisted as advisory), rechecked when a
  device is created or deleted; ValidationIssue.advisory is a field
- a device publishing on a bound port is credited to its node by the
  server; the panel endpoint kind is gone from `_endpoints` and the canvas
- a rebuild that moves a device's bindings sends `dashboard_changed` with
  an empty name, which the firmware refetches on
- startup moves each device's `io` rows into device nodes in the flows
  declaring their messages, draft and published, one commit; rows no
  flow declares are logged and left out
- frontend: Device node settings (device, Publishes, Follows, a port
  picker from a report), palette shortcuts per reported port, DeviceDetail
  lists bindings with "Use in flow" in place of the I/O editor
- docs: devices, flow editor, node types, issues, API
2026-10-16 16:40:31 +02:00
Melvin Strobl 28a565e501 Panels become devices: per-device documents in the flow repo, a Devices tab
- flow/devices.py: DeviceDef in FLOWS_DIR/_devices/<id>.json, committed under
  the flow store's write lock, 409 on a stale version; nonces in DEVICES_FILE
  outside git; in-memory presence announced as device_status
- api/routes/devices.py: CRUD, pairing with an optional {name, board,
  firmware}, a pending list adopted without a code from the LAN only, unpair;
  /panels/pair, /panels/pair/{code} and GET /panels/{id} kept as aliases,
  PUT /panels/ gone
- _device_may replaces _panel_may, JWT claims unchanged
- panels.json moves into devices once at startup, then .migrated
- MCP list_devices, get_device; search category device
- Devices tab (list, detail with presence, pairing, ordered dashboards,
  touch, I/O), PanelsDialog removed, serialised optimistic saves
2026-10-16 15:48:36 +02:00
stroblme dea25ec08d One image for the whole app: engine, API and dashboard on one port
The backend image now builds the SPA and serves it on the API's own origin
(fluksio/spa.py), so an instance is a single container with no proxy, DNS,
port 80 or .env: `docker compose up -d` with the new compose.yml runs it on
localhost:8080. It starts on fluksio serve (persisted secret key, first
admin printed once) and brings its own Redis when REDIS_HOST is unset.

Slimmer: python:3.13-slim, no dev group, uv once, fastapi without the cloud
CLI. Optional features are per-feature extras (mqtt, influx, email, sentry,
ml, server for all) picked with the EXTRAS build argument; numpy moved to
`ml`. ffmpeg left the image for imageio-ffmpeg via the Modules screen.

The operator stack runs the same image with app. and api. routed to it; the
nginx frontend service, the LAN nginx proxy and DOMAIN in the backend are
gone, and the MCP issuer is FRONTEND_HOST.
2026-10-16 11:40:02 +02:00
stroblme c37a191c82 Overview lists: a duplicate button beside the trash while selecting copies the picked flows or dashboards as drafts, one request each, then clears the selection. Publish, delete and duplicate share one batch helper with their own toasts. Docs 2026-10-16 09:21:42 +02:00
stroblme ce2ce4e3e7 Header removals wait for the code editor to be left: autosave saves with drop=false, so follow_header adds and renames from the header but keeps a port or setting whose parameter is gone, and leaving the editor (blur or closing the panel) saves once more with removals; source saves of a flow run one at a time 2026-10-15 21:31:04 +02:00
stroblme abfb03c441 Header of process and node ports follow each other both ways: a panel edit rewrites only the def line of edited code, a code save reads the header back into input ports and settings (draft and response); Space types in the node editor again 2026-10-15 18:19:19 +02:00
stroblme 1b5a16f636 Assistant streams its answer token by token
ai/client.py streams the completion and puts the chunks back together
(Assembler), covering OpenAI's piecewise tool-call arguments, Ollama's whole
calls, OpenRouter's comment lines, unnumbered calls and in-stream errors.
The turn emits a delta event per piece and a message per round as before,
and the chat grows the last answer as deltas arrive.
2026-10-15 17:37:56 +02:00
stroblme 8f8330c9e0 Assistant answers render as markdown
react-markdown, lazy-loaded with the first answer (37 kB gzip chunk, main
bundle unchanged). No raw HTML, images dropped, links open in a new tab.
The system prompt asks for lists, inline code and fenced code blocks.
2026-10-15 17:18:12 +02:00
stroblme 4b216dcbcf Modules resolve the manifest with uv pip compile before syncing, so listed packages bring their dependencies 2026-10-15 17:06:20 +02:00
stroblme d3b52bd7d6 Docs: speech-to-text and text-to-speech recipes for a panel's mic and speaker
faster-whisper and piper as plain Function nodes, the mic -> STT -> logic ->
TTS -> speaker wiring, the Modules manifest they need (uv pip compile, av<19)
and measured sizes, memory and CPU times. Modules docs now say uv pip sync
installs no dependencies.
2026-10-15 17:01:25 +02:00
stroblme 96618e40a2 Opt-in WITH_FFMPEG build arg installs Debian's ffmpeg in the backend image
Off by default, since it adds about 400 MB. Documented with the clip recipe
and in the configuration reference.
2026-10-15 15:55:02 +02:00
stroblme 3a9fc43707 Workers screen shows each agent's version and protocol, and who is behind
The hello carries the fluksio-worker version. An agent refused for an older
protocol stays listed as Needs upgrade until it attaches again. Docs gain an
Upgrading workers section.
2026-10-15 11:09:54 +02:00
stroblme f569f325a7 Serve MCP and hooks with a secret through the portal tunnel
The connector now serves /mcp, its OAuth documents and /hooks/ besides
/api/v1/, refuses any dot segment (httpx resolves them, which carried
/api/v1/../../mcp onto the MCP mount), addresses calls to the hub's host,
and bridges served sockets by calling the app with a websocket scope.
A hook without a secret answers the tunnel with the 404 (or 1008) of a
wrong secret. Requests through the hub get the hub's issuer, resource and
consent page in the OAuth metadata and the 401, and the MCP host check
admits the hub while enrolled.
2026-10-15 11:00:19 +02:00
stroblme 4c20f913e8 Pin the UTC offset on the time a flow version carries
Every datetime the API emits for versions (flow summaries' updated_at,
runs' created/started/finished with flow_version) is already ISO 8601
with Z or +00:00. A test holds it and the API docs say so.
2026-10-15 10:57:41 +02:00
stroblme b9078dd19f fluksio run takes inputs after -- as name=value
So an input named like a flag of run, timeout or sync, can still be set.
Flags before -- keep naming inputs as they did.
2026-10-15 10:56:46 +02:00
stroblme e597c34823 Refuse an origin.repo of /, the home directory or a relative path
Every run hashes the python files below origin.repo, so saving or syncing
a flow with one of those now answers 422 with what to do instead.
2026-10-15 10:55:40 +02:00
stroblme 6fa44bacf3 Flow delete offers to delete its runs, history included
DELETE /runs?flow= removes every Run row of the flow but left its flow_run
rows, so a deleted flow kept appearing in the history. history=true now
drops those too, and the flow delete dialogs (editor and bulk) send it when
"Also delete its runs" is ticked.
2026-10-15 10:52:33 +02:00
stroblme 7ea53ccf17 DELETE /artifacts/{digest}: superuser only, 409 while referenced
The keep-set the hourly sweep uses is now referenced_artifacts, shared by
the sweep and the route. The volatile ring is not touched.
2026-10-15 10:48:59 +02:00
stroblme 21cfed7d39 GET /runs/{id}/nodes/{node}/outputs serves what a node returned
The stage cache's stored outputs, as kept (up to 256 kB of JSON). A node
that kept none answers 404 saying why. Listed in the API docs.
2026-10-15 10:23:39 +02:00
stroblme 43498037be PUT /panels/ carries a version and refuses a stale one with 409
The Panels dialog re-reads the list on 409. A panels.json without a
version reads as 0.
2026-10-15 10:07:50 +02:00
stroblme f2a823218a GPUs follow the engine's own CUDA_VISIBLE_DEVICES
Its entries are the cards present, and a local child's allocation is
named through them, so under =2,3 card 0 is device 2. Remote workers
keep mapping against their own mask.
2026-10-15 09:53:38 +02:00
stroblme 290b9fa705 Superuser route to flush the metrics collector now
POST /observability/flush handles what the collector has queued and writes
it, so the soak harness no longer sleeps FLUSH_INTERVAL_S before clearing
its own rows.
2026-10-15 09:53:19 +02:00