- `stt`: one realtime transcription session per utterance on the speech
server's url, opened at `listening` true or the first chunk, committed at
false; a whole clip goes to /v1/audio/transcriptions. Returns at once and
publishes `partial` and `text` when the server answers.
- `tts`: one /v1/audio/speech call per sentence, cached by voice and text in
node state; a reference is marked `append` while earlier ones still play.
- wren: `reply` says what was applied, `unhandled` carries the prompt when
nothing was. The install hint names the git pin (PyPI's `wren` is another
package).
- ai/client: transcribe, speak and transcribe_stream beside stream_completion.
- A device socket publishes scalar `in` ports as {"type": "publish"} text,
in order with its binary frames.
- Pipeline.publish walks only what its values reach: a node downstream of a
consumer that published nothing no longer runs again (wren re-acted on
every microphone chunk).
C4: a device's `viewers` (superusers by default, or members) covers the
image and video messages its ports publish. Anyone else gets none of their
values on the socket, in the catalogue or a flow's state, 403 for their
history, for the frames they hold (the last 8 included) and for TURN
credentials, no bytes when asking for them, and `bye: forbidden` for a live
view. A device credential stays bounded by its dashboards. The device page
sets it in its Camera section, and a tile the viewer may not see says
"No access to this camera" and asks the camera for nothing.
C3:
- A volatile upload is held to MAX_FRAME_BYTES (4 MiB, 413 above it, read
without extra copies), and so is a frame on a device's socket.
- A frame the engine holds in memory is written to the ring directory
before a local worker is handed it, so load_artifact finds it.
- A socket the bus dropped events for catches up with a snapshot, demand
included, and a device's credential is checked again per frame, cached by
the flow store revision and its nonce, so a lost event never keeps an
unpaired device's frames coming.
- A report is at most 64 KB once its named config_schemas are put in place.
- A device away for 60 s has its true bool messages set to false.
- Refused frames are logged by fixed keys, a frame is named after its
message, `fluksio serve` waits 5 s for open connections, a tile with a
stream URL creates no demand, the 8-name cap keeps the newest names.
- Every container's log rotates (json-file, 3 x 10 MB).
A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.
A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.
- Volatile frames are held in memory with a byte-bounded queue instead of
the /dev/shm ring scanned on every put; workers keep writing to the
directory, which is trimmed on a timer. A volatile upload no longer spools
to disk. ARTIFACT_VOLATILE_BYTES=0 turns the ring off.
- A device sends camera frames as binary messages on its socket, held and
published in one step and credited to its device node; refused frames are
logged and dropped. Its snapshot carries demand and the uplink (binary, or
http through the portal).
- One demand registry for every socket's wanted media, the tunnel's
included, plus what running flows read, mapped onto on-demand ports and
published as device_demand; at most 8 media names per socket.
- Per-message deflate off in serve and dev; a device may fetch the last 8
digests of a message; published dashboards cached per store revision; a
revoked device socket is closed with 1008.
- Media tiles want bytes only while on screen; the Devices tab previews a
wired camera; the device node warns when a flow reads an on-demand port.
Every call through the portal arrives off the tunnel, so the per-address limit
skips those (the hub limits them by the real address), and the cap of 50 counts
and evicts portal and local requests apart, so a portal flood never pushes out
a device waiting on this network.
- POST /devices/pair answers a 2-character `check`, listed by /devices/pending
and shown beside each waiting entry; five codes a minute per address, and at
50 waiting the oldest gives way instead of a 429
- portal-minted device tokens are asked for with the device's nonce (`pnc`) and
held against it, so unpair and delete revoke them; one without `pnc` stands
only while the nonce is 0
- a device credential acts as the device (`device_principal`), not as the
approving superuser, so deleting that account unpairs nothing
- the report is validated port by port: unknown enum values read as `unknown`,
unreadable ports and sections are left out and logged, `config` may name an
entry of `config_schemas`; 422 only for no schema 1 or no object
- devices.json is parsed once per change (cached by stat), delete drops the
report and keeps the nonce, a busy recheck runs in the background
- scope and io only from enabled flows (start/stop tells the devices), a port
bound twice gives one io row
- the Devices tab's saves cancel and hold back device refetches until settled
- A device credential that cannot be checked now (device store not running,
nonces unreadable) raises DeviceUnavailable: 503 over HTTP, and the
/flows/ws handshake and the tunnel stream accept and close with 1013.
1008 (a 403 before accepting) stays for a revoked or invalid one
- Device files are written through a temporary file, so a read mid-save no
longer looks like a deleted device
- The first report turns Touch friendly on only for a device paired since
pairings are noted (DeviceState.paired_at), so devices paired before keep
the switch they were given
- NodeDef reads `device`/`device_policy` from older clients as the worker
label, for a release
- Tests: transient vs revoked socket, the touch default, the old worker keys,
and a restart with a device node and a node_health rule sending no push
A device reports its board, screen, camera and ports from its own
credential (16 KB, schema 1, replacing the last report), and the flows
are validated against it again. The node panel turns a reported port's
config schema into a form, a reported screen is a canvas preset and a
touch screen's first report turns Touch friendly on. An offline device
node is degraded: amber on the canvas, a warning on Home, never a fault.
- NodeDef `device`/`device_policy` become `worker`/`worker_policy`, the
issue `ignored_device` becomes `ignored_worker`, and the SDK's `@node`
takes `worker=` and `worker_policy=`
- FlowStore._migrate renames the keys in published and draft documents
once, bumping their version; a queued run's `needs` reads the old key
- placement passes the label as `label`, client regenerated
- docs: worker placement wording, MCP issuer without DOMAIN, device rows in
the CLI data directory and the API table, device links in compose comments
- flow/nodes/device.py: `device` node with params `device` and `config`;
provides the device's `in` ports, requires its `out` ports without
waking anything, never run (Node.passive), down "Offline" from presence
- DeviceDef.io and DevicePort go; GET /devices/{id} computes `io` and
`bindings` from the device nodes of published flows, in a fixed order,
cached by FlowStore.revision; the credential's scope comes from them
- issues `unknown_device` and `device_port` (wired twice; against a report
the wrong direction or dtype, or unlisted as advisory), rechecked when a
device is created or deleted; ValidationIssue.advisory is a field
- a device publishing on a bound port is credited to its node by the
server; the panel endpoint kind is gone from `_endpoints` and the canvas
- a rebuild that moves a device's bindings sends `dashboard_changed` with
an empty name, which the firmware refetches on
- startup moves each device's `io` rows into device nodes in the flows
declaring their messages, draft and published, one commit; rows no
flow declares are logged and left out
- frontend: Device node settings (device, Publishes, Follows, a port
picker from a report), palette shortcuts per reported port, DeviceDetail
lists bindings with "Use in flow" in place of the I/O editor
- docs: devices, flow editor, node types, issues, API
- flow/devices.py: DeviceDef in FLOWS_DIR/_devices/<id>.json, committed under
the flow store's write lock, 409 on a stale version; nonces in DEVICES_FILE
outside git; in-memory presence announced as device_status
- api/routes/devices.py: CRUD, pairing with an optional {name, board,
firmware}, a pending list adopted without a code from the LAN only, unpair;
/panels/pair, /panels/pair/{code} and GET /panels/{id} kept as aliases,
PUT /panels/ gone
- _device_may replaces _panel_may, JWT claims unchanged
- panels.json moves into devices once at startup, then .migrated
- MCP list_devices, get_device; search category device
- Devices tab (list, detail with presence, pairing, ordered dashboards,
touch, I/O), PanelsDialog removed, serialised optimistic saves
Copy widget in the widget panel or mod+c keeps one widget in localStorage;
a paste button in the dock and mod+v insert it below the other widgets with
a fresh id. Placement is shared with adding a widget.
faster-whisper and piper as plain Function nodes, the mic -> STT -> logic ->
TTS -> speaker wiring, the Modules manifest they need (uv pip compile, av<19)
and measured sizes, memory and CPU times. Modules docs now say uv pip sync
installs no dependencies.
Changing dashboard kept the grid mounted, so every new tile rose at once.
The grid is drawn afresh per dashboard and its tiles come in row by row:
from below, top row first, for one further down the rail, and from above,
bottom row first, for one further up, as a round panel's dashboards slide.
A phone stacks a dashboard instead of drawing it on a canvas, and the canvas
was what carried the ground, so a stack had only the palette's colour. The
panel and the editor now draw the look's ground behind the stack, held still
while it scrolls. Glass's drift stops under reduced motion as it does on a
canvas.
A widget that fell off the panel, after the canvas was made smaller, was
drawn under the clip and could only be re-packed with Reflow. Editing, the
canvas now extends to the lowest widget and scrolls, with the panel's own
edge dashed and the area past it hatched, so the widget can be dragged
back. The scale stays the panel's, and view mode is unchanged.
The canvas states the look of the dashboard shown, so a neighbour beside it
and the one a turn leaves wore that look too. Each page now restates its own
look, theme and palette. The ring's paint moves from rules under
[data-look] to variables each look sets on its root, which stop at the
nearer root where a descendant rule reached into both.
A website tile now runs in an opaque origin (allow-scripts, popups, forms).
Its new "Trusted site" setting adds allow-same-origin for sites that need
their own cookies and storage, and a trusted tile pointing at the origin the
app runs on (its own, or the hub's) shows a notice instead of the frame.
The one-hue ramp told lines apart by lightness alone. --chart-1 is now
--primary and slots 2-8 are distinct hues (orange, violet, green, orchid,
ochre, teal, magenta) that clear 3:1 on --card and stay apart under
protanopia and deuteranopia; red and sage are left out as the status
colours. A chart takes the slots in order instead of spreading a ramp,
and a dashboard's own palette continues after its primary and listed
colours with slots 2-8 rather than tints of its scheme. Line charts stay
at five series. Tokens mirror index's, as design-check requires.
Below md a dashboard stacks to one column and dropped its rail. The rail is
now a horizontal row in the dashboard's own look, floated over the bottom
of the screen on /panel/{id} and above the dock in the app's view, with
44px items, sideways scrolling when it overflows, and padding so the last
tile clears it; a panel's read-only notice moves up out of its way. The
three looks share the geometry from ui/core.
The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.
`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.
The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.
Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.
stroblme
Melvin Strobl