Cameras on devices (openpanel C1): frames in memory, a binary uplink, demand

- Volatile frames are held in memory with a byte-bounded queue instead of
  the /dev/shm ring scanned on every put; workers keep writing to the
  directory, which is trimmed on a timer. A volatile upload no longer spools
  to disk. ARTIFACT_VOLATILE_BYTES=0 turns the ring off.
- A device sends camera frames as binary messages on its socket, held and
  published in one step and credited to its device node; refused frames are
  logged and dropped. Its snapshot carries demand and the uplink (binary, or
  http through the portal).
- One demand registry for every socket's wanted media, the tunnel's
  included, plus what running flows read, mapped onto on-demand ports and
  published as device_demand; at most 8 media names per socket.
- Per-message deflate off in serve and dev; a device may fetch the last 8
  digests of a message; published dashboards cached per store revision; a
  revoked device socket is closed with 1008.
- Media tiles want bytes only while on screen; the Devices tab previews a
  wired camera; the device node warns when a flow reads an on-demand port.
This commit is contained in:
Melvin Strobl
2026-10-16 20:43:14 +02:00
parent d79203a5f7
commit ecc4bee4bb
29 changed files with 1425 additions and 220 deletions
+35
View File
@@ -203,6 +203,13 @@ For local streams, `volatile=True` artifacts can be pushed over the websocket
with their message. Stored artifacts are fetched individually, as are frames
viewed through a portal. Use occasional snapshots for remote screens.
A tile asks for frames only while it is on screen in a tab that is being looked
at, and stops when it scrolls away, the tab is hidden or the tile closes. A
clip keeps playing either way. One page is pushed frames for at most 8
messages at once. A ninth tile fetches its frames one by one and does not turn
a camera on. A [device's camera](#a-devices-camera) sends only while a tile like
this wants it.
For continuous video, set **Live stream** to a browser-compatible camera URL.
The browser must be able to reach that URL; enrollment does not expose the
camera itself. See [Media payloads](../reference/payload-types.md#image-audio-video).
@@ -591,6 +598,34 @@ the dashboard settings, named after the device, and the first report of a
touch screen turns **Touch friendly** on. A device paired before this version
keeps the switch as it was set.
### A device's camera
A camera device (openpanel on a NANO) reports four ports:
| Port | Type | Settings | What it sends |
|---|---|---|---|
| `camera` | `image`, on demand | `width` (640), `fps` (5), `idle_fps` (0, off), `quality` (65) | JPEG frames, held in memory and never stored |
| `person` | `bool` | `hold_s` (10) | whether someone is in view, held for `hold_s` after they leave |
| `detections` | `json` | | `{count, boxes: [{score, x0, y0, x1, y1}]}`, boxes scaled 0 to 1, on change and at most once a second |
| `snapshot` | `image` | | one stored frame each time `person` turns true |
`camera` sends only while it has demand: while a page wants its frames (a
dashboard tile, the preview on the device's page, a screen through the
portal), or a node of a running flow reads its message. Once its flow wires
`camera`, the device's page shows a **Camera** preview. A flow that reads
`camera` keeps it sending all the time, about 300 GB a month through a cloud
instance, so the Device node warns when a node in its flow reads a port that
sends on demand. Have flows read `snapshot` or `person` instead.
The device is told which of its ports have demand on its live socket, and sends
its frames there too, as described under [Live events](../code/api.md#live-events).
Through the portal the socket carries text only, so a device paired there
uploads each frame over HTTP instead.
A device may fetch the last 8 artifacts each of its messages pointed at, not
only the current one, so a screen fetching a camera frame by frame through the
portal is not refused a frame the camera has just replaced.
### Before device nodes
Before device nodes a device carried its ports itself. On its first start an