Live view over WebRTC (openpanel L1, server and browser)

A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.

A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.
This commit is contained in:
Melvin Strobl
2026-10-17 08:37:39 +02:00
parent efd883d092
commit 6a98dcd23f
20 changed files with 1535 additions and 24 deletions
+22 -2
View File
@@ -208,7 +208,8 @@ at, and stops when it scrolls away, the tab is hidden or the tile closes. A
clip keeps playing either way. One page is pushed frames for at most 8
messages at once. A ninth tile fetches its frames one by one and does not turn
a camera on. A [device's camera](#a-devices-camera) sends only while a tile like
this wants it.
this wants it. A tile on a device's camera that streams live shows **Live**:
tap it for [live video](#live-view).
For continuous video, set **Live stream** to a browser-compatible camera URL.
The browser must be able to reach that URL; enrollment does not expose the
@@ -600,7 +601,7 @@ keeps the switch as it was set.
### A device's camera
A camera device (openpanel on a NANO) reports four ports:
A camera device (openpanel on a NANO) reports these ports:
| Port | Type | Settings | What it sends |
|---|---|---|---|
@@ -608,6 +609,7 @@ A camera device (openpanel on a NANO) reports four ports:
| `person` | `bool` | `hold_s` (10) | whether someone is in view, held for `hold_s` after they leave |
| `detections` | `json` | | `{count, boxes: [{score, x0, y0, x1, y1}]}`, boxes scaled 0 to 1, on change and at most once a second |
| `snapshot` | `image` | | one stored frame each time `person` turns true |
| `live` | `video`, on demand | | H.264 over WebRTC while a [live view](#live-view) runs; never wired in a flow |
`camera` sends only while it has demand: while a page wants its frames (a
dashboard tile, the preview on the device's page, a screen through the
@@ -633,6 +635,24 @@ A device may fetch the last 8 artifacts each of its messages pointed at, not
only the current one, so a screen fetching a camera frame by frame through the
portal is not refused a frame the camera has just replaced.
#### Live view
A media tile showing a device's `camera`, and the preview on the device's
page, offer **Live** while the device is online and reports `live`. Tap the
tile and the browser connects to the device with WebRTC, without a plugin,
and plays its video in the tile under a **Live** badge. Tap again to stop. It
stops by itself when the tile scrolls away, the tab is hidden or the page
closes. While it plays the tile asks for no frames, so the camera encodes one
picture rather than two, and frames come back when it ends.
A device serves 2 live viewers. A third tile, a failed connection or no
picture within 10 seconds stays on frames and says so briefly. On one network
the two ends find each other directly. Between networks they need STUN and,
behind strict NATs, a TURN relay: set `STUN_URLS`, `TURN_URLS` and
`TURN_SECRET` (see [Configuration](../reference/configuration.md#live-view)).
The server only passes on the few messages that set a connection up, the
video never crosses it. A wall panel's own screen does not offer live view.
### Before device nodes
Before device nodes a device carried its ports itself. On its first start an