Live view over WebRTC (openpanel L1, server and browser)

A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.

A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.
This commit is contained in:
Melvin Strobl
2026-10-17 08:37:39 +02:00
parent efd883d092
commit 6a98dcd23f
20 changed files with 1535 additions and 24 deletions
+32
View File
@@ -201,6 +201,7 @@ on its own, so this is for reclaiming space sooner.
| `POST` | `/devices/{id}/pair` | adopt by `{code}`, or by `{request}` from the pending list for a request from your network |
| `POST` | `/devices/{id}/unpair` | revoke the device credential, one the portal issued included |
| `PUT` | `/devices/{id}/report` | what the device has, from its own credential only; at most 16 KB, replaces the last report, read leniently (see [What a device reports](../interface/dashboards.md#what-a-device-reports)) |
| `GET` | `/devices/{id}/ice` | STUN and TURN servers for a live view of the device, as `{ice_servers: [{urls, username?, credential?}], ttl}`, with TURN credentials of the caller's own; not for a device credential |
`/panels/pair`, `/panels/pair/{code}` and `GET /panels/{id}` answer as their
`/devices/` counterparts, for firmware that has not updated yet.
@@ -275,6 +276,37 @@ logged and dropped and the socket stays open. With `http` it uploads with
`POST /messages/<message>`. A device that is unpaired or deleted while
connected is closed with 1008.
### Live view
A device that reports a `live` port streams its camera over WebRTC. The
video goes peer to peer, or through a TURN relay, and the socket carries only
the signalling, as text, so it crosses the portal too. A person's socket
sends and hears `{"type": "live", "device": "<id>", "session": "<id>",
"action": ...}`, a device's socket `{"type": "signal", "session": "<id>",
"action": ...}`. A session id is 8 to 64 letters, digits, `-` or `_`, chosen by
the viewer. In order:
| Step | Viewer's socket | Device's socket |
|---|---|---|
| 1 | sends `start` | hears `start` with `ice_servers` minted for the device |
| 2 | hears `sdp` (the offer) with `ice_servers` minted for the viewer | sends `sdp` with its offer |
| 3 | sends `sdp` with its answer | hears `sdp` |
| 4 | sends and hears `candidate`, the bare `candidate:...` line | hears and sends `candidate` |
| 5 | sends `bye`, or hears `bye` or `busy` with a `reason` | hears `bye` with a `reason`, or sends `bye` or `busy` |
Any signed-in person may watch any device. A device speaks only for the
sessions it was handed on that socket. The server answers `busy` itself once a
device has 2 sessions, and `bye` with `offline` or `unsupported` when the
device is not connected or reports no `live` port. A session the viewer has not
answered within 10 seconds ends with `timeout`, and either socket closing ends
its sessions with `closed` or `offline`. A socket that falls 64 signalling
messages behind ends the session with `overflow` instead of losing one.
`ice_servers` follow `RTCIceServer`: `{urls: [...]}` for STUN,
`{urls, username, credential}` for TURN, in coturn's REST scheme with a
username of `<expiry>:<who>`, good for an hour. A device that sends its camera's
frames on the same socket pauses them while a session negotiates, so its
offer and candidates are not queued behind frames.
## Errors and client generation
| Code | Meaning |