Live view over WebRTC (openpanel L1, server and browser)

A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.

A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.
This commit is contained in:
Melvin Strobl
2026-10-17 08:37:39 +02:00
parent efd883d092
commit 6a98dcd23f
20 changed files with 1535 additions and 24 deletions
+33 -3
View File
@@ -37,6 +37,7 @@ from fluksio.api.routes.messages import port_source
from fluksio.api.socket_auth import TICKET_TTL_S, issue_ticket, socket_token
from fluksio.core.config import settings
from fluksio.core.db import engine
from fluksio.flow import live as live_view
from fluksio.flow.artifacts import is_reference
from fluksio.flow.controller import NODE_TYPES, FlowController, follow_header
from fluksio.flow.dashboards import DashboardStore
@@ -1369,6 +1370,9 @@ async def flow_events(websocket: WebSocket, ticket: str = "", token: str = "") -
[...]}``) and is pushed those frames. A device also sends its camera's
frames here, as binary frames in the layout it is pushed them in, and is
told when its on-demand ports have demand.
A live view's signalling rides here too (``fluksio.flow.live``): ``live``
messages on a person's socket, ``signal`` messages on a device's.
"""
token = socket_token(websocket.headers, ticket, token)
@@ -1425,6 +1429,9 @@ async def flow_events(websocket: WebSocket, ticket: str = "", token: str = "") -
async with event_bus.subscribe() as queue:
receiver = asyncio.create_task(_receive(websocket))
# Where a live view this socket is one end of reaches it. Its own
# outbox rather than the bus's queue, which drops when full.
endpoint = live_view.Endpoint()
if device:
presence.opened(device)
# Demand that moved between the snapshot and the subscription
@@ -1432,12 +1439,23 @@ async def flow_events(websocket: WebSocket, ticket: str = "", token: str = "") -
ports = demand.ports(device)
if ports != told:
await websocket.send_text(demand_frame(ports))
live_view.sessions.attach(device, endpoint)
outbox = asyncio.create_task(endpoint.next())
try:
while True:
sender = asyncio.create_task(queue.get())
done, _ = await asyncio.wait(
{sender, receiver}, return_when=asyncio.FIRST_COMPLETED
{sender, receiver, outbox}, return_when=asyncio.FIRST_COMPLETED
)
if outbox in done:
# Signalling, for this socket alone.
if sender not in done:
sender.cancel()
text = outbox.result()
outbox = asyncio.create_task(endpoint.next())
await websocket.send_text(text)
if receiver not in done and sender not in done:
continue
if receiver in done:
if sender not in done:
# Nobody will ever read this one: the next turn makes
@@ -1479,11 +1497,21 @@ async def flow_events(websocket: WebSocket, ticket: str = "", token: str = "") -
if asked is not None:
wanted = asked
demand.want(viewer, wanted)
elif (signal := live_view.parse(frame)) is not None:
live_view.sessions.heard(
signal,
endpoint,
person=str(user.id) if only is None else "",
device=device,
devices=getattr(
websocket.app.state, "device_store", None
),
)
receiver = asyncio.create_task(_receive(websocket))
if sender not in done:
continue
# `sender` is done from here: `FIRST_COMPLETED` returns with at
# least one of the two, so `receiver` missing means this one is.
# `sender` is done from here: the turns where it is not have
# gone round again above.
# Everything the bus has right now, not just the one event
# that woke this: a cascade puts a dozen in at once, and one
@@ -1544,7 +1572,9 @@ async def flow_events(websocket: WebSocket, ticket: str = "", token: str = "") -
pass
finally:
receiver.cancel()
outbox.cancel()
demand.drop(viewer)
live_view.sessions.detach(endpoint)
if device:
presence.closed(device)