Live view over WebRTC (openpanel L1, server and browser)
Docs / docs (push) Successful in 22s
Playwright Tests / test-playwright (1, 2) (push) Failing after 4m37s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m19s
pre-commit / pre-commit (push) Failing after 2m25s
Test Backend / test-backend (3.12) (push) Successful in 4m57s
Test Backend / test-backend (3.14) (push) Successful in 4m41s
Compose Smoke Test / test-compose (push) Successful in 28s
Playwright Tests / merge-reports (push) Successful in 1m38s

A device reporting a `live` port streams its camera to the browser's own
RTCPeerConnection. The flows socket carries the signalling as text, so it
crosses the portal: `live` messages on a person's socket, `signal` on a
device's (start, sdp, candidate, bye, busy). The server routes by session,
holds a device to two viewers, ends a session when either socket closes or
no answer comes within 10 s, and gives each socket an outbox of its own
that ends a session rather than drop a message. TURN credentials are
minted per session in coturn's REST scheme (STUN_URLS, TURN_URLS,
TURN_SECRET), the viewer's with the device's offer; GET /devices/{id}/ice
hands them out too. The tunnel keeps a stream's early frames instead of
dropping them past eight and forgets them when the stream or link goes.

A media tile on a device's camera, and the Devices tab's preview, play
live when tapped, pause frames meanwhile, and fall back to frames with a
short note on busy, failure or no picture within 10 s. coturn is an
optional compose service behind the `turn` profile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0138N7k5PT4dcR1iEqXaaQ9k
This commit is contained in:
Melvin Strobl
2026-10-17 08:37:39 +02:00
co-authored by Claude Opus 5.5
parent 6a6af9c291
commit 02b736c9e8
20 changed files with 1535 additions and 24 deletions
@@ -44,6 +44,7 @@ import {
type DeviceReport,
deviceNodeOf,
freeNodeId,
LIVE_PORT,
type ReportedPort,
reportedPorts,
reportOf,
@@ -233,7 +234,7 @@ function Camera({ device }: { device: DevicePublic }) {
<section className={CARD}>
<PanelSection
title="Camera"
help="Live while this preview is on screen. The camera sends frames only while someone watches them here or on a dashboard, or a running flow reads them."
help="Frames while this preview is on screen, which the camera sends only while someone watches them here or on a dashboard, or a running flow reads them. Tap it for live video when the device streams live."
>
<div
className="flex aspect-video w-full overflow-hidden rounded-md bg-muted text-center text-sm"
@@ -748,7 +749,11 @@ function Capabilities({
</span>
) : null}
</span>
{canEdit && said && wired.length === 0 ? (
{said?.port === LIVE_PORT ? (
<span className="shrink-0 text-xs text-muted-foreground">
Live view
</span>
) : canEdit && said && wired.length === 0 ? (
<UseInFlow device={device} port={said} size="sm" />
) : null}
</li>
+8 -1
View File
@@ -3,6 +3,12 @@ import type { DevicePublic, DType, MessageSpec, NodeDef_Input } from "@/client"
/** The node type a device is in a flow. */
export const DEVICE_TYPE = "device"
/**
* The port a device reports when its camera can stream live over WebRTC.
* Signalled on the socket, never wired in a flow.
*/
export const LIVE_PORT = "live"
/** One port as a device reports it. */
export type ReportedPort = {
port: string
@@ -57,8 +63,9 @@ export function screensOf(devices: DevicePublic[] = []) {
})
}
/** Whether a reported port may be wired in `direction`. */
/** Whether a reported port may be wired in `direction`. `live` never is. */
export const wiresAs = (port: ReportedPort, direction: "in" | "out") =>
port.port !== LIVE_PORT &&
(port.modes?.length ? port.modes : [port.direction]).includes(direction)
/** What the server accepts as a port name. */