Push a frame instead of storing and fetching it

The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.

`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.

The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.

Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
This commit is contained in:
2026-09-02 10:15:14 +02:00
co-authored by Claude Opus 5
parent 518231aa39
commit d471614e6a
29 changed files with 1101 additions and 147 deletions
+16 -8
View File
@@ -117,19 +117,27 @@ sequence numbers are for whoever consumes the media.
Bytes still never travel as a message. A camera publishes one reference per
frame and a microphone one per chunk, which makes a media stream an ordinary
[streaming port](../concepts/flows.md#streaming-ports), and each frame an
artifact. What that costs is worth knowing before pointing a camera at it:
[streaming port](../concepts/flows.md#streaming-ports). What that costs
depends on how the bytes reach the screen:
| Rate | Where it works |
|---|---|
| A clip a second (speech) | anywhere, including through the portal |
| A frame every second or two (a glance at a door) | locally; through the portal, every few seconds |
| Live video, 1530 fps | not here — see below |
| Ten frames a second (a camera worth watching) | on the local network, with `volatile=True` |
| A frame every second or two (a glance at a door) | anywhere, including through the portal |
| Full-rate video, 30 fps and up | not here — see below |
Real-time video is not a message-plane problem: every frame would be an
artifact, an event and a fetch. Point a media widget's **stream URL** at
whatever the camera already serves and the browser plays it from source; the
messages then carry the occasional still, and the flow reacts to those.
The difference is one flag. A frame saved with
[`volatile=True`](../code/nodes.md#media) is held in memory and pushed down the
websocket in front of the value naming it, so a screen draws it without asking
for anything. A stored one is fetched instead: a round trip per frame, which is
a glance rather than a view. Through a portal every frame is fetched, so a
remote panel is in the second row whatever the flag says.
Above that, video is not a message-plane problem — every frame would still be
an event. Point a media widget's **stream URL** at whatever the camera already
serves and the browser plays it from source; the messages then carry the
occasional still, and the flow reacts to those.
### `json`