Push a frame instead of storing and fetching it
The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.
`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.
The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.
Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
This commit is contained in:
@@ -105,21 +105,24 @@ so an answer to a different question is ignored.
|
||||
|
||||
A media widget draws what its message points at: a picture, a clip with
|
||||
controls, a video. Media does not travel as a message; a reference to it does.
|
||||
The tile fetches the bytes behind whichever reference the message holds,
|
||||
and redraws when a new one arrives.
|
||||
The tile takes the bytes behind whichever reference the message holds, and
|
||||
redraws when a new one arrives.
|
||||
|
||||
**Crop or fit** decides how a picture fills the tile. **Play as it arrives**
|
||||
starts a clip by itself, though a browser only plays sound once somebody has
|
||||
touched the page, so a screen nobody has tapped stays silent.
|
||||
|
||||
Rate is the thing to get right. A frame every second or two is a glance at a
|
||||
door, and works; through the portal, make that every few seconds. Live video is
|
||||
not something to push through the message plane at all. Put the camera's own
|
||||
address in **Live stream** and the browser plays it from source, leaving the
|
||||
messages to carry the occasional still that a flow can actually react to.
|
||||
Rate is the thing to get right, and it is decided by the node publishing rather
|
||||
than by the tile. A frame the engine is holding in memory is pushed down the
|
||||
same websocket that carries the value, so ten a second is a real view on the
|
||||
local network; a stored one is fetched, a round trip each, which suits a glance
|
||||
at a door every second or two. Through the portal everything is fetched, so
|
||||
make it every few seconds there. Above that, put the camera's own address in
|
||||
**Live stream** and the browser plays it from source, leaving the messages to
|
||||
carry the occasional still that a flow can actually react to.
|
||||
|
||||
Panels see media the same way, and only their own: a screen may fetch the bytes
|
||||
its own tiles are showing and nothing else.
|
||||
Panels see media the same way, and only their own: a screen is sent, and may
|
||||
fetch, the bytes its own tiles are showing and nothing else.
|
||||
|
||||
## Player tiles
|
||||
|
||||
|
||||
Reference in New Issue
Block a user