Push a frame instead of storing and fetching it

The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.

`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.

The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.

Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
This commit is contained in:
2026-09-02 10:15:14 +02:00
co-authored by Claude Opus 5
parent 518231aa39
commit d471614e6a
29 changed files with 1101 additions and 147 deletions
+80
View File
@@ -838,3 +838,83 @@ def test_a_panel_fetches_only_the_media_its_tiles_are_showing(
).status_code
== 403
)
def test_a_socket_pushes_only_the_frames_it_was_asked_for(tmp_path) -> None:
"""Bytes are the one thing this socket cannot send speculatively.
A frame a second per subscriber is affordable; every frame to every open
editor tab is not. So nothing goes until a client names what it is drawing,
and a panel credential can only name what it was already allowed to see.
"""
import orjson
from fluksio.api.routes.flows import media_frames, wanted_names
from fluksio.flow.artifacts import ArtifactStore, VolatileStore
store = ArtifactStore(tmp_path / "artifacts")
store.volatile = VolatileStore(tmp_path / "ring", limit_bytes=1_000_000)
frame = store.put([b"\x89PNG..."], name="f.png", media_type="image/png",
volatile=True)
kept = store.put([b"checkpoint"], name="w.pt")
asking = orjson.dumps({"type": "media", "names": ["cam.frame", "other.frame"]})
# A person's socket gets what it asked for; a panel's is intersected with
# what it draws, so naming a message is not a way around the scope.
assert wanted_names(asking.decode(), None, set()) == {"cam.frame", "other.frame"}
assert wanted_names(asking.decode(), {"cam.frame"}, set()) == {"cam.frame"}
# Anything else on this socket leaves the set alone.
assert wanted_names('{"type":"ping"}', None, set()) is None
assert wanted_names("not json", None, set()) is None
events = [
{"type": "message_value", "name": "cam.frame", "value": frame, "ts": 1.0},
{"type": "message_value", "name": "other.frame", "value": frame, "ts": 1.0},
{"type": "message_value", "name": "cam.model", "value": kept, "ts": 1.0},
{"type": "node_log", "name": "cam.frame", "value": frame},
]
frames = media_frames(events, {"cam.frame", "cam.model"}, store)
# One frame: the ring's, for the name that asked. The durable checkpoint is
# what a fetch is for, and a log is not a value.
assert len(frames) == 1
length = int.from_bytes(frames[0][:4], "big")
header = orjson.loads(frames[0][4 : 4 + length])
assert header == {
"type": "media",
"name": "cam.frame",
"digest": frame["digest"],
"media_type": "image/png",
"ts": 1.0,
}
assert frames[0][4 + length :] == b"\x89PNG..."
# Nothing asked for, nothing sent.
assert media_frames(events, set(), store) == []
def test_only_the_newest_frame_of_a_batch_is_pushed(tmp_path) -> None:
"""A client that fell behind is not handed frames it would only draw over."""
import orjson
from fluksio.api.routes.flows import media_frames
from fluksio.flow.artifacts import ArtifactStore, VolatileStore
store = ArtifactStore(tmp_path / "artifacts")
store.volatile = VolatileStore(tmp_path / "ring", limit_bytes=1_000_000)
first = store.put([b"one"], media_type="image/png", volatile=True)
second = store.put([b"two"], media_type="image/png", volatile=True)
frames = media_frames(
[
{"type": "message_value", "name": "cam.frame", "value": first, "ts": 1.0},
{"type": "message_value", "name": "cam.frame", "value": second, "ts": 2.0},
],
{"cam.frame"},
store,
)
assert len(frames) == 1
length = int.from_bytes(frames[0][:4], "big")
assert orjson.loads(frames[0][4 : 4 + length])["digest"] == second["digest"]
assert frames[0][4 + length :] == b"two"