Push a frame instead of storing and fetching it
The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.
`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.
The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.
Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
This commit is contained in:
@@ -838,3 +838,83 @@ def test_a_panel_fetches_only_the_media_its_tiles_are_showing(
|
||||
).status_code
|
||||
== 403
|
||||
)
|
||||
|
||||
|
||||
def test_a_socket_pushes_only_the_frames_it_was_asked_for(tmp_path) -> None:
|
||||
"""Bytes are the one thing this socket cannot send speculatively.
|
||||
|
||||
A frame a second per subscriber is affordable; every frame to every open
|
||||
editor tab is not. So nothing goes until a client names what it is drawing,
|
||||
and a panel credential can only name what it was already allowed to see.
|
||||
"""
|
||||
import orjson
|
||||
|
||||
from fluksio.api.routes.flows import media_frames, wanted_names
|
||||
from fluksio.flow.artifacts import ArtifactStore, VolatileStore
|
||||
|
||||
store = ArtifactStore(tmp_path / "artifacts")
|
||||
store.volatile = VolatileStore(tmp_path / "ring", limit_bytes=1_000_000)
|
||||
frame = store.put([b"\x89PNG..."], name="f.png", media_type="image/png",
|
||||
volatile=True)
|
||||
kept = store.put([b"checkpoint"], name="w.pt")
|
||||
|
||||
asking = orjson.dumps({"type": "media", "names": ["cam.frame", "other.frame"]})
|
||||
# A person's socket gets what it asked for; a panel's is intersected with
|
||||
# what it draws, so naming a message is not a way around the scope.
|
||||
assert wanted_names(asking.decode(), None, set()) == {"cam.frame", "other.frame"}
|
||||
assert wanted_names(asking.decode(), {"cam.frame"}, set()) == {"cam.frame"}
|
||||
# Anything else on this socket leaves the set alone.
|
||||
assert wanted_names('{"type":"ping"}', None, set()) is None
|
||||
assert wanted_names("not json", None, set()) is None
|
||||
|
||||
events = [
|
||||
{"type": "message_value", "name": "cam.frame", "value": frame, "ts": 1.0},
|
||||
{"type": "message_value", "name": "other.frame", "value": frame, "ts": 1.0},
|
||||
{"type": "message_value", "name": "cam.model", "value": kept, "ts": 1.0},
|
||||
{"type": "node_log", "name": "cam.frame", "value": frame},
|
||||
]
|
||||
frames = media_frames(events, {"cam.frame", "cam.model"}, store)
|
||||
|
||||
# One frame: the ring's, for the name that asked. The durable checkpoint is
|
||||
# what a fetch is for, and a log is not a value.
|
||||
assert len(frames) == 1
|
||||
length = int.from_bytes(frames[0][:4], "big")
|
||||
header = orjson.loads(frames[0][4 : 4 + length])
|
||||
assert header == {
|
||||
"type": "media",
|
||||
"name": "cam.frame",
|
||||
"digest": frame["digest"],
|
||||
"media_type": "image/png",
|
||||
"ts": 1.0,
|
||||
}
|
||||
assert frames[0][4 + length :] == b"\x89PNG..."
|
||||
|
||||
# Nothing asked for, nothing sent.
|
||||
assert media_frames(events, set(), store) == []
|
||||
|
||||
|
||||
def test_only_the_newest_frame_of_a_batch_is_pushed(tmp_path) -> None:
|
||||
"""A client that fell behind is not handed frames it would only draw over."""
|
||||
import orjson
|
||||
|
||||
from fluksio.api.routes.flows import media_frames
|
||||
from fluksio.flow.artifacts import ArtifactStore, VolatileStore
|
||||
|
||||
store = ArtifactStore(tmp_path / "artifacts")
|
||||
store.volatile = VolatileStore(tmp_path / "ring", limit_bytes=1_000_000)
|
||||
first = store.put([b"one"], media_type="image/png", volatile=True)
|
||||
second = store.put([b"two"], media_type="image/png", volatile=True)
|
||||
|
||||
frames = media_frames(
|
||||
[
|
||||
{"type": "message_value", "name": "cam.frame", "value": first, "ts": 1.0},
|
||||
{"type": "message_value", "name": "cam.frame", "value": second, "ts": 2.0},
|
||||
],
|
||||
{"cam.frame"},
|
||||
store,
|
||||
)
|
||||
|
||||
assert len(frames) == 1
|
||||
length = int.from_bytes(frames[0][:4], "big")
|
||||
assert orjson.loads(frames[0][4 : 4 + length])["digest"] == second["digest"]
|
||||
assert frames[0][4 + length :] == b"two"
|
||||
|
||||
Reference in New Issue
Block a user