Push a frame instead of storing and fetching it
The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.
`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.
The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.
Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
This commit is contained in:
@@ -7,6 +7,7 @@ import { dashboardKeys, panelKeys } from "@/components/Dashboard/queries"
|
||||
import { healthKeys } from "@/components/Health/queries"
|
||||
import { runKeys } from "@/components/Runs/queries"
|
||||
import { connectionStore } from "@/lib/connectionStore"
|
||||
import { parseMediaFrame } from "@/lib/media"
|
||||
import { apiToken } from "@/lib/portal"
|
||||
import { type LogLine, liveStore, type ValueSource } from "./liveStore"
|
||||
import { flowKeys } from "./queries"
|
||||
@@ -160,9 +161,25 @@ function schedule() {
|
||||
retry = Math.min(retry * 2, RECONNECT_MAX)
|
||||
}
|
||||
|
||||
/**
|
||||
* Say which messages this page wants frames for.
|
||||
*
|
||||
* Bytes are the one thing the socket does not send unasked: a camera is
|
||||
* hundreds of kilobytes a second and most tabs are drawing no media at all. So
|
||||
* a Media tile or a thumbnail registers its message, and this tells the engine
|
||||
* — again on every reconnect, since the new socket knows nothing.
|
||||
*/
|
||||
function sendWanted() {
|
||||
if (socket?.readyState !== WebSocket.OPEN) return
|
||||
socket.send(JSON.stringify({ type: "media", names: liveStore.wantedNames() }))
|
||||
}
|
||||
|
||||
liveStore.onWanted(sendWanted)
|
||||
|
||||
function connect() {
|
||||
if (watchers === 0 || socket || timer) return
|
||||
const ws = new WebSocket(socketUrl())
|
||||
ws.binaryType = "arraybuffer"
|
||||
socket = ws
|
||||
|
||||
ws.onopen = () => {
|
||||
@@ -185,6 +202,7 @@ function connect() {
|
||||
]) {
|
||||
client?.invalidateQueries({ queryKey })
|
||||
}
|
||||
sendWanted()
|
||||
}
|
||||
|
||||
const handle = (message: FlowEvent) => {
|
||||
@@ -318,6 +336,13 @@ function connect() {
|
||||
// frame rather than the connection: an exception thrown here escapes into
|
||||
// `window.onerror` and leaves whatever it had already applied behind.
|
||||
try {
|
||||
if (event.data instanceof ArrayBuffer) {
|
||||
// Media, sent in front of the value that names it — so the tile has
|
||||
// the bytes by the time it hears the message changed.
|
||||
const frame = parseMediaFrame(event.data)
|
||||
if (frame) liveStore.setBytes(frame.header.digest, frame.bytes)
|
||||
return
|
||||
}
|
||||
const payload = JSON.parse(event.data)
|
||||
if (payload?.type === "batch") {
|
||||
// A cascade publishes a dozen events at once and the engine coalesces
|
||||
|
||||
Reference in New Issue
Block a user