Fluksio/app
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Push a frame instead of storing and fetching it

Browse Source 
The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.

`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.

The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.

Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
This commit is contained in:
stroblme
2026-09-02 10:15:14 +02:00
co-authored by Claude Opus 5
parent 518231aa39
commit d471614e6a
29 changed files with 1101 additions and 147 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/code/nodes.md
+12 -2
View File
@@ -137,7 +137,8 @@ def process(speech): # an `audio` port
def process(camera_url):
for index, jpeg in enumerate(grab(camera_url)): # a generator
frame = fluksio.save_artifact(
jpeg, f"frame-{index:05d}.jpg", media_type="image/jpeg"
jpeg, f"frame-{index:05d}.jpg",
media_type="image/jpeg", volatile=True,
)
frame["meta"] = {"seq": index}
yield {"frame": frame} # an `image` stream port
@@ -148,12 +149,21 @@ has to match, so a node declaring `audio` never receives a video by accident.
See [Payload types](../reference/payload-types.md#image-audio-video) for what
each carries and what rates are realistic.
`volatile=True` is for a frame rather than a result. The bytes go to a ring in
memory instead of the data volume, and the engine pushes them down the
websocket to whichever screens are drawing that message — so a camera runs at
ten frames a second without writing anything to disk. They last as long as it
takes newer frames to need the room. Leave it off for a clip somebody asked to
keep.
!!! warning "Emitted media is not kept; returned media is"
Only what a node *returns* is recorded against its run. Frames yielded
along the way are replaced in state by the next one, and the artifact sweep
removes bytes nothing refers to any more, which is what stops a camera
filling the disk. If a particular frame matters, return it.
filling the disk. If a particular frame matters, return it: a volatile one
is copied out of the ring when the run records it, so returning it is also
what makes it outlive the next few seconds.
## Printing
docs/code/workers.md
+8
View File
@@ -141,6 +141,14 @@ filesystem writes to it directly; one that does not fetches and uploads over
HTTP, using the artifact endpoint beside the socket it already has. Either way
your node code is the same two calls.
A fetch is cached on the worker by digest, since content addressing means an
entry is never stale. Nothing expires on its own, so the cache is bounded by
size and the oldest fall out: `FLUKSIO_ARTIFACT_CACHE` says where it lives
(default a directory in the temporary directory) and
`FLUKSIO_ARTIFACT_CACHE_BYTES` how much it holds (default 1 GiB). Worth raising
where a worker reads the same large inputs repeatedly, and worth leaving alone
where it reads a media stream — those are chunks nothing asks for twice.
## Seeing what is attached
```sh
docs/interface/dashboards.md
+12 -9
View File
@@ -105,21 +105,24 @@ so an answer to a different question is ignored.
A media widget draws what its message points at: a picture, a clip with
controls, a video. Media does not travel as a message; a reference to it does.
The tile fetches the bytes behind whichever reference the message holds,
and redraws when a new one arrives.
The tile takes the bytes behind whichever reference the message holds, and
redraws when a new one arrives.
**Crop or fit** decides how a picture fills the tile. **Play as it arrives**
starts a clip by itself, though a browser only plays sound once somebody has
touched the page, so a screen nobody has tapped stays silent.
Rate is the thing to get right. A frame every second or two is a glance at a
door, and works; through the portal, make that every few seconds. Live video is
not something to push through the message plane at all. Put the camera's own
address in **Live stream** and the browser plays it from source, leaving the
messages to carry the occasional still that a flow can actually react to.
Rate is the thing to get right, and it is decided by the node publishing rather
than by the tile. A frame the engine is holding in memory is pushed down the
same websocket that carries the value, so ten a second is a real view on the
local network; a stored one is fetched, a round trip each, which suits a glance
at a door every second or two. Through the portal everything is fetched, so
make it every few seconds there. Above that, put the camera's own address in
**Live stream** and the browser plays it from source, leaving the messages to
carry the occasional still that a flow can actually react to.
Panels see media the same way, and only their own: a screen may fetch the bytes
its own tiles are showing and nothing else.
Panels see media the same way, and only their own: a screen is sent, and may
fetch, the bytes its own tiles are showing and nothing else.
## Player tiles
docs/reference/configuration.md
+12
View File
@@ -131,6 +131,9 @@ warning into a refusal to start.
| `OBS_RETENTION_DAYS` | `30` | how long metrics, events and run records are kept |
| `ARTIFACT_GC_INTERVAL_S` | `3600` | how often artifact bytes nothing refers to are swept away; 0 never sweeps |
| `ARTIFACT_GC_GRACE_S` | `3600` | how long a freshly written artifact is spared, whatever refers to it |
| `MAX_ARTIFACT_BYTES` | `2147483648` | the largest body `PUT /artifacts` will take; 0 removes the limit |
| `ARTIFACT_VOLATILE_DIR` | worked out | where frames a flow only shows live are held; empty picks a directory under `/dev/shm` named for the data directory, and falls back to the temporary directory |
| `ARTIFACT_VOLATILE_BYTES` | `50331648` | how much that ring holds before the oldest frames fall out; 0 turns it off and volatile saves land in the store |
The three concurrency limits are also flags on `fluksio serve`
(`--max-workers`, `--max-cascades`, `--max-runs`), as is the card count,
@@ -146,6 +149,15 @@ sweep is what keeps a flow streaming media from filling the disk. It stands
aside entirely while a run is in flight, since a node may store a checkpoint
long before it returns the reference to it.
Frames saved with `volatile=True` skip all of that. They go to a ring in memory
instead of the volume, the oldest falling out once the newest need the room,
and the engine pushes them down the websocket to whichever screens are drawing
them — which is what a camera at ten frames a second needs and the store cannot
give it. A frame a run *records* is copied into the store on the way, so
returned media is kept and emitted media is not. Under Docker the ring lives in
the container's `/dev/shm`, whose default is 64 MB: raise `shm_size` alongside
`ARTIFACT_VOLATILE_BYTES`.
A node that declares nothing is not accounted against `FLOW_CPUS`; it runs on
the shared pool and is given `FLOW_CPUS / FLOW_MAX_WORKERS` as a thread cap, so
several at once cannot each size themselves to the whole machine. Setting
docs/reference/connector-contract.md
+13 -3
View File
@@ -177,7 +177,9 @@ connector publishes a reference to it instead:
async def poll(self) -> dict[str, Any] | None:
jpeg = await asyncio.to_thread(self._grab)
return {
"frame": self.save_artifact(jpeg, "frame.jpg", media_type="image/jpeg")
"frame": self.save_artifact(
jpeg, "frame.jpg", media_type="image/jpeg", volatile=True
)
}
```
@@ -186,10 +188,18 @@ async def poll(self) -> dict[str, Any] | None:
works once the node has started, since the store is the engine's and is handed
over then.
`volatile=True` is what a camera publishes with. The frame goes to a ring in
memory rather than the data volume and is pushed down the websocket to
whichever screens are drawing it, so a wall panel sees ten frames a second and
the SD card under it is never written to. Frames last until newer ones need the
room; leave the flag off for a reading somebody asked to keep.
Each reading is a new artifact, which the poll loop publishes because its
digest differs from the last. Set `poll_interval` to what somebody actually
wants to look at: a frame every second or two is a glance, and live video
belongs on the camera's own stream rather than in the graph.
wants to look at. Through a portal the bytes are fetched rather than pushed, so
a remote panel wants a frame every second or two; higher rates than that are
for the local network, and full-rate video still belongs on the camera's own
stream.
## Lifecycle
docs/reference/payload-types.md
+16 -8
View File
@@ -117,19 +117,27 @@ sequence numbers are for whoever consumes the media.
Bytes still never travel as a message. A camera publishes one reference per
frame and a microphone one per chunk, which makes a media stream an ordinary
[streaming port](../concepts/flows.md#streaming-ports), and each frame an
artifact. What that costs is worth knowing before pointing a camera at it:
[streaming port](../concepts/flows.md#streaming-ports). What that costs
depends on how the bytes reach the screen:
| Rate | Where it works |
|---|---|
| A clip a second (speech) | anywhere, including through the portal |
| A frame every second or two (a glance at a door) | locally; through the portal, every few seconds |
| Live video, 15–30 fps | not here — see below |
| Ten frames a second (a camera worth watching) | on the local network, with `volatile=True` |
| A frame every second or two (a glance at a door) | anywhere, including through the portal |
| Full-rate video, 30 fps and up | not here — see below |
Real-time video is not a message-plane problem: every frame would be an
artifact, an event and a fetch. Point a media widget's **stream URL** at
whatever the camera already serves and the browser plays it from source; the
messages then carry the occasional still, and the flow reacts to those.
The difference is one flag. A frame saved with
[`volatile=True`](../code/nodes.md#media) is held in memory and pushed down the
websocket in front of the value naming it, so a screen draws it without asking
for anything. A stored one is fetched instead: a round trip per frame, which is
a glance rather than a view. Through a portal every frame is fetched, so a
remote panel is in the second row whatever the flag says.
Above that, video is not a message-plane problem — every frame would still be
an event. Point a media widget's **stream URL** at whatever the camera already
serves and the browser plays it from source; the messages then carry the
occasional still, and the flow reacts to those.
### `json`