stroblmeandClaude Opus 5
d471614e6a
The rate the media dtypes could carry was one frame every second or two: each
was a file on the data volume, an event on the socket, and a request back for
the bytes. This closes both halves of that, and they are one feature.
`save_artifact(..., volatile=True)` writes to a `VolatileStore` — the same
content-addressed store, in `/dev/shm`, bounded by size with the oldest falling
out (`ARTIFACT_VOLATILE_BYTES`, 48 MB under the container's raised `shm_size`).
Nothing sweeps it: a frame nobody kept is not worth walking the store to find.
`ArtifactStore.path` falls through to it, which is what lets a volatile frame be
an ordinary reference everywhere else — the dtype check, a panel's digest scope,
`load_artifact` in a node, and the widget's own fetch all work on one unchanged.
`adopt` copies one into the store when a run records it, so "returned media is
kept, emitted media is not" stays true.
The bytes then go down the flows websocket as a length-prefixed binary frame,
sent just ahead of the `message_value` naming them, so a tile has the frame when
it hears the value moved. Nothing is pushed unasked: a client names the messages
it is drawing (`{"type":"media","names":[…]}`), a panel's list is intersected
with the scope it already had, and only the newest frame per name in a batch is
sent — a client that fell behind is not handed frames it would draw over. The
tunnel relays text only, so a screen reached through a portal falls back to
fetching, which is why the rate table now has two rows.
Around the edges: the remote worker's fetch cache is bounded at last
(`FLUKSIO_ARTIFACT_CACHE_BYTES`), since content addressing means nothing in it
ever expires and a media stream fills it with chunks nothing asks for twice; a
port carrying an image draws the frame in the node panel rather than only
saying `image/png · frame.png · 1.79kB`; and an edge chip says that much instead
of a line of hash. The media screenshot stops waiting for `networkidle` — a
camera is a socket that never goes quiet, which is the point of it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YC4u66vjzW54fnHu5Juhh9
6.8 KiB
Payload types
Every port declares a dtype, and every value that passes through it is
checked against that declaration.
This is not decoration. It is what lets the dashboard editor offer you only the messages a gauge can actually draw, what lets the canvas refuse a binding before anything runs, and what lets a downstream node know the shape of what it is getting before the flow starts.
Everything on the wire is JSON. That is what lets the same value pass through the state backend, the work queue and the worker protocol unchanged.
The scalars
dtype |
Accepts |
|---|---|
float |
any number — int or float, but not bool |
int |
a whole number, not bool |
str |
a string |
bool |
exactly true or false |
bool is an int subclass in Python but is not a number here, so a switch
cannot bind to a temperature.
NaN and infinity are refused, wherever they sit, including inside a json,
record, series or list. JSON cannot spell either, so one that travelled
would come back as a response nobody can parse and a row the database rejects,
a long way from the node that made it. An empty subset or a division with no
denominator is what usually produces one; publish None instead.
The structured ones
These are declared shapes rather than "some JSON", which is what makes a widget binding checkable.
record
Flat named scalars.
{"title": "Boiler", "body": "Pressure low", "severity": "warning"}
A record cannot contain a record, so its shape is readable from the declaration alone.
Read by the Notification widget. It is also what an alert channel of kind dashboard writes.
list
Ordered items of one declared shape. The port also declares item:
item |
Meaning |
|---|---|
| unset | record — what the agenda and forecast widgets read |
float, int, str, bool |
a list of scalars |
json |
anything |
A list of lists, or a list of series, is refused. One declared level is the point.
series
Labelled lines of (timestamp, value) pairs, which is what a chart draws.
{
"lines": [
{"label": "living", "points": [[1717000000, 21.4], [1717000060, 21.5]]}
],
"range": "-24h"
}
Keys beside lines are carried through untouched, which is how a querying
chart puts the window and resolution it asked for on the request and reads them
back off the answer. That is what stops an answer to a different question
from overwriting the picture.
GET /runs/series/compare answers in this shape, which is why comparing three
training curves is a widget binding rather than a screen of its own.
artifact
A reference to stored bytes.
{"digest": "sha256:…", "size": 4194304, "media_type": "application/octet-stream", "name": "weights.pt"}
Binary payloads (tensors, checkpoints, images) never travel as a message. The bytes go to a content-addressed store and the message carries this. A thirty-megabyte checkpoint never sits in the state backend, and the reference stays valid wherever the store is reachable from, including on another machine.
Node code produces one with fluksio.save_artifact and opens one with
fluksio.load_artifact. See Writing node code.
image, audio, video
The same reference, narrowed to a kind of media by its media_type.
{"digest": "sha256:…", "size": 61344, "media_type": "image/jpeg", "name": "frame.jpg",
"meta": {"width": 1280, "height": 720, "seq": 41}}
An audio port takes audio/* and refuses anything else, so a speech
recogniser declares what it eats rather than taking any bytes at all and
finding out. An artifact port still accepts all three: media narrows
artifact, not the other way round.
meta is optional and nothing here reads it: sample rates, dimensions and
sequence numbers are for whoever consumes the media.
Bytes still never travel as a message. A camera publishes one reference per frame and a microphone one per chunk, which makes a media stream an ordinary streaming port. What that costs depends on how the bytes reach the screen:
| Rate | Where it works |
|---|---|
| A clip a second (speech) | anywhere, including through the portal |
| Ten frames a second (a camera worth watching) | on the local network, with volatile=True |
| A frame every second or two (a glance at a door) | anywhere, including through the portal |
| Full-rate video, 30 fps and up | not here — see below |
The difference is one flag. A frame saved with
volatile=True is held in memory and pushed down the
websocket in front of the value naming it, so a screen draws it without asking
for anything. A stored one is fetched instead: a round trip per frame, which is
a glance rather than a view. Through a portal every frame is fetched, so a
remote panel is in the second row whatever the flag says.
Above that, video is not a message-plane problem — every frame would still be an event. Point a media widget's stream URL at whatever the camera already serves and the browser plays it from source; the messages then carry the occasional still, and the flow reacts to those.
json
Anything JSON-serializable. The escape hatch, and the right answer when a payload genuinely has no fixed shape.
Reach for it last. A json port tells the canvas, the widget picker and the
next author nothing.
Naming a run's output
Any run parameter is also accepted as text, since nobody wants to paste an
object into a shell. @run:<id>.<output> names what an earlier run produced,
whatever its type, an artifact reference or a json config alike. A bare
sha256:… digest names content in the artifact store. Both resolve before the
run starts, so the CLI, the run dialog and a python caller all mean the same
thing by the same string.
Both spellings are reserved on every input, str included: an input that has to
carry one of them literally is asking for a value this engine reads as a name.
What a widget will bind to
| Widget | Accepts |
|---|---|
| Gauge, Chart, Slider, Bar | float, int |
| Switch | bool |
| Agenda, Forecast | list |
| Notification | record |
| Value | anything |
| Icon | weather strings, booleans and numbers alike |
| Media | image, audio, video, artifact |
| Clock, Text | nothing — they bind to no message |
Enforced on the server as well as in the editor.
Type failures
A value that does not match its port's declaration raises on the node that published it, naming the port and what arrived. It does not get published, and it does not reach anything downstream. A wrong value stopping at its source is easier to diagnose than one propagating.