Media dtypes: image, audio and video as narrowed artifact references
Docs / docs (push) Successful in 30s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m7s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m54s
pre-commit / pre-commit (push) Failing after 4m24s
Test Backend / test-backend (push) Successful in 3m8s
Compose Smoke Test / test-compose (push) Successful in 40s
Playwright Tests / merge-reports (push) Successful in 1m33s

A port may now declare `image`, `audio` or `video`. Each is the artifact
reference the engine already had, narrowed by the `media_type` on it, so a
speech recogniser declares what it eats rather than taking any bytes at all and
finding out. Bytes still never travel as a message and nothing on the wire
stops being JSON: a camera publishes one reference per frame, a microphone one
per chunk, and a reference may carry a `meta` dict nothing here interprets.

Streaming media is therefore an ordinary streaming port — with one change to
what that means. An emission used to journal an item with no payload, so
downstream read whatever was current when the item was claimed; a consumer
slower than its producer saw only the newest chunk and the ones between were
lost. That is right for a training curve and wrong for a second of speech, so
an emission now journals a `kind="emission"` item carrying its values, and the
executor hands them to the nodes reading that message instead of writing them
to state again. The value in state stays the latest, which is what everything
else reads, and the wave is filtered by what actually changed rather than
walking everything reachable. No queue serialization change — the existing
`outputs` field carries it.

Continuous media makes the store's missing GC a real problem, so this closes
it: `sweep_artifacts` runs hourly, keeps every digest a `run_artifact` row
records or a live message holds, spares anything written in the last hour, and
stands aside entirely while a run is in flight, since a node may store a
checkpoint long before it returns the reference to it. That also collects the
orphans a deleted flow has always left behind. `ARTIFACT_GC_INTERVAL_S=0` turns
it off.

Around the edges: `GET /artifacts/{digest}` serves the media type the caller
passes and answers ranged requests, so a browser plays a clip rather than
downloading it; `PUT` spools to disk instead of holding the whole body in
memory, as does `save_artifact` given a path; a Media widget draws whatever its
message points at, and a wall panel may fetch the bytes its own tiles are
showing and nothing else; and a connector gets `save_artifact`, for a device
whose readings are bytes.

What this cannot do is live video: a frame every second or two is a glance, and
the honest answer above that is the camera's own stream, which the widget takes
as a URL and the browser plays from source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-26 23:44:55 +02:00
co-authored by Claude Opus 5
parent 8be7e424ba
commit 0ffcabfdb9
37 changed files with 1271 additions and 62 deletions
+56 -14
View File
@@ -928,7 +928,11 @@ class Pipeline:
return True, outputs
def _execute_node(
self, node: Node, state: StateBackend, entry_id: str = ""
self,
node: Node,
state: StateBackend,
entry_id: str = "",
overrides: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
"""Run one node and record its outputs. Never raises."""
started = time.perf_counter()
@@ -938,6 +942,17 @@ class Pipeline:
# single bulk read does not, and it was the engine's one global
# mutex — every node of every cascade queued behind it.
inputs = state.get_present(list(node.requires))
if overrides:
# The value this wave is delivering wins over whatever state
# holds by now. Not written back: the newest value is still the
# one everything else reads.
inputs.update(
{
name: value
for name, value in overrides.items()
if name in node.requires
}
)
key = ""
if self.run_cache is not None and node.fingerprint:
@@ -1074,13 +1089,16 @@ class Pipeline:
return
self._record_outputs(node, passed, self._state)
if self._queue is not None:
# Journalled with no payload of its own: the value is already in
# state, published in the order it was produced. An item carrying
# it would re-apply that value whenever it happened to be claimed,
# which is how an emission from the middle of a node overwrites the
# one it returned at the end. Downstream reads what is current,
# which is what "the latest value wins" has always meant here.
self._enqueue_cascade(node, None)
# Journalled carrying the emitted values, as an ``emission`` item:
# the executor hands them to the nodes reading them instead of
# writing them to state a second time. That distinction is the
# whole of it — re-applying would let a mid-node emission overwrite
# the value the node returned at the end, while reading state
# instead means a consumer slower than its producer sees only the
# newest chunk and the ones between are lost. A frame of video or a
# second of speech is worth delivering; the value in state stays
# the latest, which is what everything else reads.
self._enqueue_cascade(node, passed, kind="emission")
def _observe(self, outcome: NodeOutcome) -> None:
"""Tell the run watching this pipeline, if there is one."""
@@ -1166,6 +1184,7 @@ class Pipeline:
entry_id: str = "",
replay: bool = False,
changed: set[str] | None = None,
overrides: dict[str, Any] | None = None,
) -> StateBackend:
"""Execute nodes concurrently, scheduling each as its inputs arrive.
@@ -1173,6 +1192,10 @@ class Pipeline:
whose triggering inputs are all untouched is completed without being
run, so what is downstream of *it* is judged on the same footing. None
runs everything the subset holds, which is what a manual run means.
``overrides`` reaches the nodes that read those names directly, and no
further: a value carried by this wave is what its readers should see,
while everything past them reads what those readers produced.
"""
# One view of the graph for the whole wave: a flow replaced halfway
# through must not have this wave asking the new dependencies about a
@@ -1248,7 +1271,7 @@ class Pipeline:
continue
submitted.add(n)
node_futures[n] = executor.submit(
self._execute_node, n, state, entry_id
self._execute_node, n, state, entry_id, overrides
)
def drain(executor: ThreadPoolExecutor) -> None:
@@ -1348,6 +1371,7 @@ class Pipeline:
entry_id: str = "",
replay: bool = False,
changed: set[str] | None = None,
overrides: dict[str, Any] | None = None,
) -> StateBackend:
"""Run everything downstream of a node that has just published.
@@ -1361,6 +1385,10 @@ class Pipeline:
already read, which is how one node came to publish 619 messages a
minute off inputs that changed six times. ``None`` means walk
everything reachable, which is what a manual run wants.
``overrides`` carries values to hand to whoever reads them instead of
what state holds — an emission delivering the chunk that caused this
wave rather than whichever one is newest by the time it runs.
"""
if changed is not None and not changed:
# Everything the cascade carried was held back by a rate limit, so
@@ -1376,6 +1404,7 @@ class Pipeline:
entry_id=entry_id,
replay=replay,
changed=changed,
overrides=overrides,
)
def trigger(
@@ -1439,13 +1468,21 @@ class Pipeline:
return False
def _run_here(
self, node: Node, outputs: dict[str, Any] | None, cause: str = "manual"
self,
node: Node,
outputs: dict[str, Any] | None,
cause: str = "manual",
delivered: bool = False,
) -> StateBackend:
"""Run a cascade in this thread, under a run id of its own.
The queued path gets its run id from the journal entry. A run that never
went through the queue still belongs in the history, so it makes one —
marked as such, because it is no one's idempotency key.
``delivered`` marks an emission the queue could not take: its values are
already in state, so they are handed to their readers rather than
written again.
"""
run_id = f"{MANUAL_RUN_PREFIX}{uuid.uuid4().hex[:12]}"
self._publish(
@@ -1460,13 +1497,16 @@ class Pipeline:
}
)
try:
published = self.apply_outputs(node, outputs)
published = (
set(outputs or {}) if delivered else self.apply_outputs(node, outputs)
)
state = self.run_downstream(
node,
entry_id=run_id,
# No payload means the value is already in state and this is a
# wake-up, which has nothing to name as changed.
changed=published if outputs else None,
overrides=outputs if delivered else None,
)
finally:
# Paired, or a cascade that raised leaves the run open until the
@@ -1564,12 +1604,14 @@ class Pipeline:
logger.error("Could not defer work for '%s': %s", node.id, exc)
return False
def _enqueue_cascade(self, node: Node, outputs: dict[str, Any] | None) -> None:
def _enqueue_cascade(
self, node: Node, outputs: dict[str, Any] | None, kind: str = "cascade"
) -> None:
"""Journal a trigger, or fall back to running it here if that fails."""
from fluksio.flow.queue import WorkItem
item = WorkItem(
kind="cascade",
kind=kind,
node=node.id,
flow=node.flow,
outputs=outputs or {},
@@ -1591,7 +1633,7 @@ class Pipeline:
}
)
# Losing the value outright would be worse than running it here.
self._run_here(node, outputs, cause="external")
self._run_here(node, outputs, cause="external", delivered=kind == "emission")
def values(self, flow: str | None = None) -> dict[str, dict[str, Any]]:
"""Last value and timestamp of every message, optionally one flow's.