Refuse what a node cannot publish, and stop timing out work that is fine

Four things the python SDK turned up, each fixed where every client sees it.

A key no port declares is now an error rather than a silent drop, on the
return, the yield and the emit alike — the contract the docs already stated.
The SDK reads literal yields at sync time, so a typo fails before anything
runs, and an emission of one fails the call rather than being logged where
nobody looks.

NaN and infinity are refused at the port. JSON cannot spell either, so one
that travelled came back as a 500, a socket frame that stopped the canvas, or
a metric batch the database dropped whole.

An artifact input takes `@run:<id>.<output>` or a bare digest, resolved on the
engine — so the CLI, the run dialog and a python caller mean the same thing,
and a sweep can pass one at all.

Node timeouts are off by default. The clock measured silence, which a training
node is full of, and remote workers had already stopped enforcing it — their
heartbeat reset it. Now a heartbeat proves the agent rather than the node,
ninety seconds of nothing fails the call either way, and the engine touches
work it is still running so a long node is not redelivered at sixty seconds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69
This commit is contained in:
2026-08-25 07:30:14 +02:00
co-authored by Claude Opus 5
parent c33fa404a4
commit 93374a310e
32 changed files with 968 additions and 67 deletions
+71
View File
@@ -1,7 +1,9 @@
"""The work queue, and what the execution service does with it."""
import threading
import time
from fluksio.flow import executor
from fluksio.flow.executor import ExecutionService
from fluksio.flow.messages import DType, MessageSpec
from fluksio.flow.nodes import Node
@@ -264,3 +266,72 @@ def test_a_replayed_item_does_not_repeat_a_side_effect():
service._run_item(item)
assert calls == [5.0]
# -----------------------------------------------------------------------------
# Telling the queue that a long node is working, not lost
#
# What marks an item abandoned is nobody touching it. A node with no timeout
# may run for hours, so the engine holding it says so on a timer — and an
# engine that died says nothing, which is the distinction the reaper needs.
# -----------------------------------------------------------------------------
class RecordingQueue(MemoryWorkQueue):
"""A memory queue that writes down what it was asked to hold on to."""
def __init__(self) -> None:
super().__init__()
self.touched: list[list[str]] = []
def touch(self, entry_ids: list[str]) -> None:
self.touched.append(list(entry_ids))
def test_work_in_flight_is_touched_until_it_finishes(monkeypatch):
monkeypatch.setattr(executor, "TOUCH_INTERVAL_S", 0.0)
monkeypatch.setattr(executor, "DELAYED_INTERVAL_S", 0.05)
running = threading.Event()
release = threading.Event()
def slow(reading, params):
running.set()
release.wait(5)
return {"doubled": reading * 2}
source = Node(
f=lambda params: None,
provides=[MessageSpec(name="reading", dtype=DType.FLOAT)],
name="source",
)
consumer = Node(
f=slow,
requires=[MessageSpec(name="reading", dtype=DType.FLOAT)],
provides=[MessageSpec(name="doubled", dtype=DType.FLOAT)],
name="consumer",
)
source.assign_flow("f", "source")
consumer.assign_flow("f", "consumer")
queue = RecordingQueue()
pipeline = Pipeline(
nodes=[source, consumer], state=MemoryState(), work_queue=queue
)
service = ExecutionService(queue)
service.bind(pipeline)
service.start()
try:
source.inject({"reading": 3.0})
assert running.wait(5)
# Give the timer a couple of passes while the node is still in there.
time.sleep(0.2)
held = [ids for ids in queue.touched if ids]
assert held, "a running item was never touched"
release.set()
time.sleep(0.3)
# Once it is done it is acknowledged, so there is nothing to hold.
assert queue.touched[-1] == []
finally:
release.set()
service.stop()