Coalesce the event bus, and fix the socket that ended on a client frame

A three-node cascade publishes 13-16 events and each one crossed to the
event loop on its own. They are one `call_soon_threadsafe` now — whatever
was published between two turns of the loop goes over together — and every
subscriber still receives every event, oldest still dropped first when one
falls behind.

The socket end of the same path:

- **any frame from the client ended its stream.** `receive_text` was
  awaited once, outside the loop, so a keepalive — or anything else a
  client decided to say — satisfied it and was read as the client going
  away. It is recreated per iteration; only a disconnect ends the stream.
- events go out in one frame per wave (`{"type": "batch", "events": [...]}`,
  capped at 64), serialised once with orjson rather than per client with
  the stdlib's `json.dumps` through `send_json`. The client unpacks a batch
  and still understands single frames, so an older engine behind a newer
  bundle keeps working.
- authenticating and building the snapshot happen on a thread. Both were on
  the event loop: one is a database round trip, the other reads the whole
  of state, per connect and again per `dashboard_changed` per panel.

`Pipeline.values()` — what that snapshot is — no longer SCANs the whole
Redis namespace. It scanned five bookkeeping keys for every message to find
the messages; `RedisState` keeps a set of the names beside them and answers
from it. Maintained wherever a message is written, so a seeded value or a
deleted flow keeps it exact.

On the client, while in the same file:

- a `node_health` event invalidates the flow's detail. The canvas draws
  health from the server-derived `issues`, so a node going down or
  recovering only showed on mount, navigation or a rebuild. The store had
  a health map of its own that nothing ever read; it and `useNodeHealth`
  are gone rather than wired up, since the server's view is the one the
  canvas already uses.
- a reconnect invalidates the five key families this socket feeds instead
  of the entire cache, and the backoff is jittered. The usual reason a
  socket dropped is the engine restarting, so every tab and every wall
  panel refetched everything, together, at the moment it was least able to
  answer.
- a frame that will not parse costs the frame, not the connection. It was
  the one unguarded `JSON.parse` in the app; an exception there escaped to
  `window.onerror` and left whatever it had already applied behind.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M6hPWS6YEbT1P8LxhhFb2T
This commit is contained in:
2026-08-29 20:08:50 +02:00
co-authored by Claude Opus 5
parent da528340a9
commit 1069247085
10 changed files with 287 additions and 78 deletions
+83
View File
@@ -0,0 +1,83 @@
"""What the bus costs the event loop, and that it still delivers everything."""
import asyncio
import threading
from fluksio.flow.events import QUEUE_SIZE, EventBus
def _event(i: int) -> dict[str, object]:
return {"type": "node_executed", "node": f"n{i}", "outputs": 1}
def test_a_cascade_costs_one_loop_callback() -> None:
"""The whole point of coalescing.
A three-node cascade publishes 13-16 events and each used to cross to the
event loop on its own. They are one callback now — and still every event.
"""
async def run() -> tuple[int, int]:
bus = EventBus()
loop = asyncio.get_running_loop()
bus.bind(loop)
calls = 0
real = loop.call_soon_threadsafe
def counting(*args: object, **kwargs: object) -> object:
nonlocal calls
calls += 1
return real(*args, **kwargs) # type: ignore[arg-type]
loop.call_soon_threadsafe = counting # type: ignore[method-assign]
try:
async with bus.subscribe() as queue:
# From a worker thread, which is where nodes run.
thread = threading.Thread(
target=lambda: [bus.publish(_event(i)) for i in range(16)]
)
thread.start()
thread.join()
await asyncio.sleep(0.05)
return calls, queue.qsize()
finally:
loop.call_soon_threadsafe = real # type: ignore[method-assign]
calls, delivered = asyncio.run(run())
assert calls == 1
assert delivered == 16
def test_a_slow_subscriber_still_loses_its_oldest() -> None:
"""Coalescing must not turn the drop-oldest bound into unbounded growth."""
async def run() -> tuple[int, object]:
bus = EventBus()
bus.bind(asyncio.get_running_loop())
async with bus.subscribe() as queue:
thread = threading.Thread(
target=lambda: [bus.publish(_event(i)) for i in range(QUEUE_SIZE + 50)]
)
thread.start()
thread.join()
await asyncio.sleep(0.05)
return queue.qsize(), (await queue.get())["node"]
size, oldest = asyncio.run(run())
assert size == QUEUE_SIZE
# The first fifty were dropped, not the last.
assert oldest == "n50"
def test_nothing_is_buffered_with_nobody_listening() -> None:
"""An engine with no browser open holds no events."""
async def run() -> int:
bus = EventBus()
bus.bind(asyncio.get_running_loop())
for i in range(10):
bus.publish(_event(i))
await asyncio.sleep(0.01)
return len(bus._pending)
assert asyncio.run(run()) == 0