Take the engine off the path node code imports from, and let it stop

The worker script is handed to the interpreter by path, so app/flow was
sys.path[0] for every node: `import queue` got the engine's. It now drops
its own directory before anything else imports, and runs with the
deployment's credentials scrubbed out of its environment.

Also: reload builds off the event loop, the pool wakes what is blocked on
it when it stops, a refused metrics flush is kept for the next one rather
than dropped, and the cascade events are paired through failures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
This commit is contained in:
2026-08-16 23:46:08 +02:00
co-authored by Claude Fable 5
parent b33be3fdd4
commit 83c30aa1c7
19 changed files with 346 additions and 65 deletions
+11 -5
View File
@@ -14,10 +14,8 @@ should reopen it.
### To be sorted
- INFRA: merge the philosophy statement at the beginning of vision.md into the rest of the document. Dissolve the decision dates and fold the decisions into a clean structure
- BUG/UI mobile friendly support is degraded: 1) toolbar in the "Flows" viewport extend mobile viewport width 2) position of nodes should never be static; always adjust such that there are as few as possible overlaps and direction is left to right (desktop) or top to bottom (mobile) 3) Dashboard view is not mobile friendly at all; as dashboard design is infeasible on mobile, render all widgets in a vertically stacked order. This allows to inspect each widget and make changes. Layout changes are not a feature on mobile 4) the home view is not responsive; all items shown there should re-order on mobile such that no scrollbars appear
- Move the "brain" into the "Home" at the top; flat without a box and the same background, such the brain renders just on the surface of the home screen
- Move the "Health" page into "Home" as well underneath the flow control toggles (replaces the running / failing flows row in the current home view)
- BUG/UI mobile friendly support is degraded: 1) toolbar in the "Flows" viewport extend mobile viewport width 2) position of nodes should never be static; always adjust such that there are as few as possible overlaps (of nodes and edge labels) and direction is left to right (desktop) or top to bottom (mobile) with a minimal (but clean) overall edge length 3) Dashboard view is not mobile friendly at all; as dashboard design is infeasible on mobile, render all widgets in a vertically stacked order. This allows to inspect each widget and make changes. Layout changes are not a feature on mobile 4) the home view is not responsive; all items shown there should re-order on mobile such that no scrollbars appear
- BUG/UI remove the up/down counters from integer fields (global)
### Connector write paths
@@ -44,6 +42,8 @@ is what M4 still waits on, together with porting the flows.
- CHORE/FLOW: `compile_check` sends the *draft* source under the running node's cache key, so the worker recompiles the published source on its next call. Correct, but one wasted compile per save on a busy node.
- FEAT/API: `POST /modules/apply` rebuilds the whole pipeline so a node that could not import its package stops being red. That resubscribes every MQTT node in the deployment; a targeted rebuild of the flows that actually failed to load would be gentler.
- CHORE/FLOW: a node's return value now round-trips through JSON, so tuples arrive downstream as lists and anything non-JSON is an explicit error. That is the message contract, but flows written before this may notice.
- BUG/FLOW: a node whose cold-start imports plus body exceed its timeout can never succeed. The timeout covers the first call's imports, a timeout kills the worker so the next attempt is cold again, and `compile()` only ever warms one of the N workers. Broadcasting `compile` to every worker is the candidate fix, at the cost of N module executions per reload.
- CHORE/FLOW: worker protocol loose ends — the request `id` is echoed but never checked, `json.dumps` runs twice per result (once to prove it is JSON, once to send it), `_remote_types` is an unbounded cache keyed on class names that user code chooses, and `PythonWorkerPool._lock` guards less than its name suggests.
### Engine history
@@ -53,8 +53,13 @@ is what M4 still waits on, together with porting the flows.
- CHORE/API: `/observability/summary` reports the work queue's `depth` as the Redis stream length, which is the journal size (capped at `STREAM_MAXLEN`) rather than a backlog. The health screen shows `pending` instead; the field name still invites the wrong reading.
- FEAT/UI: the health screen's window is fixed at 24 hours and the charts fold minute buckets in Python. A range picker (and `date_bin()` behind it) is the next step if anyone wants a week.
- CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but `forget_flow` could offer to clear it.
- BUG/API: `MetricsCollector.flush` takes its buckets and pending events out of the collector before writing them, so a failed write loses everything accumulated since the last flush rather than retrying it. Unobserved so far — the soak run never took Postgres away — but it is a silent hole whenever the database blips. Putting the batch back, merged with whatever arrived meanwhile, closes it.
- CHORE/API: nothing can ask the collector to flush now, so anything needing the tables to be current has to wait out `FLUSH_INTERVAL_S` — which is what the soak harness does before clearing its own rows.
- BUG/UI: `MemoryWorkQueue.stats()` hard-codes `pending: 0`, so the health tile always reads zero on a stack without Redis.
- CHORE/API: `MetricsCollector._start_run`'s `existing is not None` branch is unreachable: a redelivery only arrives after the record it would update has been dropped.
- CHORE/API: the collector's `_tracebacks` stash is never pruned for nodes that stop failing — one entry per node that ever errored, kept for the life of the process.
- CHORE/API: audit rows ride the same drop-oldest bus as telemetry, so a storm can lose one. Writing a node's source is not audited either; publishing is.
- PERF/API: two unmarked ceilings — `/observability/flows` scans every `metric_minute` row in the window on each 30 s poll, and `queue.stats()` does a keyspace `scan_iter` on every call while two endpoints poll it.
- CHORE/INFRA: dev only — memory-queue ids (`mem-{seq}`) restart at 0 each boot and `FlowRun.id` is the primary key, so a restart without Redis upserts over the previous boot's run rows.
### Dashboard follow-ups
@@ -79,6 +84,7 @@ is what M4 still waits on, together with porting the flows.
- PERF/UI: `useParamSuggestions` fetches every flow's detail to build the suggestion list. An aggregate endpoint if an installation ever has many flows.
- CHORE/UX: the derived-cron chip also appears on the delay node, where `interval` is a rate limit rather than a schedule. May want it inject-only.
- CHORE/UX: free-form params (python nodes) get no suggestions, since there is no schema to key them off.
- PERF/UI: `BrainView` runs 300 force-layout ticks synchronously inside a `useMemo`, so the graph is laid out on the render thread.
- FEAT/UI: the brain graph draws connections as plain lines, so which way a value travels is not shown. An arrowhead needs the path shortened to the target circle's rim, which needs the node radius on the edge.
- CHORE/UI: fixing the yellow message pulse gave `edge-pulse` an explicit `to`, so a *selected* edge now decays to `--muted-foreground` and snaps back to its blue at the end instead of decaying into it. Barely visible, and only while an edge is selected.