Run python nodes out of process, with modules of their own

User code no longer execs in the engine. A pool of persistent worker
subprocesses speaks one JSON object per line; the controller installs a
proxy as the node's function, so every execution path funnels through it
and the pipeline is untouched. A crash costs one subprocess, a per-node
timeout is a kill, and cancelling from the canvas is that same kill.

The workers run a venv of the user's own on the data volume, filled from
a pip manifest versioned beside the flows. Applying it retires the
workers and rebuilds, so a package lands without restarting the engine.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
This commit is contained in:
2026-08-16 21:43:36 +02:00
co-authored by Claude Fable 5
parent 979c9d3c1f
commit f300c43f3a
27 changed files with 1536 additions and 46 deletions
@@ -26,6 +26,7 @@ type FlowEvent =
ts: number
source?: ValueSource
}
| { type: "node_started"; node: string }
| { type: "node_executed"; node: string; outputs: number }
| { type: "node_error"; node: string; error: string }
| { type: "node_status"; node: string; status: string; error?: string | null }
@@ -97,6 +98,9 @@ export function useFlowSocket(onAuthFailure?: () => void): void {
source: message.source,
})
break
case "node_started":
liveStore.setStatus(message.node, { status: "running" })
break
case "node_executed":
liveStore.setStatus(message.node, { status: "success" })
if (message.outputs > 0) liveStore.recordEmit(message.node)