The worker script is handed to the interpreter by path, so app/flow was sys.path[0] for every node: `import queue` got the engine's. It now drops its own directory before anything else imports, and runs with the deployment's credentials scrubbed out of its environment. Also: reload builds off the event loop, the pool wakes what is blocked on it when it stops, a refused metrics flush is kept for the next one rather than dropped, and the cascade events are paired through failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
16 KiB
This file captures tasks which derive from roadmap tasks (unfinished, deferred), bugs encountered during usage and feature requests/improvements which are not fitting directly in the roadmap. Always sort by priority and put tasks blocked by other tasks/features at the dedicated section. When working on a task, check for other, similar tasks that could be resolved on the way. Use following pattern to classify tasks: TYPE/SCOPE Where TYPE could be BUG, FEAT, PERF, CHORE and SCOPE could be UX, UI, FLOW, NODE, API, INFRA, DOCS appended by MOBILE if only for mobile use case. Don't write temporary reasons for deferring a task in the task description (only strategical reasons should be noted). Deferring because out of scope is fine, but don't mention deferring than.
Deferred holds what stays open on purpose, each with the condition that
should reopen it.
Open
To be sorted
- INFRA: merge the philosophy statement at the beginning of vision.md into the rest of the document. Dissolve the decision dates and fold the decisions into a clean structure
- BUG/UI mobile friendly support is degraded: 1) toolbar in the "Flows" viewport extend mobile viewport width 2) position of nodes should never be static; always adjust such that there are as few as possible overlaps (of nodes and edge labels) and direction is left to right (desktop) or top to bottom (mobile) with a minimal (but clean) overall edge length 3) Dashboard view is not mobile friendly at all; as dashboard design is infeasible on mobile, render all widgets in a vertically stacked order. This allows to inspect each widget and make changes. Layout changes are not a feature on mobile 4) the home view is not responsive; all items shown there should re-order on mobile such that no scrollbars appear
- BUG/UI remove the up/down counters from integer fields (global)
Connector write paths
Needs someone watching the real hardware, so it is not a background task. This is what M4 still waits on, together with porting the flows.
- FEAT/NODE: the connectors only read. Enable the write paths with someone watching: WF-RAC
setAirconStat(needs an operatorId registered with the unit first, which is itself a write) and Art-Nettransmit. - FEAT/NODE: the second WF-RAC unit (the one Node-RED addresses with operatorId "0") closes the connection on an anonymous read. It likely wants an account registered; the first unit answers without one.
- CHORE/NODE:
wfracreportsmodeas "unknown" while the unit is off, because the mode bits hold a value outside the known set. Faithful to the reference decoder, but "off" would read better.
Bugs found while building the screens
- BUG/API:
POST /alerts/test/{channel}always answers 200.AlertManager.send()catches and logs every delivery failure, so the alerts screen's Test button cannot tell a working channel from a broken one — the one thing it exists for. Letsend()raise or return a result on the test path. - BUG/FLOW: deleting a flow leaves its
pipeline:{flow}.*Redis keys behind, and renaming one does not migrate them — the live instance carriespipeline:__history__:dashboar.testbeside the correctdashboard.test. One cleanup on the delete/rename path covers both. - CHORE/API: revoking an OAuth client does not invalidate access tokens already issued; they are stateless JWTs valid up to
MCP_TOKEN_EXPIRE_MINUTES. Immediate revocation meansapp/mcp/http.pychecking the client row still exists. - CHORE/FLOW:
Pipeline.trigger's docstring says a paused flow still publishes so the value shows on the canvas. True only without a queue; with one the item parks beforeapply_outputsand nothing shows. Docstring and behaviour disagree. - CHORE/FLOW:
_to_messageskeeps itsif not retval: return Noneguard ahead of the new type check, so a falsy non-dict return (0,"",[]) is still silently "no output" rather than the named error. Deliberate for now; worth a decision. - CHORE/FLOW:
WorkItem.kind == "node"("executes exactly one node") was documented but never implemented. If a run-one-node item is wanted, it still needs writing.
Out-of-process nodes and modules
- BUG/API:
POST /flows/{name}/nodes/{node_id}/triggeranswers 500 when the node's code raises, because the inline trigger path runsNode.__call__rather thanPipeline._execute_nodeand nothing catches it. Predates the worker pool, which only made it easier to hit; the person waiting on the response should get the node's error, not a stack trace in the server log. - CHORE/FLOW:
PythonWorkerPool._runningis keyed by node id and last-wins, so two concurrent runs of one node mean cancel kills the newest. Key by run id once M5's run records exist. - CHORE/FLOW:
compile_checksends the draft source under the running node's cache key, so the worker recompiles the published source on its next call. Correct, but one wasted compile per save on a busy node. - FEAT/API:
POST /modules/applyrebuilds the whole pipeline so a node that could not import its package stops being red. That resubscribes every MQTT node in the deployment; a targeted rebuild of the flows that actually failed to load would be gentler. - CHORE/FLOW: a node's return value now round-trips through JSON, so tuples arrive downstream as lists and anything non-JSON is an explicit error. That is the message contract, but flows written before this may notice.
- BUG/FLOW: a node whose cold-start imports plus body exceed its timeout can never succeed. The timeout covers the first call's imports, a timeout kills the worker so the next attempt is cold again, and
compile()only ever warms one of the N workers. Broadcastingcompileto every worker is the candidate fix, at the cost of N module executions per reload. - CHORE/FLOW: worker protocol loose ends — the request
idis echoed but never checked,json.dumpsruns twice per result (once to prove it is JSON, once to send it),_remote_typesis an unbounded cache keyed on class names that user code chooses, andPythonWorkerPool._lockguards less than its name suggests.
Engine history
- CHORE/FLOW: a rate-limit flush gets no run record — it is the tail of the run that scheduled it, and there is no id linking the two. A flush that fails therefore shows as a failure with no run beside it.
- CHORE/FLOW:
Pipeline.flushreleasing a held value runs its cascade without a run id, so those executions land in the minute rollups but in no run. Threading the scheduling run's id through the queue item would close it. - CHORE/API: the metrics collector is a bus subscriber, so a storm that overflows the bus queue undercounts. The events dropped are the same ones the websocket drops; exact accounting would need the collector to be fed from the engine rather than the bus.
- CHORE/API:
/observability/summaryreports the work queue'sdepthas the Redis stream length, which is the journal size (capped atSTREAM_MAXLEN) rather than a backlog. The health screen showspendinginstead; the field name still invites the wrong reading. - FEAT/UI: the health screen's window is fixed at 24 hours and the charts fold minute buckets in Python. A range picker (and
date_bin()behind it) is the next step if anyone wants a week. - CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but
forget_flowcould offer to clear it. - CHORE/API: nothing can ask the collector to flush now, so anything needing the tables to be current has to wait out
FLUSH_INTERVAL_S— which is what the soak harness does before clearing its own rows. - BUG/UI:
MemoryWorkQueue.stats()hard-codespending: 0, so the health tile always reads zero on a stack without Redis. - CHORE/API:
MetricsCollector._start_run'sexisting is not Nonebranch is unreachable: a redelivery only arrives after the record it would update has been dropped. - CHORE/API: the collector's
_tracebacksstash is never pruned for nodes that stop failing — one entry per node that ever errored, kept for the life of the process. - CHORE/API: audit rows ride the same drop-oldest bus as telemetry, so a storm can lose one. Writing a node's source is not audited either; publishing is.
- PERF/API: two unmarked ceilings —
/observability/flowsscans everymetric_minuterow in the window on each 30 s poll, andqueue.stats()does a keyspacescan_iteron every call while two endpoints poll it. - CHORE/INFRA: dev only — memory-queue ids (
mem-{seq}) restart at 0 each boot andFlowRun.idis the primary key, so a restart without Redis upserts over the previous boot's run rows.
Dashboard follow-ups
- BUG/UI: ensure dashboard wallpanel (read-only) links hot reload automatically on dashboard changes
- BUG/UI: shrinking the canvas silently clips whatever now falls past its bottom edge.
maxRowsonly constrains a new drag, not a stored placement, so nothing warns and nothing offers to reflow. - CHORE/UX: dropping a widget also selects it, which opens its panel — which rescales the canvas the instant you let go. Correct, but it lurches; either leave the panel closed on a drag-release or animate the scale.
- CHORE/UI:
ROW_HEIGHTis a fixed 80px while column width follows the canvas, so a 1920-wide panel at 12 columns has 160×80 cells. If that reads too wide, the row height could derive from the canvas too. - FEAT/UI: multi-page and multi-section dashboards have no UI. The backend has
PageDef/SectionDefand rename; the editor only ever editssectionsOf(page)[0], so nothing can create a second page. - FEAT/UI: only
layout.lgis ever written. Belowlgthe view stacks widgets full width in CSS, somd/smstay unused until a per-breakpoint editor exists. - PERF/UI:
ChartWidgetre-joins the whole table on every live value. Fine at IoT rates; atHISTORY_CAP× 5 series it should append into a ring buffer. - CHORE/UI: opening edit mode on a dashboard whose widgets predate placement writes the migrated positions immediately, bumping the version once.
- CHORE/API: no backend test for the
WidgetDefdtype validator orcolumns.
Flow editor follow-ups
-
CHORE/UI: a node's error status clears as soon as it runs again, so a failure that genuinely fired an alert can leave no trace on the canvas by the time anyone looks. The logs panel keeps the traceback; the node itself reads as healthy.
-
PERF/FLOW: every save rebuilds the whole pipeline. Fine at the current flow count; rebuild only the touched flow when it starts to show.
-
CHORE/API:
POST /flows/{name}/renameis no longer reachable from the UI. A flow's title is what the panel edits, matching how nodes work; the canonical name is fixed at creation, so either the endpoint goes or renaming comes back deliberately. -
BUG/UI:
renderedNodesoverwrites xyflow's ownselectedflag withid === selectedId, so a box-selection of several nodes is invisible even though delete and copy act on all of them. -
CHORE/UI: ⌘C/⌘V
preventDefaulton the canvas blocks the native clipboard there (fields are guarded). The node clipboard islocalStorage, so it does not cross browsers or profiles. -
PERF/UI:
useParamSuggestionsfetches every flow's detail to build the suggestion list. An aggregate endpoint if an installation ever has many flows. -
CHORE/UX: the derived-cron chip also appears on the delay node, where
intervalis a rate limit rather than a schedule. May want it inject-only. -
CHORE/UX: free-form params (python nodes) get no suggestions, since there is no schema to key them off.
-
PERF/UI:
BrainViewruns 300 force-layout ticks synchronously inside auseMemo, so the graph is laid out on the render thread. -
FEAT/UI: the brain graph draws connections as plain lines, so which way a value travels is not shown. An arrowhead needs the path shortened to the target circle's rim, which needs the node radius on the edge.
-
CHORE/UI: fixing the yellow message pulse gave
edge-pulsean explicitto, so a selected edge now decays to--muted-foregroundand snaps back to its blue at the end instead of decaying into it. Barely visible, and only while an edge is selected.
Infrastructure
- CHORE/INFRA: the
playwrightcompose service cannot reachapi.localhost, somake verify-dockeris the only containerised route. (Native Playwright now works: the headless-shell libs are installed. Only the headless shell is downloaded —--headedstill needsbunx playwright install chromium, and there is no emoji font, so 👋 renders as tofu in screenshots.) - CHORE/DOCS:
app/development.mdstill presentsdocker compose watchas the dev flow; it and the Makefile targets disagree about how the stack is started. - CHORE/UI:
make lint-frontendisbiome check --write --unsafe ./— a lint target that rewrites the whole tree rather than checking it. A checking target plus a separateformatwould be safer. - CHORE/UI:
routeTree.gen.tswas generated by an older router version than the installed one; the next build reorders ~130 lines regardless of who touched it. - CHORE/UI: the alerts screen duplicates the backend's
ALERTING_EVENTS; the chooser drifts if the backend set grows. A rule with nothing ticked covers everything, so it fails soft. - CHORE/UI:
tests/runtime.spec.tsstill calls the home page "the dashboard" (dashboard-flow-row), which now collides with the dashboards feature. - CHORE/INFRA:
make soak's redis scenario stops the container the whole stack shares, so every flow briefly fails to journal, not just the soak fixtures. They recover on their own — nothing was dead-lettered or quarantined in the run this note comes from — but it is not a thing to run against a stack someone is relying on. - CHORE/INFRA: the soak harness's engine kill only catches a couple of items unacknowledged, because a cascade finishes in about four milliseconds. Redelivery is proven but barely stressed; a fixture node with a deliberate sleep would widen the window enough to test it properly.
Deferred
Open on purpose. Each names what should bring it back.
- PERF/UI: the app's entry chunk exceeds the warning threshold. React Flow and Monaco are already lazy; a manualChunks split measured no better, so this needs route-level work on the shell rather than chunking config.
- PERF/UI: the Monaco chunk is 2.6 MB. It only loads when a node panel opens, but the editor could be trimmed further or swapped for CodeMirror if that becomes a problem.
- CHORE/API: node source saves carry no version precondition, so two clients editing the same node's code are last-writer-wins. The flow document is what the optimistic lock protects; code files would need their own, and an exact-match one produces false conflicts against a single client's own interleaved flow and source saves. Revisit with the M5 multi-user work.
- CHORE/FLOW: shared node sources bypass the draft/publish split. Editing one writes the library copy and reloads immediately, since the code is not any single flow's to hold back. Deliberate, but it means a shared node is the one thing publish does not gate.
- CHORE/INFRA:
requires-pythonis capped below 3.14 because the MCP SDK wants a newer starlette there than the pinnedsentry-sdk<2allows. Lift the cap when sentry-sdk moves to 2.x. - CHORE/INFRA:
bun run --filter frontend buildfails on this workspace withcrypto.hash is not a function— Vite 7 wants Node 20.12+ and the host has 18. The Docker image builds fine, so it only bites local bundling;bunx tscstill type-checks. - FEAT/UI: an endpoint's edge routes straight across the graph, so it can pass behind a node that sits between the lane and the node it wires to. Readable, but a routed edge would be tidier.
Blocked
- CHORE/INFRA:
bun installinside the frontend Docker build intermittently fails with "Fail extracting tarball" for several packages at once, and succeeds on a plain rebuild. It looks like concurrent extraction under memory pressure. Pin down or retry in the Dockerfile if it starts costing CI time.