/observability/runs gains since/until, so the throughput chart's pin asks the server for its minute instead of filtering a fixed recent list. This engine writes ~60 runs a minute, so any minute but the newest read empty. since is inclusive and until exclusive, matching the minute buckets the charts are drawn from. Hover stays the client-side preview it was: scrubbing a day would otherwise be a request per minute rested on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
120 lines
16 KiB
Markdown
120 lines
16 KiB
Markdown
This file captures tasks which derive from roadmap tasks (unfinished, deferred), bugs encountered during usage and feature requests/improvements which are not fitting directly in the roadmap.
|
||
Always sort by priority and put tasks blocked by other tasks/features at the dedicated section.
|
||
When working on a task, check for other, similar tasks that could be resolved on the way.
|
||
Use following pattern to classify tasks: TYPE/SCOPE
|
||
Where TYPE could be BUG, FEAT, PERF, CHORE and SCOPE could be UX, UI, FLOW, NODE, API, INFRA, DOCS appended by MOBILE if only for mobile use case.
|
||
Don't write temporary reasons for deferring a task in the task description (only strategical reasons should be noted).
|
||
Deferring because out of scope is fine, but don't mention deferring than.
|
||
|
||
`Deferred` holds what stays open on purpose, each with the condition that
|
||
should reopen it.
|
||
|
||
## Open
|
||
|
||
### To be sorted
|
||
|
||
- INFRA: merge the philosophy statement at the beginning of vision.md into the rest of the document. Dissolve the decision dates and fold the decisions into a clean structure
|
||
- BUG/UI mobile friendly support is degraded: 1) toolbar in the "Flows" viewport extend mobile viewport width 2) position of nodes should never be static; always adjust such that there are as few as possible overlaps (of nodes and edge labels) and direction is left to right (desktop) or top to bottom (mobile) with a minimal (but clean) overall edge length 3) Dashboard view is not mobile friendly at all; as dashboard design is infeasible on mobile, render all widgets in a vertically stacked order. This allows to inspect each widget and make changes. Layout changes are not a feature on mobile 4) the home view is not responsive; all items shown there should re-order on mobile such that no scrollbars appear
|
||
|
||
### Connector write paths
|
||
|
||
Needs someone watching the real hardware, so it is not a background task. This
|
||
is what M4 still waits on, together with porting the flows.
|
||
|
||
- FEAT/NODE: the connectors only read. Enable the write paths with someone watching: WF-RAC `setAirconStat` (needs an operatorId registered with the unit first, which is itself a write) and Art-Net `transmit`.
|
||
- FEAT/NODE: the second WF-RAC unit (the one Node-RED addresses with operatorId "0") closes the connection on an anonymous read. It likely wants an account registered; the first unit answers without one.
|
||
- CHORE/NODE: `wfrac` reports `mode` as "unknown" while the unit is off, because the mode bits hold a value outside the known set. Faithful to the reference decoder, but "off" would read better.
|
||
|
||
### Bugs found while building the screens
|
||
|
||
- BUG/API: `POST /alerts/test/{channel}` always answers 200. `AlertManager.send()` catches and logs every delivery failure, so the alerts screen's Test button cannot tell a working channel from a broken one — the one thing it exists for. Let `send()` raise or return a result on the test path.
|
||
- BUG/FLOW: deleting a flow leaves its `pipeline:{flow}.*` Redis keys behind, and renaming one does not migrate them — the live instance carries `pipeline:__history__:dashboar.test` beside the correct `dashboard.test`. One cleanup on the delete/rename path covers both.
|
||
- CHORE/API: revoking an OAuth client does not invalidate access tokens already issued; they are stateless JWTs valid up to `MCP_TOKEN_EXPIRE_MINUTES`. Immediate revocation means `app/mcp/http.py` checking the client row still exists.
|
||
- CHORE/FLOW: `Pipeline.trigger`'s docstring says a paused flow still publishes so the value shows on the canvas. True only without a queue; with one the item parks before `apply_outputs` and nothing shows. Docstring and behaviour disagree.
|
||
- CHORE/FLOW: `_to_messages` keeps its `if not retval: return None` guard ahead of the new type check, so a falsy non-dict return (`0`, `""`, `[]`) is still silently "no output" rather than the named error. Deliberate for now; worth a decision.
|
||
- CHORE/FLOW: `WorkItem.kind == "node"` ("executes exactly one node") was documented but never implemented. If a run-one-node item is wanted, it still needs writing.
|
||
|
||
### Out-of-process nodes and modules
|
||
|
||
- BUG/API: `POST /flows/{name}/nodes/{node_id}/trigger` answers 500 when the node's code raises, because the inline trigger path runs `Node.__call__` rather than `Pipeline._execute_node` and nothing catches it. Predates the worker pool, which only made it easier to hit; the person waiting on the response should get the node's error, not a stack trace in the server log.
|
||
- CHORE/FLOW: `PythonWorkerPool._running` is keyed by node id and last-wins, so two concurrent runs of one node mean cancel kills the newest. Key by run id once M5's run records exist.
|
||
- CHORE/FLOW: `compile_check` sends the *draft* source under the running node's cache key, so the worker recompiles the published source on its next call. Correct, but one wasted compile per save on a busy node.
|
||
- FEAT/API: `POST /modules/apply` rebuilds the whole pipeline so a node that could not import its package stops being red. That resubscribes every MQTT node in the deployment; a targeted rebuild of the flows that actually failed to load would be gentler.
|
||
- CHORE/FLOW: a node's return value now round-trips through JSON, so tuples arrive downstream as lists and anything non-JSON is an explicit error. That is the message contract, but flows written before this may notice.
|
||
- BUG/FLOW: a node whose cold-start imports plus body exceed its timeout can never succeed. The timeout covers the first call's imports, a timeout kills the worker so the next attempt is cold again, and `compile()` only ever warms one of the N workers. Broadcasting `compile` to every worker is the candidate fix, at the cost of N module executions per reload.
|
||
- CHORE/FLOW: worker protocol loose ends — the request `id` is echoed but never checked, `json.dumps` runs twice per result (once to prove it is JSON, once to send it), `_remote_types` is an unbounded cache keyed on class names that user code chooses, and `PythonWorkerPool._lock` guards less than its name suggests.
|
||
|
||
### Engine history
|
||
|
||
- CHORE/FLOW: a rate-limit flush gets no run record — it is the tail of the run that scheduled it, and there is no id linking the two. A flush that fails therefore shows as a failure with no run beside it.
|
||
- CHORE/FLOW: `Pipeline.flush` releasing a held value runs its cascade without a run id, so those executions land in the minute rollups but in no run. Threading the scheduling run's id through the queue item would close it.
|
||
- CHORE/API: the metrics collector is a bus subscriber, so a storm that overflows the bus queue undercounts. The events dropped are the same ones the websocket drops; exact accounting would need the collector to be fed from the engine rather than the bus.
|
||
- CHORE/API: `/observability/summary` reports the work queue's `depth` as the Redis stream length, which is the journal size (capped at `STREAM_MAXLEN`) rather than a backlog. The health screen shows `pending` instead; the field name still invites the wrong reading.
|
||
- FEAT/UI: the health screen's window is fixed at 24 hours and the charts fold minute buckets in Python. A range picker (and `date_bin()` behind it) is the next step if anyone wants a week.
|
||
- CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but `forget_flow` could offer to clear it.
|
||
- CHORE/API: nothing can ask the collector to flush now, so anything needing the tables to be current has to wait out `FLUSH_INTERVAL_S` — which is what the soak harness does before clearing its own rows.
|
||
- BUG/UI: `MemoryWorkQueue.stats()` hard-codes `pending: 0`, so the health tile always reads zero on a stack without Redis.
|
||
- CHORE/API: `MetricsCollector._start_run`'s `existing is not None` branch is unreachable: a redelivery only arrives after the record it would update has been dropped.
|
||
- CHORE/API: the collector's `_tracebacks` stash is never pruned for nodes that stop failing — one entry per node that ever errored, kept for the life of the process.
|
||
- CHORE/API: audit rows ride the same drop-oldest bus as telemetry, so a storm can lose one. Writing a node's source is not audited either; publishing is.
|
||
- PERF/API: two unmarked ceilings — `/observability/flows` scans every `metric_minute` row in the window on each 30 s poll, and `queue.stats()` does a keyspace `scan_iter` on every call while two endpoints poll it.
|
||
- CHORE/INFRA: dev only — memory-queue ids (`mem-{seq}`) restart at 0 each boot and `FlowRun.id` is the primary key, so a restart without Redis upserts over the previous boot's run rows.
|
||
|
||
### Dashboard follow-ups
|
||
|
||
- BUG/UI: ensure dashboard wallpanel (read-only) links hot reload automatically on dashboard changes
|
||
- BUG/UI: shrinking the canvas silently clips whatever now falls past its bottom edge. `maxRows` only constrains a new drag, not a stored placement, so nothing warns and nothing offers to reflow.
|
||
- CHORE/UX: dropping a widget also selects it, which opens its panel — which rescales the canvas the instant you let go. Correct, but it lurches; either leave the panel closed on a drag-release or animate the scale.
|
||
- CHORE/UI: `ROW_HEIGHT` is a fixed 80px while column width follows the canvas, so a 1920-wide panel at 12 columns has 160×80 cells. If that reads too wide, the row height could derive from the canvas too.
|
||
- FEAT/UI: multi-page and multi-section dashboards have no UI. The backend has `PageDef`/`SectionDef` and rename; the editor only ever edits `sectionsOf(page)[0]`, so nothing can create a second page.
|
||
- FEAT/UI: only `layout.lg` is ever written. Below `lg` the view stacks widgets full width in CSS, so `md`/`sm` stay unused until a per-breakpoint editor exists.
|
||
- PERF/UI: `ChartWidget` re-joins the whole table on every live value. Fine at IoT rates; at `HISTORY_CAP` × 5 series it should append into a ring buffer.
|
||
- CHORE/UI: opening edit mode on a dashboard whose widgets predate placement writes the migrated positions immediately, bumping the version once.
|
||
- CHORE/API: no backend test for the `WidgetDef` dtype validator or `columns`.
|
||
|
||
### Flow editor follow-ups
|
||
|
||
- CHORE/UI: a node's error status clears as soon as it runs again, so a failure that genuinely fired an alert can leave no trace on the canvas by the time anyone looks. The logs panel keeps the traceback; the node itself reads as healthy.
|
||
|
||
- PERF/FLOW: every save rebuilds the whole pipeline. Fine at the current flow count; rebuild only the touched flow when it starts to show.
|
||
- CHORE/API: `POST /flows/{name}/rename` is no longer reachable from the UI. A flow's title is what the panel edits, matching how nodes work; the canonical name is fixed at creation, so either the endpoint goes or renaming comes back deliberately.
|
||
- BUG/UI: `renderedNodes` overwrites xyflow's own `selected` flag with `id === selectedId`, so a box-selection of several nodes is invisible even though delete and copy act on all of them.
|
||
- CHORE/UI: ⌘C/⌘V `preventDefault` on the canvas blocks the native clipboard there (fields are guarded). The node clipboard is `localStorage`, so it does not cross browsers or profiles.
|
||
- PERF/UI: `useParamSuggestions` fetches every flow's detail to build the suggestion list. An aggregate endpoint if an installation ever has many flows.
|
||
- CHORE/UX: the derived-cron chip also appears on the delay node, where `interval` is a rate limit rather than a schedule. May want it inject-only.
|
||
- CHORE/UX: free-form params (python nodes) get no suggestions, since there is no schema to key them off.
|
||
- PERF/UI: `BrainView` runs 300 force-layout ticks synchronously inside a `useMemo`, so the graph is laid out on the render thread.
|
||
- FEAT/UI: the brain is a band on a scrolling page now, so it neither pans nor zooms — the fit keeps the whole graph in view instead. An installation with enough flows to make the labels unreadable at that fit needs a way to open the graph larger.
|
||
- FEAT/UI: the brain graph draws connections as plain lines, so which way a value travels is not shown. An arrowhead needs the path shortened to the target circle's rim, which needs the node radius on the edge.
|
||
- CHORE/UI: fixing the yellow message pulse gave `edge-pulse` an explicit `to`, so a *selected* edge now decays to `--muted-foreground` and snaps back to its blue at the end instead of decaying into it. Barely visible, and only while an edge is selected.
|
||
|
||
### Infrastructure
|
||
|
||
- CHORE/INFRA: the `playwright` compose service cannot reach `api.localhost`, so `make verify-docker` is the only containerised route. (Native Playwright now works: the headless-shell libs are installed. Only the headless shell is downloaded — `--headed` still needs `bunx playwright install chromium`, and there is no emoji font, so 👋 renders as tofu in screenshots.)
|
||
- CHORE/DOCS: `app/development.md` still presents `docker compose watch` as the dev flow; it and the Makefile targets disagree about how the stack is started.
|
||
- CHORE/UI: `make lint-frontend` is `biome check --write --unsafe ./` — a lint target that rewrites the whole tree rather than checking it. A checking target plus a separate `format` would be safer.
|
||
- CHORE/UI: `routeTree.gen.ts` was generated by an older router version than the installed one; the next build reorders ~130 lines regardless of who touched it.
|
||
- CHORE/UI: the alerts screen duplicates the backend's `ALERTING_EVENTS`; the chooser drifts if the backend set grows. A rule with nothing ticked covers everything, so it fails soft.
|
||
- CHORE/UI: `tests/runtime.spec.ts` still calls the home page "the dashboard" (`dashboard-flow-row`), which now collides with the dashboards feature.
|
||
- CHORE/INFRA: `make soak`'s redis scenario stops the container the whole stack shares, so every flow briefly fails to journal, not just the soak fixtures. They recover on their own — nothing was dead-lettered or quarantined in the run this note comes from — but it is not a thing to run against a stack someone is relying on.
|
||
- CHORE/INFRA: the soak harness's engine kill only catches a couple of items unacknowledged, because a cascade finishes in about four milliseconds. Redelivery is proven but barely stressed; a fixture node with a deliberate sleep would widen the window enough to test it properly.
|
||
|
||
## Deferred
|
||
|
||
Open on purpose. Each names what should bring it back.
|
||
|
||
- PERF/UI: the app's entry chunk exceeds the warning threshold. React Flow and Monaco are already lazy; a manualChunks split measured no better, so this needs route-level work on the shell rather than chunking config.
|
||
- PERF/UI: the Monaco chunk is 2.6 MB. It only loads when a node panel opens, but the editor could be trimmed further or swapped for CodeMirror if that becomes a problem.
|
||
- CHORE/API: node source saves carry no version precondition, so two clients editing the same node's code are last-writer-wins. The flow document is what the optimistic lock protects; code files would need their own, and an exact-match one produces false conflicts against a single client's own interleaved flow and source saves. Revisit with the M5 multi-user work.
|
||
- CHORE/FLOW: shared node sources bypass the draft/publish split. Editing one writes the library copy and reloads immediately, since the code is not any single flow's to hold back. Deliberate, but it means a shared node is the one thing publish does not gate.
|
||
- CHORE/INFRA: `requires-python` is capped below 3.14 because the MCP SDK wants a newer starlette there than the pinned `sentry-sdk<2` allows. Lift the cap when sentry-sdk moves to 2.x.
|
||
- CHORE/INFRA: `bun run --filter frontend build` fails on this workspace with `crypto.hash is not a function` — Vite 7 wants Node 20.12+ and the host has 18. The Docker image builds fine, so it only bites local bundling; `bunx tsc` still type-checks.
|
||
- FEAT/UI: an endpoint's edge routes straight across the graph, so it can pass behind a node that sits between the lane and the node it wires to. Readable, but a routed edge would be tidier.
|
||
|
||
## Blocked
|
||
|
||
- CHORE/INFRA: `bun install` inside the frontend Docker build intermittently fails with
|
||
"Fail extracting tarball" for several packages at once, and succeeds on a plain rebuild.
|
||
It looks like concurrent extraction under memory pressure. Pin down or retry in the
|
||
Dockerfile if it starts costing CI time.
|