Files
app/NOTEPAD.md
T
stroblmeandClaude Opus 5 e7a1466d7b dashboard: pace a querying chart by its window, and an example to evaluate it
A chart is drawn in buckets, and nothing it can show changes until the
bucket it is drawing closes — so the resolution sets the refresh rather
than a flat five-second floor. A week at quarter-hour buckets now asks
four times an hour instead of sixty, for the same picture. Leaving the
field empty follows the window; a slower rate is still honoured.

`make seed-example` builds the thing to evaluate it with: a flow that
logs a temperature to InfluxDB, a flow that answers a chart's request by
turning the window into Flux and the rows back into a series, and a
dashboard holding the chart. The reading flow declares the request as an
input with a starting value, which is how a flow says a value reaches it
from a panel rather than from a node upstream.

Axis labels keep enough decimals to stay distinct — `si` rounds to three
figures, so every tick of a chart living inside one degree read "19".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:34:32 +02:00

28 KiB
Raw Blame History

This file captures tasks which derive from roadmap tasks (unfinished, deferred), bugs encountered during usage and feature requests/improvements which are not fitting directly in the roadmap. Always sort by priority and put tasks blocked by other tasks/features at the dedicated section. When working on a task, check for other, similar tasks that could be resolved on the way. Use following pattern to classify tasks: TYPE/SCOPE Where TYPE could be BUG, FEAT, PERF, CHORE and SCOPE could be UX, UI, FLOW, NODE, API, INFRA, DOCS appended by MOBILE if only for mobile use case. Don't write temporary reasons for deferring a task in the task description (only strategical reasons should be noted). Deferring because out of scope is fine, but don't mention deferring than.

Deferred holds what stays open on purpose, each with the condition that should reopen it.

Open

To be sorted

  • BUG/UI when a dashboard widget is selectd, the border does not cleanly draw on the left side of the widget
  • BUG/UI there is currently no option to discard changes (in dashboard or flow viewport); we should make add a "X" button to discard and replace "Publish" by the already existing checkmark button which is either clickable when there are unpublished changes or unclickable when the changes have been applied (but the icon stays). The discard button should only show when there are changes (hidden else)
  • INFRA: ensure that all the packages/ dependencies needed to run fluksio are available on arm to make this software runnable on e.g. raspbian
  • INFRA: merge the philosophy statement at the beginning of vision.md into the rest of the document. Dissolve the decision dates and fold the decisions into a clean structure
  • BUG/UI mobile friendly support is degraded: 1) toolbar in the "Flows" viewport extend mobile viewport width 2) position of nodes should never be static (holds true for desktop as well); always adjust such that there are as few as possible overlaps (of nodes and edge labels) and direction is left to right (desktop) or top to bottom (mobile) with a minimal (but clean) overall edge length. This should also remove the ability to drag nodes around; their position is fixed by an algorithm. This design choice is what enforces small atomic flows (different from nodered) 3) Dashboard view is not mobile friendly at all; as dashboard design is infeasible on mobile, render all widgets in a vertically stacked order. This allows to inspect each widget and make changes. Layout changes are not a feature on mobile 4) the home view is not responsive; all items shown there should re-order on mobile such that no scrollbars appear. Make sure the mobile support is anchored in the design such that future work does not break it
  • FEAT/UI add a loading animation for the initial app load and when loading individual pages; make sure that elements e.g. in the home dashboard load independently to ensure a fast loading of the initial site but figures charts, tables, graph etc. follow after that
  • FEAT/UI introduce a graph panel which renders at the top right next to the graph view (to make more use of the horizontal space) and which allows (de-) selecting flows to be excluded from the graph view or search for individual nodes where only the flows containing this node should be shown (like slicing the brain)
  • FEAT/UI durations are written as a shortened number beside a fixed unit, so a slow run reads "1.2k ms" rather than "1.2 s". A duration formatter that steps the unit itself (µs/ms/s/min) would read better wherever si is followed by "ms"
  • CHORE/UI biome check ./src reports an ineffective suppression at FlowEditor.tsx:473 (useExhaustiveDependencies no longer fires there)
  • FEAT/UI the brain's activity falloff is session-observed: a page just opened shows every neuron and connection at the same neutral base, and only sorts itself out as values arrive. A "last published" timestamp per node from the backend would let it open already sorted.
  • CHORE/UI the brain's hover labels have no touch equivalent — a tap navigates to the flow, and there is no hover to reveal a name first. The native title carries it on desktop only.
  • BUG/UI slightly increase the margin between the top of a graph in the node/flow panel and the consumer/producer field
  • FEAT/UI labels in flows (indicating dashboard widget connections) naturally can't pulse. Instead add an animation (enlightning fade) from either ltr or rtl depending if the label is in- or outbound
  • FEAT/UI (deferred until MCP lands): add a "bot" icon button to the home view (graph panel) which opens a chat window (reuse general concept of a side panel like in flows/nodes to make it a chat panel which can open on any screen (stacks below any other existing panel -> introduce stacking) to give support on errors/write code, generate dashboards etc) to explain the error(s)

Persistence and databases

From the 2026-08 database review. Verdict recorded under Deferred: the Postgres + Redis + git-files split stays; the actionable part is durability.

  • BUG/INFRA: scripts/backup.sh dumps Postgres only — the flow git repo, secrets.enc, oauth-key.pem (losing it invalidates every issued MCP token), alerts.json and the Redis AOF are not backed up. Extend it to tar app-flow-data and copy a BGSAVE'd Redis snapshot. Its header comment still claims "the flow engine keeps no on-disk state", which stopped being true when flows became a git repo.
  • CHORE/FLOW: state backend and work queue share the pipeline: prefix in db 0, so RedisState.clear() would DEL the work-queue stream and RedisState.keys() enumerates queue keys — only callers filtering __-prefixed names keep it safe. A separate prefix (or db index) for the queue removes the hazard.
  • CHORE/INFRA: Redis AOF runs at appendfsync everysec, so up to ~1 s of journaled work-queue entries can vanish on a crash — softer than "journaled before it runs" reads. Queue write volume is low, so appendfsync always is likely affordable; otherwise document the loss window.
  • CHORE/INFRA: Redis has no auth (requirepass unset). Fine on the compose-internal network; a blocker for M5 remote workers, which turn Redis into a network-exposed shared bus.
  • CHORE/INFRA: the SQLAlchemy engine sets no pool_pre_ping, so long-idle connections throw once after a Postgres restart before the pool recovers.

Connector write paths

Needs someone watching the real hardware, so it is not a background task. This is what M4 still waits on, together with porting the flows.

  • FEAT/NODE: the connectors only read. Enable the write paths with someone watching: WF-RAC setAirconStat (needs an operatorId registered with the unit first, which is itself a write) and Art-Net transmit.
  • FEAT/NODE: the second WF-RAC unit (the one Node-RED addresses with operatorId "0") closes the connection on an anonymous read. It likely wants an account registered; the first unit answers without one.
  • CHORE/NODE: wfrac reports mode as "unknown" while the unit is off, because the mode bits hold a value outside the known set. Faithful to the reference decoder, but "off" would read better.

Bugs found while building the screens

  • CHORE/API: revoking an OAuth client does not invalidate access tokens already issued; they are stateless JWTs valid up to MCP_TOKEN_EXPIRE_MINUTES. Immediate revocation means app/mcp/http.py checking the client row still exists.
  • CHORE/FLOW: Pipeline.trigger's docstring says a paused flow still publishes so the value shows on the canvas. True only without a queue; with one the item parks before apply_outputs and nothing shows. Docstring and behaviour disagree.
  • CHORE/FLOW: WorkItem.kind == "node" ("executes exactly one node") was documented but never implemented. If a run-one-node item is wanted, it still needs writing.
  • CHORE/API: GET /observability/runs caps at 200 rows, so pinning a minute busier than that on the Home chart shows its newest 200 with nothing saying more exist. Fine at ~60 runs/min; a count alongside the rows, or paging, is what a busier instance needs.

Out-of-process nodes and modules

  • CHORE/FLOW: PythonWorkerPool._running is keyed by node id and last-wins, so two concurrent runs of one node mean cancel kills the newest. Key by run id once M5's run records exist.
  • CHORE/FLOW: compile_check sends the draft source under the running node's cache key, so the worker recompiles the published source on its next call. Correct, but one wasted compile per save on a busy node.
  • FEAT/API: POST /modules/apply rebuilds the whole pipeline so a node that could not import its package stops being red. That resubscribes every MQTT node in the deployment; a targeted rebuild of the flows that actually failed to load would be gentler.
  • CHORE/FLOW: a node's return value now round-trips through JSON, so tuples arrive downstream as lists and anything non-JSON is an explicit error. That is the message contract, but flows written before this may notice.
  • BUG/FLOW: a node whose cold-start imports plus body exceed its timeout can never succeed. The timeout covers the first call's imports, a timeout kills the worker so the next attempt is cold again, and compile() only ever warms one of the N workers. Broadcasting compile to every worker is the candidate fix, at the cost of N module executions per reload.
  • CHORE/FLOW: worker protocol loose ends — the request id is echoed but never checked, json.dumps runs twice per result (once to prove it is JSON, once to send it), _remote_types is an unbounded cache keyed on class names that user code chooses, and PythonWorkerPool._lock guards less than its name suggests.

Engine history

  • CHORE/FLOW: a rate-limit flush gets no run record — it is the tail of the run that scheduled it, and there is no id linking the two. A flush that fails therefore shows as a failure with no run beside it.
  • CHORE/FLOW: Pipeline.flush releasing a held value runs its cascade without a run id, so those executions land in the minute rollups but in no run. Threading the scheduling run's id through the queue item would close it.
  • CHORE/API: the metrics collector is a bus subscriber, so a storm that overflows the bus queue undercounts. The events dropped are the same ones the websocket drops; exact accounting would need the collector to be fed from the engine rather than the bus.
  • CHORE/API: /observability/summary reports the work queue's depth as the Redis stream length, which is the journal size (capped at STREAM_MAXLEN) rather than a backlog. The health screen shows pending instead; the field name still invites the wrong reading.
  • PERF/API: the health block picks its window now, but /observability/timeseries and /observability/flows still read every metric_minute row in it and fold them in Python. bucket_s only coarsens what comes back, so the 7d preset pulls a week of rows on each 30 s poll. date_bin() is what makes the long windows cheap.
  • CHORE/API: /observability/summary still returns failures_24h, which nothing reads any more — the Home tile counts errors over the selected window from the rollups instead. Drop the field, or let the summary take a window.
  • CHORE/UI: the Home block's "Changes" list is the newest 15 audit rows whatever range is selected. Deliberate — an audit trail is worth reading past the window — but it sits under a control that governs everything else on the screen.
  • CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but forget_flow could offer to clear it.
  • CHORE/API: nothing can ask the collector to flush now, so anything needing the tables to be current has to wait out FLUSH_INTERVAL_S — which is what the soak harness does before clearing its own rows.
  • CHORE/API: MetricsCollector._start_run's existing is not None branch is unreachable: a redelivery only arrives after the record it would update has been dropped.
  • CHORE/FLOW: RedisWorkQueue.clear_flow deletes only pipeline:__parked__:{flow}, so a deleted or renamed flow's __queue__ stream entries, __delayed__ zset members and __done__:* markers stay behind. The stream is capped and the entries are dropped when they reach a node that no longer exists, so it costs work rather than correctness.
  • CHORE/FLOW: MemoryWorkQueue's in-flight count is a counter around claim/ack, and claiming already removed the item — so an item a handler leaves unacknowledged (no pipeline bound) counts as in flight until the process ends. Nothing can hand it back either way, which is what the memory queue is.
  • CHORE/API: audit rows ride the same drop-oldest bus as telemetry, so a storm can lose one. Writing a node's source is not audited either; publishing is.
  • PERF/API: two unmarked ceilings — /observability/flows scans every metric_minute row in the window on each 30 s poll, and queue.stats() does a keyspace scan_iter on every call while two endpoints poll it.
  • CHORE/INFRA: dev only — memory-queue ids (mem-{seq}) restart at 0 each boot and FlowRun.id is the primary key, so a restart without Redis upserts over the previous boot's run rows.

Wall-panel parity with the current home dashboard

What a fluksio dashboard still lacks to replace geli-dash (Dash/Plotly, e-ink panel: clock and nav chrome, indoor climate, weather forecast strip, calendar agenda, room light groups, sliders, power/battery bars, and three pages of InfluxDB time series). Component-level only; the arrangement and the styling are this design system's business, not that one's.

Decisions taken up front, because most items below depend on them:

  • Structured data reaches a widget as a declared shape, not as opaque JSON with a path per binding. A path would leave the picker with nothing to offer and widgetIssue unable to judge a tile from the document alone.

  • A chart asks a flow for its series the way every other input widget speaks: it publishes a request message and reads the answer. No query API, no database knowledge in the widget.

  • Database nodes are transport and credentials only. The InfluxDB node runs the Flux it is handed and echoes back every other field of the request; building the query and shaping the answer are Python nodes either side of it. That is what keeps a widget ignorant of the database, and it is also what a series read mode inside the node would have prevented. A "grouped nodes" concept could later package the standard chart→build→db→parse→chart quintet so a dashboard is not five nodes of wiring each time.

  • Nothing e-ink-specific in the widgets. Panel access is a credential problem (see below); the display's demands are a rendering profile, deferred.

  • FEAT/UI: assemble the range-aware demo dashboard now that charts can query — an InfluxDB node behind a build/parse pair, with the panel's own range picker governing the window. The pieces are in and verified against a real bucket; what is missing is a dashboard someone would actually hang.

  • CHORE/UI: MarkdownWidget's docstring claims "headings, bold, code, links, list items"; only headings and bullets are implemented. Either the inline spans or the docstring.

  • CHORE/UI: identical in-flight chart requests are deduplicated per browser tab, so two wall panels showing the same tile still run the query twice. An interval on the request port is the backstop, and it belongs to the flow serving the request rather than to the widget asking.

  • CHORE/FLOW: one request/answer pair per InfluxDB node — the first input carrying a flux key is the request and the answer leaves on the first output port. A second query stream through one node needs a second node.

  • FEAT/UI: bar/level widget — a horizontal bar with its value written on it, and a nested bar for a quantity contained in another (PV inside inverter input). The arc gauge is the only level display today.

  • FEAT/UI: latching toggle button (pressed state read back from a bool message) and exclusive button group (one of N, the active one read back). The current button is fire-and-forget and the switch is a lone bool, so a set of room light modes cannot be expressed.

  • FEAT/UI: icon-by-value widget — a mapping from value or range to a lucide icon and a colour token. Carries both the weather condition icon and derived hints such as "indoor dewpoint above outdoor, open a window".

  • FEAT/UI: forecast strip widget over a list message — N columns of label, icon and value, with the later columns drawn progressively dimmer.

  • FEAT/UI: clock widget — local time and date, bound to nothing.

  • FEAT/UI: chart axis titles. The unit, the fixed y range and the per-series label are in; a named axis is not.

  • FEAT/UI: the slider offers step now, but no tick labels — the datalist marks are unlabelled and drop out past fifty steps.

  • FEAT/API: a kiosk credential for /view/{name}, so a panel is not a logged-in browser session. Note it cannot be strictly read-only: a querying chart publishes its request, so the token needs that one write scope.

  • FEAT/UI: per-dashboard theme — forced light, forced dark, or switched on a schedule. View mode inherits localStorage and the OS preference today, which a panel in a room has no way to set.

  • FEAT/UI: page navigation in view mode. /view/{name} renders the first page and offers no way to reach the others; the editor side of this is the multi-page item under Dashboard follow-ups.

  • CHORE/FLOW: porting the controls needs a declared writable message per control, since an input widget can only target what a flow declares. Consider a dashboard-input node so a flow states plainly that a value arrives from a panel, rather than each control borrowing some node's port.

Deliberately not ported: the local-state/timestamp reconciliation the old dashboard does per widget — publishing on release and reading the value back covers it — and its demo mode, since an unbound or silent message already renders as an em dash.

Dashboard follow-ups

  • BUG/UI: ensure dashboard wallpanel (read-only) links hot reload automatically on dashboard changes
  • BUG/UI: shrinking the canvas silently clips whatever now falls past its bottom edge. maxRows only constrains a new drag, not a stored placement, so nothing warns and nothing offers to reflow.
  • CHORE/UX: dropping a widget also selects it, which opens its panel — which rescales the canvas the instant you let go. Correct, but it lurches; either leave the panel closed on a drag-release or animate the scale.
  • CHORE/UI: ROW_HEIGHT is a fixed 80px while column width follows the canvas, so a 1920-wide panel at 12 columns has 160×80 cells. If that reads too wide, the row height could derive from the canvas too.
  • FEAT/UI: multi-page and multi-section dashboards have no UI. The backend has PageDef/SectionDef and rename; the editor only ever edits sectionsOf(page)[0], so nothing can create a second page.
  • FEAT/UI: only layout.lg is ever written. Below lg the view stacks widgets full width in CSS, so md/sm stay unused until a per-breakpoint editor exists.
  • PERF/UI: ChartWidget re-joins the whole table on every live value. Fine at IoT rates; at HISTORY_CAP × 5 series it should append into a ring buffer.
  • CHORE/UI: opening edit mode on a dashboard whose widgets predate placement writes the migrated positions immediately, bumping the version once.
  • FEAT/UI: POST /dashboards/{name}/discard has no button. The flow settings panel offers "discard draft"; the dashboard settings panel does not, so an unwanted edit can only be undone by hand or by publishing it.
  • CHORE/API: creating a dashboard publishes it straight away (an empty document goes to the panels), while a new flow starts as a draft. Keeps read/list free of a never-published case, at the cost of the asymmetry.
  • PERF/UI: "Publish all" reads each document's detail for the version its publish must match, so a click is 2N requests. A bulk endpoint, or a version on the summaries, would make it one.

Flow editor follow-ups

  • CHORE/UI: a node's error status clears as soon as it runs again, so a failure that genuinely fired an alert can leave no trace on the canvas by the time anyone looks. The logs panel keeps the traceback; the node itself reads as healthy.

  • PERF/FLOW: every save rebuilds the whole pipeline. Fine at the current flow count; rebuild only the touched flow when it starts to show.

  • CHORE/API: POST /flows/{name}/rename is no longer reachable from the UI. A flow's title is what the panel edits, matching how nodes work; the canonical name is fixed at creation, so either the endpoint goes or renaming comes back deliberately.

  • BUG/UI: renderedNodes overwrites xyflow's own selected flag with id === selectedId, so a box-selection of several nodes is invisible even though delete and copy act on all of them.

  • CHORE/UI: ⌘C/⌘V preventDefault on the canvas blocks the native clipboard there (fields are guarded). The node clipboard is localStorage, so it does not cross browsers or profiles.

  • PERF/UI: useParamSuggestions fetches every flow's detail to build the suggestion list. An aggregate endpoint if an installation ever has many flows.

  • CHORE/UX: the derived-cron chip also appears on the delay node, where interval is a rate limit rather than a schedule. May want it inject-only.

  • CHORE/UX: free-form params (python nodes) get no suggestions, since there is no schema to key them off.

  • PERF/UI: BrainView runs 300 force-layout ticks synchronously inside a useMemo, so the graph is laid out on the render thread.

  • FEAT/UI: the brain is a band on a scrolling page now, so it neither pans nor zooms — the fit keeps the whole graph in view instead. An installation with enough flows to make the labels unreadable at that fit needs a way to open the graph larger.

  • CHORE/UI: React Flow measures a node's handle bounds out of the DOM once and never again, and in the brain that one measurement falls inside the graph's scaleIn entrance — so every sourceX/targetX it hands an edge there is the entrance's 4% short of the centre, permanently. BrainEdge takes both ends from the layout instead (position + radius). Any future view that mounts a canvas inside a transform and reads node internals meets the same thing.

  • CHORE/UI: a value passing lights the brain's connection blue for the pulse, but its two end dots only follow the slower falloff, so the flash stops where the line does.

  • CHORE/UI: flow.css drops the focus outline on every .react-flow__node. Brain neurons now answer :focus-visible with the same border colour as hover, but the flow editor's own nodes still have no visible keyboard focus.

Infrastructure

  • CHORE/INFRA: the playwright compose service cannot reach api.localhost, so make verify-docker is the only containerised route. (Native Playwright now works: the headless-shell libs are installed. Only the headless shell is downloaded — --headed still needs bunx playwright install chromium, and there is no emoji font, so 👋 renders as tofu in screenshots.)
  • CHORE/DOCS: app/development.md still presents docker compose watch as the dev flow; it and the Makefile targets disagree about how the stack is started.
  • CHORE/UI: make lint-frontend is biome check --write --unsafe ./ — a lint target that rewrites the whole tree rather than checking it. A checking target plus a separate format would be safer.
  • CHORE/UI: routeTree.gen.ts was generated by an older router version than the installed one; the next build reorders ~130 lines regardless of who touched it.
  • CHORE/UI: the alerts screen duplicates the backend's ALERTING_EVENTS; the chooser drifts if the backend set grows. A rule with nothing ticked covers everything, so it fails soft.
  • CHORE/UI: tests/runtime.spec.ts still calls the home page "the dashboard" (dashboard-flow-row), which now collides with the dashboards feature.
  • CHORE/INFRA: make soak's redis scenario stops the container the whole stack shares, so every flow briefly fails to journal, not just the soak fixtures. They recover on their own — nothing was dead-lettered or quarantined in the run this note comes from — but it is not a thing to run against a stack someone is relying on.
  • CHORE/INFRA: the soak harness's engine kill only catches a couple of items unacknowledged, because a cascade finishes in about four milliseconds. Redelivery is proven but barely stressed; a fixture node with a deliberate sleep would widen the window enough to test it properly.

Deferred

Open on purpose. Each names what should bring it back.

  • PERF/UI: the app's entry chunk exceeds the warning threshold. React Flow and Monaco are already lazy; a manualChunks split measured no better, so this needs route-level work on the shell rather than chunking config.
  • PERF/UI: the Monaco chunk is 2.6 MB. It only loads when a node panel opens, but the editor could be trimmed further or swapped for CodeMirror if that becomes a problem.
  • CHORE/API: node source saves carry no version precondition, so two clients editing the same node's code are last-writer-wins. The flow document is what the optimistic lock protects; code files would need their own, and an exact-match one produces false conflicts against a single client's own interleaved flow and source saves. Revisit with the M5 multi-user work.
  • CHORE/FLOW: shared node sources bypass the draft/publish split. Editing one writes the library copy and reloads immediately, since the code is not any single flow's to hold back. Deliberate, but it means a shared node is the one thing publish does not gate.
  • CHORE/INFRA: requires-python is capped below 3.14 because the MCP SDK wants a newer starlette there than the pinned sentry-sdk<2 allows. Lift the cap when sentry-sdk moves to 2.x.
  • CHORE/INFRA: bun run --filter frontend build fails on this workspace with crypto.hash is not a function — Vite 7 wants Node 20.12+ and the host has 18. The Docker image builds fine, so it only bites local bundling; bunx tsc still type-checks.
  • FEAT/UI: an endpoint's edge routes straight across the graph, so it can pass behind a node that sits between the lane and the node it wires to. Readable, but a routed edge would be tidier.
  • FEAT/UI: the node-panel and edge trend curves take no range, unlike the health block. They are drawn from a Redis ring of the last 120 values per message, which has no window to ask for — a hover caption names what the curve covers instead of a picker promising a span nothing can serve. Reopen if per-message history ever gains a time window.
  • FEAT/UI: an e-ink rendering profile for a dashboard — motion off, hover-only affordances resolved to something visible, high-contrast palette, thick strokes, and a repaint cadence low enough for a display that takes a second to settle. Reopen when a panel with such a display is actually hung.
  • CHORE/INFRA: Postgres stays. The 2026-08 review rejected YugabyteDB/CockroachDB (multi-node cluster systems, ~4 GB+ RAM per node, against the small-server target — the scaling story is remote workers, not a distributed DB) and found merging Postgres into Redis or vice versa buys little: the stores hold disjoint data and both sit behind abstractions. SQLite would fit the single-instance design and drop a container; reopen if the home-install footprint becomes a product concern.
  • CHORE/INFRA: NATS JetStream as the work-queue backend — durable streams whose consumer semantics match the WorkQueue interface, in one small binary. Reopen with M5 remote workers, when the queue crosses hosts.

Blocked

  • CHORE/INFRA: bun install inside the frontend Docker build intermittently fails with "Fail extracting tarball" for several packages at once, and succeeds on a plain rebuild. It looks like concurrent extraction under memory pressure. Pin down or retry in the Dockerfile if it starts costing CI time. NOTE: memory lifted; retry and close if stale