The engine and the worker speak a version-matched protocol, but compose.dev.yml mounted only backend/fluksio: a reloaded engine talked to the worker baked into the image. Adding the request id to that protocol therefore left every python node failing with "the worker answered request None", because the old worker echoed no id at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01StpRc2C6au1WJ1EUU7fsfu
49 KiB
This file captures tasks which derive from roadmap tasks (unfinished, deferred), bugs encountered during usage and feature requests/improvements which are not fitting directly in the roadmap. Always sort by priority and put tasks blocked by other tasks/features at the dedicated section. When working on a task, check for other, similar tasks that could be resolved on the way. Use following pattern to classify tasks: TYPE/SCOPE Where TYPE could be BUG, FEAT, PERF, CHORE and SCOPE could be UX, UI, FLOW, NODE, API, INFRA, DOCS appended by MOBILE if only for mobile use case. Don't write temporary reasons for deferring a task in the task description (only strategical reasons should be noted). Deferring because out of scope is fine, but don't mention deferring than.
Deferred holds what stays open on purpose, each with the condition that should reopen it.
Open
Dashboard UI rework (dedicated session)
Rework the dashboard UI to make it more flexible in terms of the look and feel. The dasboard is allowed to break out of the design guidelines. I consider following design options a) overall appearance: material look or liquid glass (einUI https://github.com/einui/einui) b) color palette (https://colorhunt.co). Based on a) and b) it should be possible to derive almost endles combinations of looks. The color palette should affect affect the overall theming of all widgets The EinUI components should be used as a reference for the different components needed, the general layout/style of components and essentially the foundation for the liquid glass option. Based on this, a set of material ui components should be derived, following https://mui.com/ . Both variants of the components should be completely custom designed and fully owned (no new dependency). The apperance should be strictly separated from the functionality (which both sets of components should share; i.e. choosing one or the other component set should not change the features of the dashboard). All widgets (and the sidebar rail) should follow the appearance and should be reworked to receive animations through the motion library. It should be possible to set a background in the dashboard (input, controllable externally via a node. In v1 this can be just an url to an image). It should be possible to disable the title of a widget. The dashboard setting should contain a "Touch" toggle, which, when activated, makes all widgets more touch friendly
Specific changes
- the nested bar should be changed in a multi-row bar with N inputs (and therefore n bars), color separated as the chart widget
- the range picker should go to the right (vertical) of the chart widgeta
- the color picker should be changed to a circular color picker (disk with colors, saturation changes towards the center) and a vertical slider for the brightness
To be sorted
-
FEAT/UI: we promise testing, but currently don't provide an UI for testing e.g. mock values or probing edge cases of a flow. This should be resolved (in a dedicated session); I'm thinking of a "Labs" page, which allows simulating an installation with all the flows (using their draft states) and which allows injecting values or mocking values based on events in the past
-
FEAT/UI: check if PWA (https://whatpwacando.today/) notifications could be used to have a panel sending notifications to the device event bus (or generally using PWA to retrieve e.g. location etc). We could introduce a general concept of having a panel (a device, like a wall panel or a phone where the pwa (dashboard) runs) being effectively a node with various outputs. Then various inputs could trigger actions like authentification (i.e. you get home and get a notification which allows you to authenticate the door unlock), get notified on alarms (native alarm connector) or to query geolocation (check where the user is before turning of all lights) etc
-
BUG/UI sync the theme state between panels and installations
-
BUG/UI some dashboard widgets (like the color picker) are scrollable; we should make sure that no widgets (except for text widgets or list-related widgets) are scrollable
-
CHORE/UI: the house panels are laid out for 1280x800 — twelve columns, twelve rows. A chart's fixed chrome is now its title line and legend: the range picker moved up onto the title and gave back its row, so the budget is nearer forty pixels than eighty and a third chart may now fit — worth measuring against the panel before adding one. The temperature history was the one dropped;
history.climate_*still answers, so it is a tile away. -
FEAT/UI: a bar's nested reading is captioned by its port name, which is chosen for the graph rather than for somebody reading it across a room.
Segmentnow takes an optionallabel; the house dashboard sets one (Solar), and it shows on the next frontend build. -
BUG/NODE: an MQTT client can hang forever on the way out.
MqttNodebuildsaiomqtt.Clientwithout atimeout, soClient.__aexit__waits for the broker's disconnect acknowledgement with no deadline —_wait_for(..., timeout=None)falls through toself.timeout, which isNonetoo. A subscriber cancelled while its socket is dead never finishes unwinding. This was what wedged the rebuild lock; the teardown now abandons such a task after five seconds rather than waiting on it, so what is left is the task itself, which is stopped only by the second cancellation it is sent on the way out. Passing atimeoutto the client fixes it at the source, but the same number also boundssubscribeandpublish, so it wants choosing deliberately. -
BUG/FLOW:
MqttNode.stop_publisher,MqttNode.stop_subscriptionandDelayNode.stop_croncatchCancelledErroraround the task they have just cancelled, which swallows a cancellation meant for the caller — the trapSupervisor.cancel_allwas just fixed for. Latent rather than live: a supervised node leaves those handlesNone, so only a node built on its own (a test, a preview) awaits there. -
BUG/INFRA: zombie
pythonprocesses in the API container, twelve of them in groups of four with matching elapsed times — one group per worker pool._Worker.kill()waits on the process it kills, so these are workers orphaned when the process holding theirPopenwent away without stopping the pool, which is what uvicorn--reloaddoes on every source edit. They reparent onto PID 1, and PID 1 in the container is the reloader, which reaps nothing that is not its own child.init: trueon the backend service hands PID 1 to an init that does reap orphans, and would close the whole class rather than one source of it. -
CHORE/INFRA: the engine and the worker speak a version-matched protocol, but only the engine hot-reloads — uvicorn watches
/app/backend, and a worker process holds the code it started with until it is respawned. So a protocol change reaches one side and not the other, which is how every python node came to fail withRemoteError: the worker answered request None.compose.dev.ymlnow mountsworker/fluksio_workerbesidebackend/fluksio, which closes the image half. The reload half is open: editing worker source restarts nothing, and the running pool keeps the old code. -
CHORE/UI: the portal's aggregate tiles have no not-reporting state. A figure an installation never sends renders through
?? 0, so a missing field reads as a confident zero rather than as unknown — which is how a deletedfailures_24hshowed as "0 failures" unnoticed. The per-card half is one line; Home sums across installations and has nowhere to say "3 of 4 reporting", so the two want doing together. -
FEAT/SEC:
lockedis a read-only surface, not a permission — the server accepts a publish from a panel whose dashboard says locked. Making it real means carrying the flag into_panel_may. -
FEAT/UI: a dashboard forced light inside a dark app shell gets correct tokens but still matches
dark:variant utilities, because those compile to&:is(.dark *)and<html>always carries a theme class. Cosmetic (a few faint shadcn backgrounds); fixing it properly needs a theme-scope mechanism CSS ancestor selectors cannot express today. Shares a root with a bug fixed since: the token scopes are keyed on the bare class nameslight/dark, so React Flow's owncolorModeclass silently re-themed an entire canvas to light until the prop was passed explicitly. Namespacing them (.theme-light/.theme-dark) would sit under both. Two repos — the token blocks are a byte-identical duplication contract withindex/src/index.css— plususeDashboardTheme. -
CHORE/UI: the lock notice is on
PanelSurfaceonly, so/dashboards/{name}in read mode shows disabled controls without the pill. -
CHORE/DOC: the docs call a cron
injectnode "a node publishing on a schedule". The node type literally namedtriggeris a debounce/hold node, not a scheduler — worth renaming one of the two eventually. -
CHORE/DEMO: the demo panel now fills all 17 rows its 2560×1600 canvas holds, every row across all 16 columns. Another tile needs a rearrangement first — worth doing together with re-authoring
PAGESas a single section. -
CHORE/TEST: no Playwright coverage for the colour tile —
tests/widgets.spec.tscovers the other controls. Wants a spec that drags the hue ring and asserts one publish on release. -
CHORE/UI: the colour wheel's arrow keys move a fixed 5°, with no PageUp/PageDown or Home/End. Fine for picking by eye; a panel that wants an exact hue has no coarse/fine step.
-
CHORE/UI: the colour wheel reads the pointer's angle only, never its distance from the centre (marked
ponytail:inColorWidget.tsx), so a colour takes two gestures — the ring, then saturation. A saturation-by-radius disc would need a second axis on a control that can announce one value. -
CHORE/DEMO:
seed_demo.pystill writes the Home dashboard as three titled sections ("Right now", "Energy and comfort", "Yield model"). The editor and the panel now read a page's sections as one grid, so those headings are no longer drawn and the first editor save collapses them into one section. Re-authorPAGESas a single section, and decide whether the three headings come back as markdown widgets or go for good. -
CHORE/UI: the widget-side segmented control in
widgets.tsx(DropdownWidget,style: "segmented") andCommon/RangePicker.tsxstill carry their own copy of the shape now inui/segmented.tsx. Pointing them there needs a size prop first — the widget one is a full-width pill with a 44px touch target, the editor one isw-fit/text-xs. -
CHORE/UI:
IconPickerinpanels.tsxhas no filter field; the grid shows all ofICONSat once. Add one when the map outgrows a popover. -
CHORE/UI: the delete-panel button in
PanelsDialoghas no confirmation, while unpairing — the strictly less destructive action — now does. -
CHORE/UI:
DashboardMosaic.blocksOfoffsets sections unconditionally, whileDashboardView.flatWidgetsskips the offset for an unarranged document. Cosmetic, in the schematic preview only; the two want to be one function once the mosaic may import the dashboard chunk. -
FEAT/API:
DashboardSummarycarries noicon, so the panels dialog cannot show which glyph each assigned dashboard draws on the rail — the icon is only visible from that dashboard's own settings panel. Setting it from the panels dialog would need a decision about publishing an icon change on its own, since that dialog has no draft concept. -
CHORE/TEST: nothing guards "a dashboard with two pages or sections survives an editor save". Verified by hand; the only runner is Playwright, so it wants a spec that PUTs a two-page document, opens the editor, drags one widget and asserts the second page came back untouched.
-
CHORE/UI: the transmit pulse and the editor's selection ring are both a 2px inset
--primaryring, so a control selected in edit mode and one mid-publish look the same. Only visible while editing. -
CHORE/UI: slider tick labels are laid out by percentage without measuring, so five labels of four characters can crowd on a tile narrower than its default four columns. A width-aware count needs a
ResizeObserver. -
FEAT/API: neither
FlowSummarynorDashboardSummarycarries a modified time, so Home's "recently modified" order is a proxy — drafts first, then version counter, then name (byRecency,Common/DashboardMosaic.tsx). The store is git-backed, so anupdated_aton both summaries would make it exact. -
PERF/UI: the home mosaic reads each dashboard's document for its footprint, capped at eight. An installation with dozens shows name-only tiles past that; a placement digest on
DashboardSummaryis the fix. -
PERF/API:
_panel_mayre-readspanels.jsonand every published dashboard document of the panel on each request a screen makes, to resolve the message allowlist. Marked# ponytail:inapi/deps.py; cache behind the dashboard store's version if it shows up in a profile. -
CHORE/TEST: the frontend has no unit-test runner (Playwright only), so pure helpers like the slider's
tickIntervalshave nowhere to be checked cheaply.color.check.tsis a worked pattern —bun runover plainnode:assert, no framework, imported by nothing so it never bundles. What is left is a make target that runs every*.check.ts. -
FEAT/UI when an installation is added to the hub, the dialog which shows the access code should disappear automatically
-
BUG/UI the "Connect" button in "Remote Access" when adding an installtion to the hub is invisible and only shows upon hovering (could also be a local browser issue)
-
CHORE/PKG the SPA is not in the wheel:
fluksio serveserves no UI, on the assumption that a pip install is paired with a portal. Bundlingdist/and mounting it withStaticFileswould give a local dashboard — it needs a hatch build hook running bun,VITE_API_URL=""for the same-origin case, and a decision about the MCPmount("/")it would collide with. -
CHORE/PKG the Docker image still starts with
fastapi run;fluksio servenow does the same thing plus the bootstrap. Switching would give the container and a pip install one code path. -
CHORE/DEPS
sentry-sdkwent to 2.x and therequires-pythoncap came off with it. Nothing exercises Python 3.13/3.14 in CI — the matrix is one version. -
BUG/UI assimilate the design of the settings in the app to mirror the design of the settings in the portal
-
CHORE/UI: pending states here are lucide's
Loader2onanimate-spin; the portal draws the brand mark instead (index/frontend/src/components/ui/fluksio-loader.tsxplus the.fluksio-loader-linerule at the foot of itsindex.css). Copying both across — the shadcn model, so the CSS travels with the component — would make one loading mark across the two shells. -
CHORE/INFRA: the
generate-frontend-sdkpre-commit hook runsscripts/generate-client.shon anybackend/**change, and that script ends by formatting the whole frontend tree — whilebiome.jsonexcludessrc/client, so it formats nothing the generator wrote. Every backend commit therefore rewrites files it never touched. Dropping the trailing format call is the fix; it was kept this wave only to preserve behaviour. -
CHORE/UI: acknowledging a node's failure clears it on the engine but publishes no event, so another browser watching the same flow keeps the marker until its next snapshot or rebuild. One event on the bus would close it, the way
dashboard_changeddoes for panels. -
CHORE/API:
save_dashboardstill catchesDashboardNotFoundfromwrite_draft, which can no longer raise it. Harmless, and the same shapesaveFlowhas: a PUT to an unknown name now creates that dashboard's first draft rather than answering 404. -
FEAT/UX add an option to the settings of an installation to configure automatic updates. If enabled, the installation would send a request e.g. every 1h to the hub at fluksio.com and the hub then checks if a new version is available. The settings should include a second toggle for automatically installing an update (which might cause a short outage). Later this mechanism should be extended to check if updating would cause things to break.
-
CHORE/UI: two of the mobile-overflow floors are over-determined. Removing
.widget-grid { min-width: 0 }, or the segmented fieldset'smin-w-0, leaves the mobile suite green — the grid tracks are alreadyminmax(0, 1fr)and the fieldset became a grid. The uPlot legend is the one offender the assertion actually catches. Both are cheap insurance for a future widget, but nothing would notice if they regressed. -
CHORE/API: artifact blobs are content-addressed and have no GC, so deleting a flow drops its
run_artifactrows and leaves the bytes on the data volume. -
CHORE/API: no run table carries a foreign key to
run.id— the baseline migration declares none forrun_node,run_metricorrun_artifact— so nothing at the database level stops a late write from orphaning rows.DELETE /flows/{name}now refuses while a run isrunningorqueued, which closes the path that was known; a constraint would need a migration and would turn_record_node's late insert into a hard error rather than the no-op it is today. -
CHORE/INFRA:
backend/htmlcovcan be left root-owned by a container run, andmake test-backendthen fails at the coverage HTML step after every test has passed — which reads like a test failure and is not one.make cleanis the fix. -
CHORE/UI: the edge popover shows the same value twice —
MessageSparklinefalls through to a collapsedValuePreviewfor a non-numeric value, andEdgeInspectorthen renders its ownValuePreview defaultOpenbelow it. Cosmetic; one of the two is redundant. -
FEAT/UI: a settings-and-inputs overview page, so what every node of an installation is configured with can be read and searched in one place rather than one panel at a time.
-
FEAT/UI: an input endpoint opens the flow panel, which is right for editing but not for reading one value. A panel of its own — the declaration, the current value, its history — is what clicking a label wants to give.
-
FEAT/UI: sync between the header of the python function and the node configuration. The config→header half exists for ports and settings —
scaffoldForwritesdef process(<ports>, <settings>)andeditNodekeeps it in step — but only while the source is still exactly the generated scaffold (SCAFFOLD_SHAPE), and never for shared code. What is missing is the same for code someone has edited, and the reverse direction: nothing parses adef process(...)header back into ports and settings. -
BUG/UI on flows like "House history" where the widget sets the range for the "draw the window" node to generate some data, the edges overlap the nodes. We should adjust the flow visualization to account for these cyclic behaviors
-
CHORE/UI: loop lag on Home reads a real number with no flows, and that is right —
LoopWatchdogtimes how lateasyncio.sleep(1.0)wakes on the API's event loop and is started unconditionally, so it measures the engine process rather than any flow, and it is what turns the health badgedegraded. Nothing to fix; recorded so it is not reopened. -
BUG/UI auto node placement on flows should be improved in regards to least crossing edges and a more vertical layout on mobile devices
-
INFRA: ensure that all the packages/ dependencies needed to run fluksio are available on arm to make this software runnable on e.g. raspbian
-
INFRA: merge the philosophy statement at the beginning of vision.md into the rest of the document. Dissolve the decision dates and fold the decisions into a clean structure
-
FEAT/UI introduce a graph panel which renders at the top right next to the graph view (to make more use of the horizontal space) and which allows (de-) selecting flows to be excluded from the graph view or search for individual nodes where only the flows containing this node should be shown (like slicing the brain)
-
FEAT/UI labels in flows (indicating dashboard widget connections) naturally can't pulse. Instead add an animation (enlightning fade) from either ltr or rtl depending if the label is in- or outbound
-
CHORE/UI:
layoutGraphtreats every node as 220 wide rather than measuring, because feeding a measurement back into the layout oscillates. A node wider than that crowds its neighbours; take the width fromnode.measuredonce it has settled if it shows. Height is no longer a guess —nodeHeight()derives it from the port count before dagre runs, so it cannot oscillate. -
FEAT/UI/MOBILE: a rank of many nodes — a connector feeding eight dashboard tiles — is thousands of pixels wide however the graph is turned, so on a phone the fit shrinks it past reading. The layout is right and the flow is simply too big for the screen; a "one rank at a time" reading mode, or wrapping a wide rank, is what would make it legible.
-
CHORE/UI: an edge's value chip sits at the bezier midpoint while the layout reserves its room at dagre's label rank. The two agree closely enough today; if chips ever pile up, take the position from the layout instead.
-
FEAT/UI (deferred until MCP lands): add a "bot" icon button to the home view (graph panel) which opens a chat window (reuse general concept of a side panel like in flows/nodes to make it a chat panel which can open on any screen (stacks below any other existing panel -> introduce stacking) to give support on errors/write code, generate dashboards etc) to explain the error(s)
-
FEAT/UI make the header (Fluksio - YEAR) and the logo in the sidebar link to the main page (fluksio.com)
-
FEAT/UI consider adding a diagram to the Home view which shows a histogram of the different classes of nodes and which time it takes to execute (logarithmic scale); this should give a hint on the load and help to detect bottle necks/hotspots
Persistence and databases
From the 2026-08 database review. Verdict recorded under Deferred: the Postgres + Redis + git-files split stays; the actionable part is durability.
- CHORE/INFRA: Redis AOF runs at
appendfsync everysec, so up to ~1 s of journaled work-queue entries can vanish on a crash — softer than "journaled before it runs" reads. Queue write volume is low, soappendfsync alwaysis likely affordable; otherwise document the loss window. - CHORE/INFRA: Redis has no auth (
requirepassunset). Fine on the compose-internal network; a blocker for M5 remote workers, which turn Redis into a network-exposed shared bus.
Connector write paths
Needs someone watching the real hardware, so it is not a background task. This is what M4 still waits on, together with porting the flows.
Art-Net can write now: ConnectorNode.write carries a node's input ports, a
per-port channels map places each on its own DMX channel, and transmit
still gates the socket. Verified on the wire against a listener (channel 33 =
255, channel 31 = 60, nothing else set) and the MQTT half was driven end to
end against the house broker. The rig was the house_control flow and its
dashboard; it has been deleted now that dmx owns the universe, and
make -C app seed-house rebuilds it if that verification is ever wanted
again.
- FEAT/NODE: Art-Net against the real fixtures is still untried. The house's own dmxnet sender re-emits universe 1 every 1000 ms, so fluksio and Node-RED overwrite each other; the test needs Node-RED's Art-Net sender stopped, and while it is stopped every channel fluksio does not set is dark.
- CHORE/NODE: the operatorId worry was unfounded — the reference Node-RED
setstatnode for this unit is configured with an empty operatorId and deviceId, so a command needs no registration. The second unit may still differ. - PERF/NODE: a
wfraccommand is two round trips (read, then set) on the scheduler's thread, so at the default timeout a command can hold a cascade for several seconds. Fine for a person pressing a button; a flow commanding it on a schedule would want the work off that thread. - CHORE/NODE:
wfracwrites carry the unit's whole state, so two flows commanding one unit will each undo whatever the other set between their read and their write. One writer per unit, the same rule Art-Net has for a universe. - FEAT/NODE: the second WF-RAC unit (the one Node-RED addresses with operatorId "0") closes the connection on an anonymous read. It likely wants an account registered; the first unit answers without one.
- CHORE/NODE:
wfracsometimes reportsmodeas "unknown" while the unit is off, because the mode bits hold a value outside the known set. Narrower than it first looked: a unit switched off by a command keeps its last mode in those bits and reads back correctly, so this is about however the remote turns it off. "off" would still read better than "unknown". - PERF/NODE:
ArtNetOut.writesends one frame per input port, so a node with two ports emits two frames per run. The last one carries both channels, so the end state is right; folding them into one send would halve the traffic. - CHORE/NODE: the Art-Net node starts from an all-zero universe and has no way to learn what the fixtures are currently at — Art-Net has no read-back. Taking over a universe therefore blanks everything the flow does not drive. A baseline setting, or driving every channel, is what a real cutover needs.
Porting the Node-RED flows
Built: scripts/tinyhouse/, make -C app seed-tinyhouse. Nineteen flows, 109
nodes, three dashboards, against the reference's 865. What each requirement
became, what was deliberately dropped, and the cutover order are in
docs/private/node-red-transition.md. What is left here is what the seeding
does not settle.
- CHORE/FLOW: the DMX channel collisions were decided rather than ported — ch 9
is one bathroom fixture, and
light/traverseAmbientLight(28-30, colliding with the window opener's 28-29) is left out on the assumption that it is decommissioned, since the live scene engine never drove it. The stray 1CH mapping on 129 beside the awning's 129-130 is dropped. Check the wiring beforedmxtransmits. - CHORE/FLOW: 1CH values are still used raw, so those fixtures still never go
above 100/255, and the 4CH master channel still takes
von a different scale from its colour channels. Both arescaleandmaster_rawsettings on the encoders now, so fixing one is a decision about one fixture rather than a surprise across all of them. - CHORE/FLOW:
artnet.baselineinhouse.jsonis empty. A frame carries the whole universe, so the first one this node sends darkens every channel it is not driving. Fill it in from what the fixtures are at, before transmitting. - FEAT/FLOW: the deferred half of the port — the media plug and the radio at
the media plug's address, the alarm clock, audio through
thgui/display/audio, and the fire alarm. The alarm's hook exists: every arbiter takes aforce_atpulse, so an alarm flow providessafety.fireand each actuator binds it with its own safe value. - CHORE/FLOW: the 433 MHz weather stations retain
status = dead. Nothing waits on them, but every humidity rule is inert until they are back. - CHORE/INFRA:
TZnow reaches the api container and is set to Europe/Berlin. The container has to be recreated once before any cron means local time. - CHORE/UI: a rollershutter is three buttons, because there is no cover widget and no position to bind one to. A cover widget with an up/stop/down control would be one tile instead of three, on four motors.
- PERF/FLOW: the MQTT node opens one connection per node, while
docs/reference/node-types.mdsays nodes sharing a broker share one. Eight nodes speak to the house broker here. Either a shared client registry or a correction to the sentence.
Bugs found while building the screens
- CHORE/API: revoking an OAuth client does not invalidate access tokens already issued; they are stateless JWTs valid up to
MCP_TOKEN_EXPIRE_MINUTES. Immediate revocation meansapp/mcp/http.pychecking the client row still exists. - CHORE/FLOW:
Pipeline.trigger's docstring says a paused flow still publishes so the value shows on the canvas. True only without a queue; with one the item parks beforeapply_outputsand nothing shows. Docstring and behaviour disagree.
Out-of-process nodes and modules
- CHORE/FLOW: a node's return value now round-trips through JSON, so tuples arrive downstream as lists and anything non-JSON is an explicit error. That is the message contract, but flows written before this may notice.
- CHORE/FLOW: a remote worker's agent spawns a fresh subprocess per call, so a node on a device pays for its imports on every call and pays them out of the node's own timeout — the cold-start problem local workers no longer have. Masked above ten seconds, because the agent's heartbeat resets the hub's silence deadline; a node with a shorter timeout and slow imports cannot run there at all. The fix is a persistent process on the agent, which is where its module cache would live.
- CHORE/FLOW:
worker_main.load_functionkeeps every source it ever compiled insys.modules, keyed by digest, and never drops one. A worker that has served many edits of one node holds all of them. Bounded in practice byrespawn_allon a module change. - CHORE/FLOW:
_remote_classcaps how many exception classes it caches but not how long a name may be, and for a remote worker that name comes off the wire. Local workers can only send a name their own code defined.
Engine history
- CHORE/FLOW: a rate-limit flush gets no run record — it is the tail of the run that scheduled it, and there is no id linking the two. A flush that fails therefore shows as a failure with no run beside it.
- CHORE/FLOW:
Pipeline.flushreleasing a held value runs its cascade without a run id, so those executions land in the minute rollups but in no run. Threading the scheduling run's id through the queue item would close it. - CHORE/FLOW:
EventBus.emitscounts a node's publishes so a reconnecting client can restore what it missed. Two deliberate shortcuts: the increment is a read-modify-write, so two threads emitting from one node can lose a count —publishis documented as never blocking, and a dropped increment is invisible in an animation — and the dict is never pruned, so a deleted flow's node ids sit there until restart. Bounded by distinct ids seen in the process, and orphans are never read, since lookups go throughbrain_graphmembers. - CHORE/API: the metrics collector is a bus subscriber, so a storm that overflows the bus queue undercounts. The events dropped are the same ones the websocket drops; exact accounting would need the collector to be fed from the engine rather than the bus.
- CHORE/UI: the Home block's "Changes" list is the newest 15 audit rows whatever range is selected. Deliberate — an audit trail is worth reading past the window — but it sits under a control that governs everything else on the screen.
- CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but
forget_flowcould offer to clear it. - CHORE/API: nothing can ask the collector to flush now, so anything needing the tables to be current has to wait out
FLUSH_INTERVAL_S— which is what the soak harness does before clearing its own rows. - CHORE/FLOW:
RedisWorkQueue.clear_flowdeletes onlypipeline:__parked__:{flow}, so a deleted or renamed flow's__queue__stream entries,__delayed__zset members and__done__:*markers stay behind. The stream is capped and the entries are dropped when they reach a node that no longer exists, so it costs work rather than correctness. - CHORE/FLOW:
MemoryWorkQueue's in-flight count is a counter around claim/ack, and claiming already removed the item — so an item a handler leaves unacknowledged (no pipeline bound) counts as in flight until the process ends. Nothing can hand it back either way, which is what the memory queue is. - CHORE/API: audit rows ride the same drop-oldest bus as telemetry, so a storm can lose one. Writing a node's source is not audited either; publishing is.
- PERF/API:
queue.stats()does a keyspacescan_iteron every call while two endpoints poll it. - CHORE/INFRA: dev only — memory-queue ids (
mem-{seq}) restart at 0 each boot andFlowRun.idis the primary key, so a restart without Redis upserts over the previous boot's run rows.
Wall-panel parity with the current home dashboard
What a fluksio dashboard still lacks to replace geli-dash (Dash/Plotly, e-ink
panel: clock and nav chrome, indoor climate, weather forecast strip, calendar
agenda, room light groups, sliders, power/battery bars, and three pages of
InfluxDB time series). Component-level only; the arrangement and the styling are
this design system's business, not that one's.
Decisions taken up front, because most items below depend on them:
-
Structured data reaches a widget as a declared shape, not as opaque JSON with a path per binding. A path would leave the picker with nothing to offer and
widgetIssueunable to judge a tile from the document alone. -
BUG/UI double check that this aligns with the new data-science pipeline feature
-
A chart asks a flow for its series the way every other input widget speaks: it publishes a request message and reads the answer. No query API, no database knowledge in the widget.
-
Database nodes are transport and credentials only. The InfluxDB node runs the Flux it is handed and echoes back every other field of the request; building the query and shaping the answer are Python nodes either side of it. That is what keeps a widget ignorant of the database, and it is also what a series read mode inside the node would have prevented. A "grouped nodes" concept could later package the standard chart→build→db→parse→chart quintet so a dashboard is not five nodes of wiring each time.
-
Nothing e-ink-specific in the widgets. Panel access is a credential problem (see below); the display's demands are a rendering profile, deferred.
-
CHORE/UI: identical in-flight chart requests are deduplicated per browser tab, so two wall panels showing the same tile still run the query twice. An
intervalon the request port is the backstop, and it belongs to the flow serving the request rather than to the widget asking. -
CHORE/FLOW: one request/answer pair per InfluxDB node — the first input carrying a
fluxkey is the request and the answer leaves on the first output port. A second query stream through one node needs a second node. -
CHORE/FLOW: porting the controls needs a declared writable message per control, since an input widget can only target what a flow declares. Declaring one is no longer API-only — the flow panel edits a flow's inputs and the canvas draws each as a label — so the dashboard-input node this asked for has largely been answered by flow inputs. What is left is the naming: an input a panel writes looks the same as one a run passes in.
Deliberately not ported: the local-state/timestamp reconciliation the old dashboard does per widget — publishing on release and reading the value back covers it — and its demo mode, since an unbound or silent message already renders as an em dash.
Dashboard follow-ups
- FEAT/NODE: the hosted demo places six of the fifteen built-in node types (
python,inject,change,join,rbe,trigger); it does cover all fifteen dashboard widget types.switchanddelayare the awkward ones — aswitchbranch needs either a dead-end port or trivial nodes to turn a branch back into a label, and neither reads as something a person would hang — while the I/O types (mqtt,http,influxdb,exec,file,ntfy,mlp) are unplaced because the demo has nothing real to talk to. Worth revisiting when the demo grows a second page. - CHORE/UI: multi-page and multi-section dashboards have no UI and need none — a panel carries
several whole dashboards instead, each with its own canvas and its own publish.
PageDef/SectionDefstay in the schema and still round-trip; the editor and the panel both read a page's sections as one grid, and section headings are no longer drawn. Recorded, not open. - CHORE/API: a panel paired through the portal is revoked here the moment the panel is deleted —
_panel_mayfinds nothing and answers 401 — but the hub's copy of the token stays valid until it expires or the installation's generation counter is bumped ("New code"). The hub has no per-panel revocation, and giving it one means telling it which panels exist, which is exactly what this design avoids. The generation bump is the lever; it is blunt, cutting every credential the portal minted for the installation. The per-panel nonce does not reach it either:pncis only checked on a token this installation signed. - CHORE/UI: the device line under a pairing code is the raw user agent plus the address the request came from. Both are self-reported and neither is proof; it is there so an admin can tell the screen they just hung from one they were not expecting, not to authenticate anything.
- CHORE/API:
POST /panels/pairis reachable from the internet once an installation is enrolled — the hub forwards it without a session, since a device with no credential is the point of it. Bounded three ways (the hub's per-installation and per-address limits, and the fifty-code cap here), but it is the first unauthenticated surface this installation exposes outward. - CHORE/UI: only
layout.lgis ever written, andmd/smstay unwritten by decision — a phone stacks the widgets (.widget-stacked) rather than carrying an arrangement of its own, since arranging is not a phone feature. The keys stay in the schema for a panel that one day wants a second size. - PERF/UI:
ChartWidget's cost per live value is theuPlot.joininUplotChart, not the tail append — the fetched half comes from React Query and is replaced wholesale on every refetch, so a ring buffer over the live tail would leave the dominant cost untouched. If this is ever profiled and fixed, thesetDataeffect must stay dependency-free: a mutable buffer's identity never changes, so keying the effect on it reintroduces the staleness that the point-count dependency used to cause, and more quietly. - CHORE/UI: opening edit mode on a dashboard whose widgets predate placement writes the migrated positions immediately, bumping the version once.
Flow editor follow-ups
- PERF/FLOW:
FlowController._teardownstops nodes one after another, each underNODE_STOP_TIMEOUT, so one flow's teardown isk * 5.0worst case for itsknodes.asyncio.gatherover the flow's nodes would bound it at five seconds flat. - CHORE/FLOW: a work item claimed just before a
replace_flow, whose node belongs to the replaced flow, still applies its outputs and publishesnode_executedfor a node that has been torn down. Cosmetic — its cascade finds no edges and stops there. The fix is a flow-keyed in-flight counter inExecutionServiceand apause_flow_intake(flow)to go with it, which is only worth building if the stale event is ever actually seen. - CHORE/FLOW:
FlowController.reload_lib_usersresolves who runs a shared node throughstore.usages, which reads drafts — so a flow whose draft references the library but whose published copy does not gets rebuilt for nothing. Harmless; it wants a published-only usage lookup rather than a second caller-side filter. - CHORE/API:
POST /flows/{name}/renameis no longer reachable from the UI. A flow's title is what the panel edits, matching how nodes work; the canonical name is fixed at creation, so either the endpoint goes or renaming comes back deliberately. - CHORE/UI: ⌘C/⌘V
preventDefaulton the canvas blocks the native clipboard there (fields are guarded). The node clipboard islocalStorage, so it does not cross browsers or profiles. - PERF/UI:
useParamSuggestionsfetches every flow's detail to build the suggestion list. An aggregate endpoint if an installation ever has many flows. - CHORE/UX: free-form params (python nodes) get no suggestions, since there is no schema to key them off.
- PERF/UI:
BrainViewruns 300 force-layout ticks synchronously inside auseMemo, so the graph is laid out on the render thread. - FEAT/UI: the brain is a band on a scrolling page now, so it neither pans nor zooms — the fit keeps the whole graph in view instead. An installation with enough flows to make the labels unreadable at that fit needs a way to open the graph larger.
- CHORE/UI: React Flow measures a node's handle bounds out of the DOM once and never again, and in the brain that one measurement falls inside the graph's
scaleInentrance — so everysourceX/targetXit hands an edge there is the entrance's 4% short of the centre, permanently.BrainEdgetakes both ends from the layout instead (position + radius). Any future view that mounts a canvas inside a transform and reads node internals meets the same thing.
Infrastructure
- CHORE/INFRA:
make soak's redis scenario stops the container the whole stack shares, so every flow briefly fails to journal, not just the soak fixtures. They recover on their own — nothing was dead-lettered or quarantined in the run this note comes from — but it is not a thing to run against a stack someone is relying on. - CHORE/INFRA: the soak harness's engine kill only catches a couple of items unacknowledged, because a cascade finishes in about four milliseconds. Redelivery is proven but barely stressed; a fixture node with a deliberate sleep would widen the window enough to test it properly.
Deferred
Open on purpose. Each names what should bring it back.
- PERF/UI: the app's entry chunk exceeds the warning threshold. React Flow and Monaco are already lazy; a manualChunks split measured no better, so this needs route-level work on the shell rather than chunking config.
- PERF/UI: the Monaco chunk is 2.6 MB. It only loads when a node panel opens, but the editor could be trimmed further or swapped for CodeMirror if that becomes a problem. NOTE: switch to codemirror; loading speed is definitely an issue.
- CHORE/API: node source saves carry no version precondition, so two clients editing the same node's code are last-writer-wins. The flow document is what the optimistic lock protects; code files would need their own, and an exact-match one produces false conflicts against a single client's own interleaved flow and source saves. Revisit with the M5 multi-user work.
- CHORE/FLOW: shared node sources bypass the draft/publish split. Editing one writes the library copy and reloads immediately, since the code is not any single flow's to hold back. Deliberate, but it means a shared node is the one thing publish does not gate.
- CHORE/INFRA:
requires-pythonis capped below 3.14 because the MCP SDK wants a newer starlette there than the pinnedsentry-sdk<2allows. Lift the cap when sentry-sdk moves to 2.x. - FEAT/UI: the node-panel and edge trend curves take no range, unlike the health block. They are drawn from a Redis ring of the last 120 values per message, which has no window to ask for — a hover caption names what the curve covers instead of a picker promising a span nothing can serve. Reopen if per-message history ever gains a time window.
- FEAT/UI: an e-ink rendering profile for a dashboard — motion off, hover-only affordances resolved to something visible, high-contrast palette, thick strokes, and a repaint cadence low enough for a display that takes a second to settle. Reopen when a panel with such a display is actually hung.
- CHORE/INFRA: Postgres stays. The 2026-08 review rejected YugabyteDB/CockroachDB (multi-node cluster systems, ~4 GB+ RAM per node, against the small-server target — the scaling story is remote workers, not a distributed DB) and found merging Postgres into Redis or vice versa buys little: the stores hold disjoint data and both sit behind abstractions. SQLite would fit the single-instance design and drop a container; reopen if the home-install footprint becomes a product concern.
- CHORE/INFRA: NATS JetStream as the work-queue backend — durable streams whose consumer semantics match the
WorkQueueinterface, in one small binary. Reopen with M5 remote workers, when the queue crosses hosts. NOTE: remote workers landed without it — a worker dials the engine's own socket and never touches Redis, so the queue still does not cross a host. Reopen if a second engine ever pulls from the same stream. - FEAT/RUNS: stage caching.
run_node.cache_keyis written on every run and the artifact store is content-addressed, so the pieces are in place; what is missing is computing the key from the node's source digest plus its input values and skipping a node whose key already has anokrow with its artifacts still present. The two research repos want this more than they want resume — neither persists checkpoints, and both re-run unchanged preprocessing every time. - FEAT/RUNS: per-label requirements overlays (
requirements-gpu.txt) synced into a remote worker's venv, with drift surfaced against the engine's manifest. Today a worker's environment is whatever--pythonpoints at, which is fine for one hand-managed GPU box and not for several.venv_digestalready arrives at attach and is shown on/workers, so the reporting half exists. - FEAT/UI: a dashboard shows a run's curve only while it is running. Emissions reach the socket live, but a run's values live in its own state namespace, so reloading the panel afterwards leaves the chart empty — the durable series is on the run (
/runs/{id}/metrics) and nothing binds a widget to it. A chart variant that reads a run's series, or the existing querying chart pointed at/runs/series/compare, is what would close it. This is also what a demo needs to show a finished experiment rather than only a live one. - FEAT/UI: launching a sweep is API-only. Pressing Run on a batch flow asks for its parameters, but the many-runs-at-once shape has no UI; a run detail screen is what it wants to land next to.
- FEAT/RUNS: a run detail screen. The API answers everything — params, per-node status with logs and tracebacks, artifacts, metrics, and
/runs/series/comparein the chart widget's ownseriesshape — but nothing in the dashboard reads it yet, so a run is inspected over HTTP. Comparing curves is a widget binding once someone builds the page around it. - FEAT/RUNS: a thin client CLI (
fluksio run/runs/sweep/worker) over the same API. The engine being resident is what makes runs cheap; a CLI is ergonomics on top, andcurlcovers it until someone is running sweeps daily. - FEAT/RUNS: the step on a run's series is the count of emissions on that message, so a node yielding every tenth training step records steps 0, 1, 2 rather than 0, 10, 20 — a faithful x-axis of its own emissions, not of the loop inside it. If a real step number ever matters, a
record-typed streaming port carrying its ownstepis the shape to read it from; the column is already there. - CHORE/RUNS: an emission publishes on the node's port and, in a live flow, enqueues a cascade with no payload of its own — the value is already in state, and an item carrying it would re-apply that value whenever it was claimed, which is how a mid-node emission overwrites the one the node returned at the end. Downstream therefore reads what is current rather than the value that caused it to run. Right for a curve; worth revisiting if something ever needs every intermediate value delivered rather than sampled.
- CHORE/RUNS:
run_metrichas no retention. Deliberately outsideOBS_RETENTION_DAYS— an experiment nobody deleted should not vanish on a rollup window — but a few thousand runs at 3000 steps will want a policy eventually, probably per-flow rather than global. - CHORE/RUNS: a run holds one worker slot per node for its whole duration, and
MAX_PARALLELrun drivers bound how many graphs are in flight. A sweep of 500 therefore queues behind the pool rather than the driver count. Fine — the GPU is the scarce thing — but the two limits are unrelated numbers that read as if they were one.
Blocked
-
CHORE/INFRA:
bun installinside the frontend Docker build intermittently fails with "Fail extracting tarball" for several packages at once, and succeeds on a plain rebuild. It looks like concurrent extraction under memory pressure. Pin down or retry in the Dockerfile if it starts costing CI time. NOTE: memory lifted; retry and close if stale -
CHORE/DOCS:
docs/(the published site) documentsdeviceanddevice_policyas API-only, because the node panel has no field for either. That is the one place the public docs have to say "use the API instead of the UI". Adding a Device section toNodePanel.tsx— a label field plus a require/prefer toggle — would close it. -
CHORE/DOCS: the site's node-type reference is hand-written from
NODE_TYPESand each node'sParams. It will drift.GET /flows/node-typesalready returns the whole thing with its schemas, so a generator (the way n3xd generates its command catalog) is the obvious fix once the type list stops moving. -
CHORE:
frontend/vite.config.jsandfrontend/vite.config.d.tsare committed build output ofvite.config.tsand failbiome checkas tracked (semicolons, 4-space indent). The pre-commit hook therefore fails for anyone who stages any other file underfrontend/. Either gitignore the two or run the formatter over them once. -
CHORE: the pre-commit biome hook selects files by the
frontend/prefix rather than by extension, so a non-JS file placed there is handed to biome and blows up with an internal error on a doubled path (frontend/frontend/...).