diff --git a/NOTEPAD.md b/NOTEPAD.md index 5119be7..a847704 100644 --- a/NOTEPAD.md +++ b/NOTEPAD.md @@ -13,11 +13,23 @@ Deferring because out of scope is fine, but don't mention deferring than. ### Dashboard UI rework (dedicated session) Rework the dashboard UI to make it more flexible in terms of the look and feel. -I consider following design options a) overall appearance: material look or liquid glass (einUI https://github.com/einui/einui) b) color palette (https://plotly.com/python/discrete-color/). +The dasboard is allowed to break out of the design guidelines. +I consider following design options a) overall appearance: material look or liquid glass (einUI https://github.com/einui/einui) b) color palette (https://colorhunt.co). Based on a) and b) it should be possible to derive almost endles combinations of looks. -The color palette should affect charts/gauges and other widgets - +The color palette should affect affect the overall theming of all widgets +The EinUI components should be used as a reference for the different components needed, the general layout/style of components and essentially the foundation for the liquid glass option. +Based on this, a set of material ui components should be derived, following https://mui.com/ . +Both variants of the components should be completely custom designed and fully owned (no new dependency). +The apperance should be strictly separated from the functionality (which both sets of components should share; i.e. choosing one or the other component set should not change the features of the dashboard). +All widgets (and the sidebar rail) should follow the appearance and should be reworked to receive animations through the motion library. +It should be possible to set a background in the dashboard (input, controllable externally via a node. In v1 this can be just an url to an image). +It should be possible to disable the title of a widget. +The dashboard setting should contain a "Touch" toggle, which, when activated, makes all widgets more touch friendly +Specific changes +- the nested bar should be changed in a multi-row bar with N inputs (and therefore n bars), color separated as the chart widget +- the range picker should go to the right (vertical) of the chart widgeta +- the color picker should be changed to a circular color picker (disk with colors, saturation changes towards the center) and a vertical slider for the brightness ### To be sorted @@ -60,25 +72,20 @@ The color palette should affect charts/gauges and other widgets reloader, which reaps nothing that is not its own child. `init: true` on the backend service hands PID 1 to an init that does reap orphans, and would close the whole class rather than one source of it. -- PERF/API: seeding nineteen flows takes two reloads each — one to publish, one to - stop — so a full rebuild of every flow in the installation runs about forty times - for one seed. It is the slowest thing about standing an installation up, and a - `PUT` that could say "published, stopped" in one call would halve it. +- BUG/NODE: **every python node on the dev stack fails to build** with + `RemoteError: the worker answered request None while was outstanding` — the reply the + pool read off the pipe parsed as JSON but carried neither an `id` nor an `event`, so + `_exchange` retires the worker and the node goes red. It is not the rebuild scope: a + whole-pipeline `reload()` produces the same error on the same nodes. Worth checking against + the orphaned workers above, since both follow uvicorn `--reload`. It is why + `runtime.spec.ts` "the logs panel shows what a node printed" fails — nothing runs, so + nothing prints. - CHORE/UI: the portal's aggregate tiles have no not-reporting state. A figure an installation never sends renders through `?? 0`, so a missing field reads as a confident zero rather than as unknown — which is how a deleted `failures_24h` showed as "0 failures" unnoticed. The per-card half is one line; Home sums across installations and has nowhere to say "3 of 4 reporting", so the two want doing together. -- PERF/API: enabling or disabling one flow calls `FlowController.reload()`, which tears down - and rebuilds *every* flow — reconnecting each node, including the ones that talk to hardware - over the network. On the tinyhouse installation (nineteen flows, MQTT + UniFi + aircon) a - single toggle takes 8-10s end to end. Same root as the seeding cost below: there is no way - to change one flow's runtime state without rebuilding the whole pipeline. -- CHORE/TEST: `runtime.spec.ts` "the home page lists flows and can stop one" asserts the row - reads "Stopped" within Playwright's 5s default. That is shorter than a real installation's - rebuild (above), so the spec passes on a small instance and fails on a populated one. It is - the rebuild that wants fixing, not the timeout — raising it would only hide the cost. - FEAT/SEC: `locked` is a read-only surface, not a permission — the server accepts a publish from a panel whose dashboard says locked. Making it real means carrying the flag into `_panel_may`. @@ -291,7 +298,6 @@ does not settle. ### Out-of-process nodes and modules -- FEAT/API: `POST /modules/apply` rebuilds the whole pipeline so a node that could not import its package stops being red. That resubscribes every MQTT node in the deployment; a targeted rebuild of the flows that actually failed to load would be gentler. - CHORE/FLOW: a node's return value now round-trips through JSON, so tuples arrive downstream as lists and anything non-JSON is an explicit error. That is the message contract, but flows written before this may notice. - CHORE/FLOW: a remote worker's agent spawns a fresh subprocess per call, so a node on a device pays for its imports on *every* call and pays them out of the node's own timeout — the cold-start problem local workers no longer have. Masked above ten seconds, because the agent's heartbeat resets the hub's silence deadline; a node with a shorter timeout and slow imports cannot run there at all. The fix is a persistent process on the agent, which is where its module cache would live. - CHORE/FLOW: `worker_main.load_function` keeps every source it ever compiled in `sys.modules`, keyed by digest, and never drops one. A worker that has served many edits of one node holds all of them. Bounded in practice by `respawn_all` on a module change. @@ -374,7 +380,9 @@ as an em dash. ### Flow editor follow-ups -- PERF/FLOW: every save rebuilds the whole pipeline. Fine at the current flow count; rebuild only the touched flow when it starts to show. +- PERF/FLOW: `FlowController._teardown` stops nodes one after another, each under `NODE_STOP_TIMEOUT`, so one flow's teardown is `k * 5.0` worst case for its `k` nodes. `asyncio.gather` over the flow's nodes would bound it at five seconds flat. +- CHORE/FLOW: a work item claimed just before a `replace_flow`, whose node belongs to the replaced flow, still applies its outputs and publishes `node_executed` for a node that has been torn down. Cosmetic — its cascade finds no edges and stops there. The fix is a flow-keyed in-flight counter in `ExecutionService` and a `pause_flow_intake(flow)` to go with it, which is only worth building if the stale event is ever actually seen. +- CHORE/FLOW: `FlowController.reload_lib_users` resolves who runs a shared node through `store.usages`, which reads *drafts* — so a flow whose draft references the library but whose published copy does not gets rebuilt for nothing. Harmless; it wants a published-only usage lookup rather than a second caller-side filter. - CHORE/API: `POST /flows/{name}/rename` is no longer reachable from the UI. A flow's title is what the panel edits, matching how nodes work; the canonical name is fixed at creation, so either the endpoint goes or renaming comes back deliberately. - CHORE/UI: ⌘C/⌘V `preventDefault` on the canvas blocks the native clipboard there (fields are guarded). The node clipboard is `localStorage`, so it does not cross browsers or profiles. - PERF/UI: `useParamSuggestions` fetches every flow's detail to build the suggestion list. An aggregate endpoint if an installation ever has many flows. diff --git a/backend/fluksio/flow/controller.py b/backend/fluksio/flow/controller.py index ab3ba68..11a5630 100644 --- a/backend/fluksio/flow/controller.py +++ b/backend/fluksio/flow/controller.py @@ -1,9 +1,15 @@ """Turns stored flows into a running pipeline. The controller is the only thing that builds nodes: it reads flow definitions -from the store, instantiates each node from its type, and rebuilds the shared -pipeline. A node that fails to load is reported and skipped — one broken node -never stops the rest. +from the store, instantiates each node from its type, and puts them in the +shared pipeline. A node that fails to load is reported and skipped — one +broken node never stops the rest. + +Two scopes: `reload` builds every flow, and `reload_flow` swaps one flow's +nodes into the graph and leaves every other flow connected. Deploying takes +the second — the cost of a rebuild is each node opening its socket again, and +one flow's are all that changed. Starting one is neither: a stopped flow's +nodes are built like any other's, so `set_enabled` only starts or stops them. """ from __future__ import annotations @@ -521,9 +527,9 @@ class FlowController: flow.name: _declared_inputs(flow)[0] for flow in published } - # A rebuild is a fresh set of nodes, but not a fresh history: every - # publish rebuilds every flow, so dropping the failures here would - # wipe them constantly. They are the operator's to dismiss. + # A rebuild is a fresh set of nodes, but not a fresh history: + # dropping the failures here would wipe them on every rebuild, + # and they are the operator's to dismiss. for node_id, entry in loaded.items(): previous = self.loaded.get(node_id) if previous is not None and previous.last_error: diff --git a/backend/fluksio/flow/pipeline.py b/backend/fluksio/flow/pipeline.py index b72d442..bc0fd81 100644 --- a/backend/fluksio/flow/pipeline.py +++ b/backend/fluksio/flow/pipeline.py @@ -4,6 +4,11 @@ One pipeline holds the nodes of every loaded flow. Edges are not declared — they follow from message names, so a node consuming ``heating.setpoint`` is downstream of every node providing it. Several producers of one message are allowed: each publication triggers the consumers, and the latest value wins. + +Because the wiring is derived rather than declared, one flow's nodes can be +swapped in place: `replace_flow` splices them into the list and derives the +whole map again, which gets the edges crossing into other flows right by +construction. A deploy does that rather than building a second pipeline. """ from __future__ import annotations