Commit Graph
819 Commits
Author SHA1 Message Date
stroblme b7f1d21712 Docs for key renames: ids and keys in the flow concepts, the rename routes, events and 503s in the API reference, hooks during a rename, sync and run metrics after one; a pinned restore refused for its recorded key names that key in the 409
Docs / docs (push) Successful in 3m11s
Playwright Tests / test-playwright (1, 2) (push) Successful in 11m42s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m22s
pre-commit / pre-commit (push) Successful in 3m12s
Test Backend / test-backend (3.12) (push) Successful in 6m19s
Test Backend / test-backend (3.14) (push) Successful in 5m59s
Compose Smoke Test / test-compose (push) Successful in 37s
Playwright Tests / merge-reports (push) Successful in 1m12s
2026-11-03 08:09:35 +01:00
stroblme f9d5f510c2 Startup deletes the work queue's keys from before it moved from pipeline: to queue: (stream, timers, dead letters, parked lists, done markers), logged with what they held; an old stream still holding undelivered work stays, is logged as an error, and counts as queued work for the keys it names 2026-11-03 07:55:14 +01:00
stroblme 3ae5255617 Rename UI: superuser Key row and rename dialog for flows and dashboards (live plan with rewritten documents, counts, refusals and warnings, typed confirmation, what the drain waits for and calling it off); applyRename and the socket's renamed remap the live store and refetch at unchanged URLs, editors whose document moved reload so neither undo nor a queued save restores old bindings, a reconnect checks again and its snapshot replaces values; a retry toast for the rename's 503s, autosaves send themselves again; dashboard saves are based on their own versions only; Playwright rename spec, liveStore check, editor and dashboard docs 2026-11-02 22:58:41 +01:00
stroblme ba4aae390d Rename orchestration, splice and routes: a Renamer runs one rename at a time, closing the gate, taking the controller lock and holding the engine, each waiting without a limit and saying what it waits for (summary rename status, rename_waiting); DELETE on the pending request or a closed connection calls it off until the journal begins (409 aborted); then journal, steps and audit, retried in the background on failure (503 completing); splice_renamed rebuilds the affected flows under the held lock keeping stopped, paused and parked work, history limits and device wiring refreshed before activation; AlertManager.rename and EventBus.rename/settle follow the names; flow and dashboard rename routes answer by code and repeat stored results; renamed, dashboard_changed and pipeline_rebuilt after release; fluksio serve reads flow files without building a FlowStore 2026-11-02 22:14:44 +01:00
stroblme 2d65118d82 Rename journal, steps and recovery: rename.json (RENAME_JOURNAL_FILE) records each step, documents are written whole and committed alone (FlowStore.replace_documents, identity.write_documents) and rolled back before the journal goes forward; history per run scoped by flow id, alert ledger, alerts.json, state and queue steps idempotent and checkpointed; audit row once per op; recover_documents before assign_ids and recover_runtime before the engine is built, retried until done; repeated requests get the stored result, stale ones 409; plan refuses rename_in_progress; alerts.json written atomically; completeness scanner in the test utilities 2026-11-02 21:30:01 +01:00
stroblme fa2a213631 Rename planner and document rewrite: rewrite_documents moves qualified names of the renamed flow in every flow (ports, inputs, outputs) and dashboard binding (one key table, WIDGET_BINDINGS, shared with the WidgetDef accessors, legacy pages included), published and draft, as plain JSON with both variants above either version; plan_rename reports documents, counts, fingerprint, warnings and refusals; superuser plan routes for flows and dashboards; key residue classifies state as the mapper does; duplicate retargets inputs and outputs too; queue reports its Redis version and other consumers 2026-11-02 21:06:39 +01:00
stroblme 81bd7c2724 Admission gate for a rename: RenameGate in flow/rename.py, installed by main.AdmissionGate inside CORS, turns mutating API requests away with 503 renaming and Retry-After while closed and drains those admitted without a limit, abortable; affected flows' hooks get 503 or socket close 1013, reads, sign-in, assistant, workers, artifacts and rename routes pass; the run cache misses while the gate is closed; trigger_node lets EngineHeld through instead of reporting a node failure 2026-11-02 20:53:08 +01:00
stroblme cb709c19ac Merge branch 'rename-s2' 2026-11-02 20:32:36 +01:00
stroblme badcf94491 Engine holds for a rename: the pipeline refuses publish and inline cascades with EngineHeld once those running have finished, the execution service stops claiming and its timers once claimed cascades have finished; both wait without a limit, report what is busy and can be aborted. REST answers 503 with Retry-After, a device frame busy. Redis adds no longer wait for each other, only a queue hold waits for them 2026-11-02 20:21:17 +01:00
stroblme be18a94089 Merge branch 'rename-s2' 2026-11-02 19:58:45 +01:00
stroblme aa95245456 Merge branch 'ids-clients' 2026-11-02 19:58:45 +01:00
stroblme fa15522cb0 Frontend addresses flows and dashboards by id: routes /flows/$flowId, /dashboards/$id and /view/$id, query keys, saves, lists, device pickers, search, brain and endpoint links and generated results dashboards go by id, while live values, issues, pauses and message names keep the key; tabs take the title from the document; creating uses POST and opens the issued id; the flow and dashboard panels show key and id with a copy button and a served HTTP or WebSocket node shows its hook URL under the flow id; specs create by POST and ids.spec.ts exercises issued ids 2026-11-02 19:57:43 +01:00
stroblme 46b57de4ec Rename migrations for live state and the work queue: state keys move with RENAMENX in batches (type, TTL and history capacity kept), the queue rebuilds its streams with the same ids, group and pending entries, rewrites timers, parked and held items and done markers, and can hold new work in a list released in order; nothing calls them yet 2026-11-02 19:50:54 +01:00
stroblme 9eb0517403 Node-types doc names the training route by flow id 2026-11-02 19:39:50 +01:00
stroblme 53916d869b Clients take a flow's id or key: one resolver in the SDK, id first, explicit id:/key:, a warning for reads and a refusal for changes when a name means two flows, deleted flows found by their last key; sync creates by POST, retries a create race through the drift checks and reports the id; MCP gets create_flow and resolves every flow argument with the caller's token; displays show flow_name; an engine from before ids is named as such 2026-11-02 19:39:14 +01:00
stroblme ebcc480a8c Rename mapper: flow/namespace.py maps state keys, work items and alert dedup keys from an old flow key to a new one; the state key families and WorkItem fields are registered there, the key builders take their prefixes from it, and done markers read back with queue.done_marker 2026-11-02 19:28:25 +01:00
stroblme fd637e0f1e Tests clear flow tombstones between modules with the other engine rows, and the tombstone test commits its prune before checking the count, so a wrong count no longer holds the write lock over the tests after it 2026-11-02 19:14:59 +01:00
stroblme d4a1b1db47 History is kept by flow id: runs, cascades, rollups and events file the id in their flow column and nodes as <flow id>.<node>, the API answers flow plus flow_name and shows and takes nodes as <key>.<node>; events carry flow_id from the moment they are published; run queue items, the stage cache and cached curves keep ids and keys apart; endpoints, the brain graph, device bindings and search name the id to open 2026-11-02 19:01:19 +01:00
stroblme e6027af8fe Prove the rename's stream rebuild on a real Redis: same ids, group, lag and pending entries after RENAME, ENTRIESREAD set after XSETID because Redis caps it; backend CI gets a Redis 8.2 service 2026-11-02 18:51:14 +01:00
stroblme 08a6376836 Flows and dashboards are created by POST under an issued or superuser-pinned id; PUT only updates; deleted flows leave a tombstone with their key; the controller stores by id and runs by key, hooks live under /hooks/<flow id>; a key with state or queued work behind it is refused until cleared 2026-11-02 17:41:23 +01:00
stroblme fae85f5ecb Flows and dashboards get ids: the store is addressed by id with a key index, the first start gives every document its key as id, fluksio store migrate --dry-run and ls; the rename routes are gone 2026-11-02 16:54:11 +01:00
stroblme de13c48c90 CI stacks come up on the runner again: mosquitto reads its config from /etc/mosquitto, where Ubuntu's AppArmor profile lets it, and Playwright reaches the app on localhost, a secure context for the clipboard spec
Docs / docs (push) Successful in 20s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m1s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m1s
pre-commit / pre-commit (push) Successful in 3m1s
Test Backend / test-backend (3.12) (push) Successful in 4m30s
Test Backend / test-backend (3.14) (push) Successful in 4m18s
Compose Smoke Test / test-compose (push) Successful in 28s
Playwright Tests / merge-reports (push) Successful in 1m17s
2026-11-02 07:22:32 +01:00
stroblme c675d2041c Ring face's unit sized inline relative to the reading, so pre-commit's arbitrary font size check passes 2026-11-02 07:03:39 +01:00
stroblme 66588c58aa Merge the v0.2.0 worker range fix
Docs / docs (push) Successful in 20s
Playwright Tests / test-playwright (1, 2) (push) Failing after 29s
Playwright Tests / test-playwright (2, 2) (push) Failing after 26s
pre-commit / pre-commit (push) Failing after 2m14s
Test Backend / test-backend (3.12) (push) Successful in 4m44s
Test Backend / test-backend (3.14) (push) Successful in 4m30s
Compose Smoke Test / test-compose (push) Failing after 22s
Playwright Tests / merge-reports (push) Failing after 2m19s
2026-11-01 13:47:19 +01:00
stroblme 7919cf5670 fluksio requires fluksio-worker 0.2.x: the 0.1 range left the 0.2.0 wheels unresolvable, and publish stopped on it
Publish / publish (push) Successful in 47s
v0.2.0
2026-11-01 13:47:01 +01:00
stroblme 78e1011089 Back to development
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m7s
Playwright Tests / test-playwright (2, 2) (push) Failing after 21s
pre-commit / pre-commit (push) Failing after 2m17s
Test Backend / test-backend (3.12) (push) Successful in 4m36s
Test Backend / test-backend (3.14) (push) Successful in 4m23s
Compose Smoke Test / test-compose (push) Failing after 26s
Playwright Tests / merge-reports (push) Failing after 2m20s
2026-11-01 13:21:13 +01:00
stroblme 5e3a2f9a62 Release v0.2.0
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m9s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
Playwright Tests / merge-reports (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 41s
Test Backend / test-backend (3.12) (push) Canceled after 0s
Test Backend / test-backend (3.14) (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Publish / publish (push) Failing after 24s
2026-11-01 13:18:37 +01:00
stroblme 64ef7c84d6 Portal members: a request that loses the adoption race reads back the winner's user instead of being refused
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m16s
Playwright Tests / test-playwright (2, 2) (push) Failing after 19s
pre-commit / pre-commit (push) Failing after 2m17s
Test Backend / test-backend (3.12) (push) Successful in 4m51s
Test Backend / test-backend (3.14) (push) Successful in 4m32s
Compose Smoke Test / test-compose (push) Failing after 26s
Playwright Tests / merge-reports (push) Failing after 2m17s
2026-11-01 11:21:42 +01:00
stroblme df1071ce73 Message suggestions stay open when an empty field is clicked: the field's own focus no longer counts as outside the list 2026-11-01 11:17:18 +01:00
stroblme e6614ff378 Image builds with uv 0.13.6; follow-redirects 1.16.2 and seroval 1.6.12; getting started covers Docker on Raspberry Pi OS and Compose 2.24
Docs / docs (push) Successful in 24s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m28s
Playwright Tests / test-playwright (2, 2) (push) Failing after 21s
pre-commit / pre-commit (push) Failing after 2m17s
Test Backend / test-backend (3.12) (push) Successful in 4m44s
Test Backend / test-backend (3.14) (push) Successful in 4m34s
Compose Smoke Test / test-compose (push) Failing after 25s
Playwright Tests / merge-reports (push) Failing after 2m21s
2026-11-01 10:59:51 +01:00
stroblme d8eb6968f0 test_wren: load numpy itself, so the BLAS test passes alone 2026-11-01 10:59:45 +01:00
stroblme b41684a608 UI: 409 toasts read the message, chat sheet named per tab, 44px phone sidebar trigger, button grids shade as controls, unambiguous chart label key, optional settings typed and cleared to null 2026-11-01 10:59:43 +01:00
stroblme 7f40ea62de Duplicating a copy names it after the original; a failed duplicate or delete says why 2026-11-01 10:59:36 +01:00
stroblme 8f4c7d8ff1 Worker: wait for a node process's exit status before telling a signal from a failure 2026-11-01 10:59:35 +01:00
stroblme 7a7315043d Alerts: a delivery no channel took leaves the cooldown unspent 2026-11-01 10:59:28 +01:00
stroblme 83c5f2dfcf A run that starts clears its 'Waiting for a worker' reason 2026-11-01 10:59:21 +01:00
stroblme 48d4be6093 Starting a running flow restarts its nodes, so interval and cron Injects keep firing; a params-dict node is told settings arrive by name 2026-11-01 10:59:14 +01:00
Melvin Strobl e0f5ddc3a4 Devices may publish to their own out ports' messages (a level set on the device itself)
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 36s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m13s
Test Backend / test-backend (3.12) (push) Successful in 4m32s
Test Backend / test-backend (3.14) (push) Successful in 4m18s
Compose Smoke Test / test-compose (push) Failing after 26s
Playwright Tests / merge-reports (push) Failing after 2m16s
2026-10-29 21:58:38 +01:00
Melvin Strobl d2e8c01785 Getting started: own settings go in compose.override.yml, uv waits 120 s for a download 2026-10-29 21:35:50 +01:00
stroblme 75b3a93094 Fix work recovery and reduce engine and chart overhead
Docs / docs (push) Successful in 33s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m37s
Playwright Tests / test-playwright (2, 2) (push) Failing after 27s
pre-commit / pre-commit (push) Failing after 2m37s
Test Backend / test-backend (3.12) (push) Successful in 5m7s
Test Backend / test-backend (3.14) (push) Successful in 5m2s
Compose Smoke Test / test-compose (push) Failing after 28s
Playwright Tests / merge-reports (push) Failing after 2m29s
2026-10-29 14:24:17 +01:00
stroblme 20a0853f69 Entry chunk: signup and the form schemas leave it, build runs without warnings
Docs / docs (push) Successful in 20s
pre-commit / pre-commit (push) Failing after 2m16s
Test Backend / test-backend (3.12) (push) Successful in 4m49s
Test Backend / test-backend (3.14) (push) Successful in 4m33s
The entry is 798 kB (259 kB gzipped) instead of 927 kB (298 kB). Signup's
`export default` kept it unsplit, and login, recover and reset-password left a
bare `z.object(...)` statement in their route stubs (reset-password also read
its search through Zod), which held all of Zod and react-hook-form in the entry.
The schemas are factories now, reset-password validates its search by hand.

The shell routes keep their component in the entry on purpose
(`codeSplitGroupings: []`, no `export default`): splitting them out measured
about 60 chunks per signed-in screen and a slower flow list. The Zod
`@__PURE__` comment warning is filtered and the chunk limit is 850 kB, each with
a comment saying why.

At 4x CPU on slow 4G the signed-in screens load 5 to 9% sooner, view 6%; login
13% later (the form chunk is one more hop). On a LAN they are within noise.
2026-10-28 18:02:06 +01:00
stroblme 87e750c776 Code editor: CodeMirror 6 replaces Monaco
Node source (Python) and the JSON dialog share one CodeMirror editor with
find and replace, autocomplete, folding, indent guides, comment toggle and a
JSON lint, themed from the CSS tokens so light and dark follow the .dark class
without a second palette. Programmatic value changes no longer fire onChange,
blur ignores focus moving into the find bar, readOnly keeps selection and copy.

The editor chunk is 470 kB (157 kB gzipped) against 2.7 MB (704 kB) for Monaco,
which also pulled in a 1.2 MB JSON mode, two workers and a font. At 4x CPU on
slow 4G a node panel is ready in 5.7 s instead of 10.8 s, the JSON dialog in
1.5 s instead of 6.7 s. Typing without a key delay keeps every character.

monaco-editor, @monaco-editor/react and the dompurify override are removed.
Specs: typed code reaches the node source, a JSON setting saves only text that
parses. Docs name Tab and Ctrl-M.
2026-10-28 17:32:49 +01:00
stroblme 5d0ea480bf Alerts: a Telegram channel, one chat or group each, connected by a one-time link
The channel sends through the Bot API with httpx only: HTML with escaped text,
cut to fit a message, silent for a warning, a short 429 waited out once, and
errors that say what to do about a blocked bot, a lost chat or a bad token and
never quote the request URL. The bot token is a stored secret: saving a
channel refuses a literal one, the card's "Store token" writes the secret and
a reference, and no endpoint returns it.

A chat id is never typed. Connect starts a ten minute code, and the chat that
sends `/start <code>` back through the bot is the one connected, so it is a
chat somebody opened a link for. The engine fetches updates only while a code
is waiting (no standing poll, no webhook, no inbound address) and ignores
everything else without a reply. Connect needs a saved form, and the form
takes the connected chat in without touching what was typed since.

The token is part of every Bot API URL, so the httpx logger gets the same
filter the request lines have, which now also masks an httpx.URL argument.
2026-10-28 13:06:58 +01:00
stroblme bfaaa7cd03 Alerts: cooldowns survive a restart, and a connection must stay down a minute before it alerts
The alert manager kept its cooldowns and its ceiling per hour in memory, so a
node that stayed down was announced again by every restart. Each delivered
alert is now a row in the new alert_sent table (additive migration), loaded
for the last seven days when the manager starts and pruned as new ones come
in. A ledger that fails never stops an alert. The manager's clock is wall
time for the same reason.

A node_health "down" alert also waits 60 s (HEALTH_GRACE_S) in a pending task
per node; a later event that is not "down" cancels it, as does stopping the
manager, so a broker restarting with the stack alerts nobody.

The cooldown used to be stamped before the ceiling per hour was checked, so
an alert the ceiling held back still spent its cooldown. It is stamped only
once the alert is sent now.
2026-10-28 12:46:42 +01:00
stroblme 00842fa175 Health summary: a node is failing after two failures in a row; one is a warning, a good run ends both
The summary read degraded for 15 minutes after any single node_error, even
once the node had run well again and including one bad input from a visitor.
The controller now counts a node's failures since it last ran well and notes
when it recovered; both carry across rebuilds and an acknowledgement resets
the count. Two in a row inside the 15 minute window is "failing" (a problem),
one is listed under warnings, and nodes.failing keeps meaning "failing now".
2026-10-28 12:42:35 +01:00
stroblme 2dc20911a3 Alerts: configuration, test and search entries are a superuser's; subscribing a browser stays open
Any signed-in account could save an ntfy channel with its own `server` and a
`{"$secret": ...}` token and press Test, which sent the stored secret to that
host. The config GET/PUT, the test route and the global search's alert entries
now answer to superusers, like /secrets. The webpush key and subscribe routes
stay open to every account, and the Alerts page shows a non-superuser only the
button for its own browser. The events table in the docs gains run_finished.
2026-10-28 12:40:44 +01:00
stroblme 29f4d93c2a Search: node settings are indexed, with credentials masked on the server
/search gains a `setting` category, `<node>.<setting>` under its flow with the
node type and the value in a new `detail` field, so Ctrl-K finds which node
uses a broker or a timeout and opens the node. `setting_text` is the one place
that decides what may be said: a reference to a stored secret shows its name
(to superusers only, as the secrets list does), a plain value in an `x-secret`
field or under a key like password, token or api_key is masked and never
indexed, nested credentials in headers or query are masked, long values are
clipped. The global search ranks what contains the query as typed ahead of
loose spellings before it caps a group, which long setting values would
otherwise have crowded out.
2026-10-28 11:07:27 +01:00
stroblme 2e0ba32124 Node panel: ports read message first on inputs, code name is text until it differs
An input row now starts from its message and ends as the argument the code
takes it as ("as prompt"); an output keeps its key on top and the message
below. The name the code uses is muted text with a pencil while it follows the
message's last segment, and a field only when it differs. Device Follows ports
get the same order (message, then the port driven). A list port's item type
moves to the second row so it no longer squeezes the name, two ports deriving
one name get a warning, and the rows rebuilt use 44px touch targets on phones.
The stored node spec is unchanged.
2026-10-28 11:04:15 +01:00
stroblme c04e4883c2 MCP: mcp 2.3 (MCPServer, transport arguments on streamable_http_app, bearer read in middleware); tests for the 2026-07-28 revision and the resource check 2026-10-28 09:21:01 +01:00
stroblme c04d6b83c6 MCP tests: the SDK's own client connects over streamable HTTP with a bearer, and older protocol revisions still answer 2026-10-28 09:11:58 +01:00