Commit Graph
421 Commits
Author SHA1 Message Date
stroblme cb709c19ac Merge branch 'rename-s2' 2026-11-02 20:32:36 +01:00
stroblme badcf94491 Engine holds for a rename: the pipeline refuses publish and inline cascades with EngineHeld once those running have finished, the execution service stops claiming and its timers once claimed cascades have finished; both wait without a limit, report what is busy and can be aborted. REST answers 503 with Retry-After, a device frame busy. Redis adds no longer wait for each other, only a queue hold waits for them 2026-11-02 20:21:17 +01:00
stroblme be18a94089 Merge branch 'rename-s2' 2026-11-02 19:58:45 +01:00
stroblme 46b57de4ec Rename migrations for live state and the work queue: state keys move with RENAMENX in batches (type, TTL and history capacity kept), the queue rebuilds its streams with the same ids, group and pending entries, rewrites timers, parked and held items and done markers, and can hold new work in a list released in order; nothing calls them yet 2026-11-02 19:50:54 +01:00
stroblme 53916d869b Clients take a flow's id or key: one resolver in the SDK, id first, explicit id:/key:, a warning for reads and a refusal for changes when a name means two flows, deleted flows found by their last key; sync creates by POST, retries a create race through the drift checks and reports the id; MCP gets create_flow and resolves every flow argument with the caller's token; displays show flow_name; an engine from before ids is named as such 2026-11-02 19:39:14 +01:00
stroblme ebcc480a8c Rename mapper: flow/namespace.py maps state keys, work items and alert dedup keys from an old flow key to a new one; the state key families and WorkItem fields are registered there, the key builders take their prefixes from it, and done markers read back with queue.done_marker 2026-11-02 19:28:25 +01:00
stroblme fd637e0f1e Tests clear flow tombstones between modules with the other engine rows, and the tombstone test commits its prune before checking the count, so a wrong count no longer holds the write lock over the tests after it 2026-11-02 19:14:59 +01:00
stroblme d4a1b1db47 History is kept by flow id: runs, cascades, rollups and events file the id in their flow column and nodes as <flow id>.<node>, the API answers flow plus flow_name and shows and takes nodes as <key>.<node>; events carry flow_id from the moment they are published; run queue items, the stage cache and cached curves keep ids and keys apart; endpoints, the brain graph, device bindings and search name the id to open 2026-11-02 19:01:19 +01:00
stroblme e6027af8fe Prove the rename's stream rebuild on a real Redis: same ids, group, lag and pending entries after RENAME, ENTRIESREAD set after XSETID because Redis caps it; backend CI gets a Redis 8.2 service 2026-11-02 18:51:14 +01:00
stroblme 08a6376836 Flows and dashboards are created by POST under an issued or superuser-pinned id; PUT only updates; deleted flows leave a tombstone with their key; the controller stores by id and runs by key, hooks live under /hooks/<flow id>; a key with state or queued work behind it is refused until cleared 2026-11-02 17:41:23 +01:00
stroblme fae85f5ecb Flows and dashboards get ids: the store is addressed by id with a key index, the first start gives every document its key as id, fluksio store migrate --dry-run and ls; the rename routes are gone 2026-11-02 16:54:11 +01:00
stroblme 66588c58aa Merge the v0.2.0 worker range fix
Docs / docs (push) Successful in 20s
Playwright Tests / test-playwright (1, 2) (push) Failing after 29s
Playwright Tests / test-playwright (2, 2) (push) Failing after 26s
pre-commit / pre-commit (push) Failing after 2m14s
Test Backend / test-backend (3.12) (push) Successful in 4m44s
Test Backend / test-backend (3.14) (push) Successful in 4m30s
Compose Smoke Test / test-compose (push) Failing after 22s
Playwright Tests / merge-reports (push) Failing after 2m19s
2026-11-01 13:47:19 +01:00
stroblme 7919cf5670 fluksio requires fluksio-worker 0.2.x: the 0.1 range left the 0.2.0 wheels unresolvable, and publish stopped on it
Publish / publish (push) Successful in 47s
2026-11-01 13:47:01 +01:00
stroblme 78e1011089 Back to development
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m7s
Playwright Tests / test-playwright (2, 2) (push) Failing after 21s
pre-commit / pre-commit (push) Failing after 2m17s
Test Backend / test-backend (3.12) (push) Successful in 4m36s
Test Backend / test-backend (3.14) (push) Successful in 4m23s
Compose Smoke Test / test-compose (push) Failing after 26s
Playwright Tests / merge-reports (push) Failing after 2m20s
2026-11-01 13:21:13 +01:00
stroblme 5e3a2f9a62 Release v0.2.0
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m9s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
Playwright Tests / merge-reports (push) Canceled after 0s
pre-commit / pre-commit (push) Canceled after 41s
Test Backend / test-backend (3.12) (push) Canceled after 0s
Test Backend / test-backend (3.14) (push) Canceled after 0s
Compose Smoke Test / test-compose (push) Canceled after 0s
Publish / publish (push) Failing after 24s
2026-11-01 13:18:37 +01:00
stroblme 64ef7c84d6 Portal members: a request that loses the adoption race reads back the winner's user instead of being refused
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m16s
Playwright Tests / test-playwright (2, 2) (push) Failing after 19s
pre-commit / pre-commit (push) Failing after 2m17s
Test Backend / test-backend (3.12) (push) Successful in 4m51s
Test Backend / test-backend (3.14) (push) Successful in 4m32s
Compose Smoke Test / test-compose (push) Failing after 26s
Playwright Tests / merge-reports (push) Failing after 2m17s
2026-11-01 11:21:42 +01:00
stroblme e6614ff378 Image builds with uv 0.13.6; follow-redirects 1.16.2 and seroval 1.6.12; getting started covers Docker on Raspberry Pi OS and Compose 2.24
Docs / docs (push) Successful in 24s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m28s
Playwright Tests / test-playwright (2, 2) (push) Failing after 21s
pre-commit / pre-commit (push) Failing after 2m17s
Test Backend / test-backend (3.12) (push) Successful in 4m44s
Test Backend / test-backend (3.14) (push) Successful in 4m34s
Compose Smoke Test / test-compose (push) Failing after 25s
Playwright Tests / merge-reports (push) Failing after 2m21s
2026-11-01 10:59:51 +01:00
stroblme d8eb6968f0 test_wren: load numpy itself, so the BLAS test passes alone 2026-11-01 10:59:45 +01:00
stroblme 7f40ea62de Duplicating a copy names it after the original; a failed duplicate or delete says why 2026-11-01 10:59:36 +01:00
stroblme 7a7315043d Alerts: a delivery no channel took leaves the cooldown unspent 2026-11-01 10:59:28 +01:00
stroblme 83c5f2dfcf A run that starts clears its 'Waiting for a worker' reason 2026-11-01 10:59:21 +01:00
stroblme 48d4be6093 Starting a running flow restarts its nodes, so interval and cron Injects keep firing; a params-dict node is told settings arrive by name 2026-11-01 10:59:14 +01:00
Melvin Strobl e0f5ddc3a4 Devices may publish to their own out ports' messages (a level set on the device itself)
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 36s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m13s
Test Backend / test-backend (3.12) (push) Successful in 4m32s
Test Backend / test-backend (3.14) (push) Successful in 4m18s
Compose Smoke Test / test-compose (push) Failing after 26s
Playwright Tests / merge-reports (push) Failing after 2m16s
2026-10-29 21:58:38 +01:00
Melvin Strobl d2e8c01785 Getting started: own settings go in compose.override.yml, uv waits 120 s for a download 2026-10-29 21:35:50 +01:00
stroblme 75b3a93094 Fix work recovery and reduce engine and chart overhead
Docs / docs (push) Successful in 33s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m37s
Playwright Tests / test-playwright (2, 2) (push) Failing after 27s
pre-commit / pre-commit (push) Failing after 2m37s
Test Backend / test-backend (3.12) (push) Successful in 5m7s
Test Backend / test-backend (3.14) (push) Successful in 5m2s
Compose Smoke Test / test-compose (push) Failing after 28s
Playwright Tests / merge-reports (push) Failing after 2m29s
2026-10-29 14:24:17 +01:00
stroblme 87e750c776 Code editor: CodeMirror 6 replaces Monaco
Node source (Python) and the JSON dialog share one CodeMirror editor with
find and replace, autocomplete, folding, indent guides, comment toggle and a
JSON lint, themed from the CSS tokens so light and dark follow the .dark class
without a second palette. Programmatic value changes no longer fire onChange,
blur ignores focus moving into the find bar, readOnly keeps selection and copy.

The editor chunk is 470 kB (157 kB gzipped) against 2.7 MB (704 kB) for Monaco,
which also pulled in a 1.2 MB JSON mode, two workers and a font. At 4x CPU on
slow 4G a node panel is ready in 5.7 s instead of 10.8 s, the JSON dialog in
1.5 s instead of 6.7 s. Typing without a key delay keeps every character.

monaco-editor, @monaco-editor/react and the dompurify override are removed.
Specs: typed code reaches the node source, a JSON setting saves only text that
parses. Docs name Tab and Ctrl-M.
2026-10-28 17:32:49 +01:00
stroblme 5d0ea480bf Alerts: a Telegram channel, one chat or group each, connected by a one-time link
The channel sends through the Bot API with httpx only: HTML with escaped text,
cut to fit a message, silent for a warning, a short 429 waited out once, and
errors that say what to do about a blocked bot, a lost chat or a bad token and
never quote the request URL. The bot token is a stored secret: saving a
channel refuses a literal one, the card's "Store token" writes the secret and
a reference, and no endpoint returns it.

A chat id is never typed. Connect starts a ten minute code, and the chat that
sends `/start <code>` back through the bot is the one connected, so it is a
chat somebody opened a link for. The engine fetches updates only while a code
is waiting (no standing poll, no webhook, no inbound address) and ignores
everything else without a reply. Connect needs a saved form, and the form
takes the connected chat in without touching what was typed since.

The token is part of every Bot API URL, so the httpx logger gets the same
filter the request lines have, which now also masks an httpx.URL argument.
2026-10-28 13:06:58 +01:00
stroblme bfaaa7cd03 Alerts: cooldowns survive a restart, and a connection must stay down a minute before it alerts
The alert manager kept its cooldowns and its ceiling per hour in memory, so a
node that stayed down was announced again by every restart. Each delivered
alert is now a row in the new alert_sent table (additive migration), loaded
for the last seven days when the manager starts and pruned as new ones come
in. A ledger that fails never stops an alert. The manager's clock is wall
time for the same reason.

A node_health "down" alert also waits 60 s (HEALTH_GRACE_S) in a pending task
per node; a later event that is not "down" cancels it, as does stopping the
manager, so a broker restarting with the stack alerts nobody.

The cooldown used to be stamped before the ceiling per hour was checked, so
an alert the ceiling held back still spent its cooldown. It is stamped only
once the alert is sent now.
2026-10-28 12:46:42 +01:00
stroblme 00842fa175 Health summary: a node is failing after two failures in a row; one is a warning, a good run ends both
The summary read degraded for 15 minutes after any single node_error, even
once the node had run well again and including one bad input from a visitor.
The controller now counts a node's failures since it last ran well and notes
when it recovered; both carry across rebuilds and an acknowledgement resets
the count. Two in a row inside the 15 minute window is "failing" (a problem),
one is listed under warnings, and nodes.failing keeps meaning "failing now".
2026-10-28 12:42:35 +01:00
stroblme 2dc20911a3 Alerts: configuration, test and search entries are a superuser's; subscribing a browser stays open
Any signed-in account could save an ntfy channel with its own `server` and a
`{"$secret": ...}` token and press Test, which sent the stored secret to that
host. The config GET/PUT, the test route and the global search's alert entries
now answer to superusers, like /secrets. The webpush key and subscribe routes
stay open to every account, and the Alerts page shows a non-superuser only the
button for its own browser. The events table in the docs gains run_finished.
2026-10-28 12:40:44 +01:00
stroblme 29f4d93c2a Search: node settings are indexed, with credentials masked on the server
/search gains a `setting` category, `<node>.<setting>` under its flow with the
node type and the value in a new `detail` field, so Ctrl-K finds which node
uses a broker or a timeout and opens the node. `setting_text` is the one place
that decides what may be said: a reference to a stored secret shows its name
(to superusers only, as the secrets list does), a plain value in an `x-secret`
field or under a key like password, token or api_key is masked and never
indexed, nested credentials in headers or query are masked, long values are
clipped. The global search ranks what contains the query as typed ahead of
loose spellings before it caps a group, which long setting values would
otherwise have crowded out.
2026-10-28 11:07:27 +01:00
stroblme c04e4883c2 MCP: mcp 2.3 (MCPServer, transport arguments on streamable_http_app, bearer read in middleware); tests for the 2026-07-28 revision and the resource check 2026-10-28 09:21:01 +01:00
stroblme c04d6b83c6 MCP tests: the SDK's own client connects over streamable HTTP with a bearer, and older protocol revisions still answer 2026-10-28 09:11:58 +01:00
stroblme 439daf5655 Backend: mcp 1.30, uv 0.13 and dnspython 2.9 for their advisories; MCP tokens checked against the resource URL 2026-10-28 08:42:07 +01:00
stroblme b5e1a4c5b2 Request log: a hook's secret is masked in uvicorn's lines and the tunnel connector's failure line 2026-10-28 08:41:52 +01:00
stroblme 5c7a8773ad SDK: sync and run import the declaring module in a child process
The command line no longer imports a research module, so import-time GPU work
ends with a short-lived child instead of holding a CUDA context for the length
of a followed run. fluksio.sdk.declare starts the child, which writes each
flow's document and shims as JSON; Declared stands in for Flow in sync().
Errors keep their type and message, a child's traceback reaches the user.
2026-10-28 08:11:14 +01:00
stroblme 34243e6902 SDK: artifacts work outside a node, run --follow prints status and queue waits, fluksio cancel
save_artifact/load_artifact outside a node use the worker's own store code in
$FLUKSIO_ARTIFACT_DIR or <tmp>/fluksio-artifacts, so a node can be called by
hand. --follow prints status changes and each node's resource wait; the
placer's waiting snapshot now carries the run. fluksio cancel <run>...
2026-10-28 08:07:35 +01:00
stroblme f52d9c3077 Service node ends its call when the loop cancels every task; test order and timing leaks
_serve_once treated a cancelled call future as finished, so an event loop
cancelling all tasks at once (shutdown) never sent the worker a cancel and the
call's thread kept asyncio.run waiting for SILENCE_S. The cameras audio test
now removes its voice flow and device, the unpaired-frame test waits for its
first frame before the unpair, and test_connector no longer inherits the
controller an app test bound to demand.
2026-10-27 21:23:40 +01:00
stroblme e37aa2bdd7 Backend messages and field descriptions that reach the UI lose their em-dashes
Per the Microcopy rule: size and duration errors, the flavor errors, run
parameter errors, the Resources field descriptions, 401/409/400 details,
secret errors, the websocket node description and its on_connect setting.
The generated client follows.
2026-10-27 17:52:55 +01:00
Melvin Strobl 00cda9d2f1 connector: wanted(port) says whether anyone watches or reads a port 2026-10-27 15:20:24 +01:00
Melvin Strobl 7b453af3fe wren node: calibration setting, passed to wren's training as the installation's calibration records 2026-10-26 22:17:48 +01:00
Melvin Strobl ba75c78820 wren node: install hints and docs pin the public v2.0.0 tag 2026-10-25 17:17:17 +01:00
Melvin Strobl 82c64a10b6 Make documentation standalone and generalize examples 2026-10-25 14:43:20 +01:00
Melvin Strobl 96ba512d40 wren node: untrained targets left out of the prompt, a read beside a held-back set goes on whole 2026-10-25 12:09:37 +01:00
stroblme e9941b770d wren node: Train in the panel, as a run of wren's training flow
POST/GET /flows/{name}/nodes/{id}/train submit wren_train on the node's
targets, examples and tests and read its latest run (run.for_node,
cause "train"); "Use this model" writes `model` into the draft.
2026-10-25 01:18:54 +02:00
stroblme 07fa5a2dc2 wren node: targets from the wiring, on wren 2
Every output is a target and every input besides prompt a reading. A
targets setting gives descriptions, values, ranges and per-target
guards, model takes a trained pack's artifact digest, min_confidence
defaults to the model's threshold. A command is applied whole or not at
all, and targets the model was not trained on read degraded. The home
path (area/kind ports, groups, brightness, temperature answers) is gone.
2026-10-24 21:28:42 +02:00
stroblme 6892651e76 Artifact sweep keeps digests named in node settings
A Service's weights or a wren node's model held only in a setting were
swept once the run that produced them was deleted.
2026-10-24 21:28:35 +02:00
Melvin Strobl a34d47b24f wren node writes the house brightness from lights.set/dim/brighten (wren s11) 2026-10-22 22:43:33 +02:00
Melvin Strobl 613eb8a744 wren node takes switches (plug, socket, appliances), refused when the model lacks the kind 2026-10-22 17:48:41 +02:00
Melvin Strobl 6b575a4989 wren applies all/every commands itself, answers temperatures, reads aliases and number words; tts says units and clock times
wren's model answers "turn off all lights" with one call, and at 0.98 a
living room light switched on. The node now applies such a command to every
wired target of the kind. Number words become digits ("in five minutes"
lost its delay), aliases put the house's words and known mishearings in the
model's, and sensors.temperature is answered from <area>_temperature or
outdoor. tts writes out units and clock times before speaking.
2026-10-22 12:34:40 +02:00