Keep the engine's own history, and a screen that reads it
A second bus subscriber folds executions, errors, timings and queue lag into per-minute rollups, keeps failures with their traceback and an audit trail of who published what, and records one row per cascade — manual runs and previews included, under an id of their own that writes no idempotency markers. Read back through /observability/*, which always answers 200 so a degraded engine still renders its own health screen. Also fixes two things found on the way: node-health alerts read `status` where the engine publishes `health`, so a device dropping never alerted anyone, and the Redis queue reported `parked: 0` whatever was held. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
This commit is contained in:
+9
-1
@@ -29,7 +29,6 @@ is what M4 still waits on, together with porting the flows.
|
||||
### Bugs found while building the screens
|
||||
|
||||
- BUG/API: `POST /alerts/test/{channel}` always answers 200. `AlertManager.send()` catches and logs every delivery failure, so the alerts screen's Test button cannot tell a working channel from a broken one — the one thing it exists for. Let `send()` raise or return a result on the test path.
|
||||
- BUG/API: `RedisWorkQueue.stats()` hard-codes `"parked": 0`, so held or stranded work never shows on the health endpoint. This is why a rebuild stranding parked items went unnoticed until it was looked for.
|
||||
- BUG/FLOW: deleting a flow leaves its `pipeline:{flow}.*` Redis keys behind, and renaming one does not migrate them — the live instance carries `pipeline:__history__:dashboar.test` beside the correct `dashboard.test`. One cleanup on the delete/rename path covers both.
|
||||
- CHORE/API: revoking an OAuth client does not invalidate access tokens already issued; they are stateless JWTs valid up to `MCP_TOKEN_EXPIRE_MINUTES`. Immediate revocation means `app/mcp/http.py` checking the client row still exists.
|
||||
- CHORE/FLOW: `Pipeline.trigger`'s docstring says a paused flow still publishes so the value shows on the canvas. True only without a queue; with one the item parks before `apply_outputs` and nothing shows. Docstring and behaviour disagree.
|
||||
@@ -44,6 +43,15 @@ is what M4 still waits on, together with porting the flows.
|
||||
- FEAT/API: `POST /modules/apply` rebuilds the whole pipeline so a node that could not import its package stops being red. That resubscribes every MQTT node in the deployment; a targeted rebuild of the flows that actually failed to load would be gentler.
|
||||
- CHORE/FLOW: a node's return value now round-trips through JSON, so tuples arrive downstream as lists and anything non-JSON is an explicit error. That is the message contract, but flows written before this may notice.
|
||||
|
||||
### Engine history
|
||||
|
||||
- CHORE/FLOW: a rate-limit flush gets no run record — it is the tail of the run that scheduled it, and there is no id linking the two. A flush that fails therefore shows as a failure with no run beside it.
|
||||
- CHORE/FLOW: `Pipeline.flush` releasing a held value runs its cascade without a run id, so those executions land in the minute rollups but in no run. Threading the scheduling run's id through the queue item would close it.
|
||||
- CHORE/API: the metrics collector is a bus subscriber, so a storm that overflows the bus queue undercounts. The events dropped are the same ones the websocket drops; exact accounting would need the collector to be fed from the engine rather than the bus.
|
||||
- CHORE/API: `/observability/summary` reports the work queue's `depth` as the Redis stream length, which is the journal size (capped at `STREAM_MAXLEN`) rather than a backlog. The health screen shows `pending` instead; the field name still invites the wrong reading.
|
||||
- FEAT/UI: the health screen's window is fixed at 24 hours and the charts fold minute buckets in Python. A range picker (and `date_bin()` behind it) is the next step if anyone wants a week.
|
||||
- CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but `forget_flow` could offer to clear it.
|
||||
|
||||
### Dashboard follow-ups
|
||||
|
||||
- BUG/UI: ensure dashboard wallpanel (read-only) links hot reload automatically on dashboard changes
|
||||
|
||||
Reference in New Issue
Block a user