Supervise the engine's host: deep health, loop watchdog, one worker

The API image ran four uvicorn workers, and each one built a full flow
controller — four sets of MQTT subscriptions, cron ticks and webhooks.
Runs one worker now; scaling out is the worker split, not more processes.

Adds a loop-lag watchdog and a deep /utils/health/ that fails when the
event loop is wedged or Redis is unreachable, the two failure modes a
process-alive check never sees. Autoheal restarts on that signal, behind
a compose profile because it mounts the Docker socket.

The private user-seeding routes now need an explicit opt-in rather than
just ENVIRONMENT=local, so a deployment that kept the default never
exposes them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011LF61rxW1FG5YCD2J9YqjY
This commit is contained in:
root
2026-08-16 07:14:28 +02:00
co-authored by Claude Fable 5
parent 8d82d6c4ec
commit 5462842b8a
16 changed files with 325 additions and 13 deletions
+4
View File
@@ -72,6 +72,10 @@ external interfaces. See `docs/architecture/structure.canvas` → *Backend M
wakes its node, at most every n seconds. State keeps the latest value, so
only the delivery is skipped
- [ ] Alert / notification handler
- [x] Deep health check (`GET /utils/health/`): reports event-loop lag and state-backend
reachability and fails the container healthcheck, so a wedged engine is restarted
rather than counted as up. One engine per deployment — the API image runs a single
worker, because a second one would be a second engine
- [ ] Test nodes: a small node dragged onto an existing one, smoke or unit, blocking
deployment on failure
- [ ] User management scoped per flow and per data set