Supervise the engine's host: deep health, loop watchdog, one worker
The API image ran four uvicorn workers, and each one built a full flow controller — four sets of MQTT subscriptions, cron ticks and webhooks. Runs one worker now; scaling out is the worker split, not more processes. Adds a loop-lag watchdog and a deep /utils/health/ that fails when the event loop is wedged or Redis is unreachable, the two failure modes a process-alive check never sees. Autoheal restarts on that signal, behind a compose profile because it mounts the Docker socket. The private user-seeding routes now need an explicit opt-in rather than just ENVIRONMENT=local, so a deployment that kept the default never exposes them.
This commit is contained in:
@@ -19,6 +19,7 @@ from app.flow.plugins import load_plugins
|
||||
from app.flow.secrets import init_secrets
|
||||
from app.flow.state import MemoryState, RedisState, StateBackend
|
||||
from app.flow.store import FlowStore
|
||||
from app.flow.watchdog import LoopWatchdog
|
||||
|
||||
|
||||
def custom_generate_unique_id(route: APIRoute) -> str:
|
||||
@@ -62,6 +63,9 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
fastapi_app=app,
|
||||
)
|
||||
app.state.flow_controller = controller
|
||||
watchdog = LoopWatchdog(event_bus)
|
||||
app.state.watchdog = watchdog
|
||||
watchdog_task = asyncio.create_task(watchdog.run(), name="loop-watchdog")
|
||||
await controller.start()
|
||||
try:
|
||||
# A mounted sub-app gets no lifespan of its own, so the MCP session
|
||||
@@ -69,6 +73,7 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
async with _mcp_sessions():
|
||||
yield
|
||||
finally:
|
||||
watchdog_task.cancel()
|
||||
await controller.stop()
|
||||
if settings.MCP_ENABLED:
|
||||
from app.mcp.http import aclose
|
||||
|
||||
Reference in New Issue
Block a user