Add the flow API: typed messages, git-backed store, REST and live events

Makes the flow engine reachable from the API, which is what M3 needs before
any of it can reach the browser.

- app/flow is a package now; the prototype's watch-dir scripts and the
  matplotlib/networkx visualiser are gone with their dependencies.
- Messages carry a serializable dtype instead of a live Python type, and a
  port name, so the graph can speak qualified names while node functions keep
  local arguments. Redis state is JSON, not pickle.
- Message names are namespaced per flow ("heating.temp"); a bare name resolves
  to its own flow, a dotted one crosses flows.
- Several nodes may provide the same message: producers are a list, so fan-in
  is a real edge instead of a silently dropped one.
- Flows are stored as flow.json plus node sources in a git repository, one
  commit per save, with identical saves skipped so autosave stays quiet.
- Node failures are isolated and reported per node; validate() returns cycles
  and unconnected inputs instead of raising deep in a run.
- Credentials live in an encrypted store and are referenced as {"$secret": …}.
- Engine events reach websocket clients through a bus, so values, node status
  and execution show up live.
This commit is contained in:
Melvin Strobl
2026-08-15 17:23:36 +02:00
parent ec19e2beda
commit e8895faa47
54 changed files with 2586 additions and 4978 deletions
+9
View File
@@ -1,5 +1,6 @@
import secrets
import warnings
from pathlib import Path
from typing import Annotated, Any, Literal
from pydantic import (
@@ -37,6 +38,14 @@ class Settings(BaseSettings):
FRONTEND_HOST: str = "http://localhost:5173"
ENVIRONMENT: Literal["local", "staging", "production"] = "local"
# Flows live on disk as a git repository; secrets stay outside it.
FLOWS_DIR: Path = Path("flow-data/flows")
SECRETS_FILE: Path = Path("flow-data/secrets.enc")
FLOW_MAX_WORKERS: int = 4
# Without a Redis host the engine keeps its state in memory.
REDIS_HOST: str | None = None
REDIS_PORT: int = 6379
BACKEND_CORS_ORIGINS: Annotated[
list[AnyUrl] | str, BeforeValidator(parse_cors)
] = []