Add the flow API: typed messages, git-backed store, REST and live events
Makes the flow engine reachable from the API, which is what M3 needs before
any of it can reach the browser.
- app/flow is a package now; the prototype's watch-dir scripts and the
matplotlib/networkx visualiser are gone with their dependencies.
- Messages carry a serializable dtype instead of a live Python type, and a
port name, so the graph can speak qualified names while node functions keep
local arguments. Redis state is JSON, not pickle.
- Message names are namespaced per flow ("heating.temp"); a bare name resolves
to its own flow, a dotted one crosses flows.
- Several nodes may provide the same message: producers are a list, so fan-in
is a real edge instead of a silently dropped one.
- Flows are stored as flow.json plus node sources in a git repository, one
commit per save, with identical saves skipped so autosave stays quiet.
- Node failures are isolated and reported per node; validate() returns cycles
and unconnected inputs instead of raising deep in a run.
- Credentials live in an encrypted store and are referenced as {"$secret": …}.
- Engine events reach websocket clients through a bus, so values, node status
and execution show up live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016WzrvW7rjQbynnhF6pxh6i
This commit is contained in:
co-authored by
Claude Fable 5
parent
61be29827d
commit
06a4506767
+16
-12
@@ -8,29 +8,33 @@ Deferring because out of scope is fine, but don't mention deferring than.
|
||||
|
||||
## Open
|
||||
|
||||
- CHORE/FLOW: `backend/app/flow/` is not an importable package. There is no `__init__.py`
|
||||
and siblings are imported top-level (`from util import …` in `nodes.py:6`, `from nodes
|
||||
import …` in `controller.py:23-26` and `pipeline.py:17-19`), so the modules only run with
|
||||
the cwd set to `app/flow/`. Convert to `from app.flow.util import …`.
|
||||
- CHORE/FLOW: the flow test files live in `backend/app/flow/test*.py`, inside the package
|
||||
rather than `backend/tests/`, so `coverage run -m pytest tests/` never collects them.
|
||||
- CHORE/INFRA: port the n3xd CI workflows (pre-commit, backend tests, Playwright shards,
|
||||
compose smoke) to `.gitea/workflows/`. The Gitea instance runs Gitea Actions and needs CI
|
||||
enabled per repository.
|
||||
- FEAT/UI: `@xyflow/react` and a Monaco editor belong here once the node canvas starts.
|
||||
The website already demos `@xyflow/react` on its features page.
|
||||
- CHORE/FLOW: `app/flow/nodes.py` and `state.py` are excluded from strict mypy
|
||||
(`[[tool.mypy.overrides]]` in `pyproject.toml`). The node classes still carry prototype
|
||||
typing and redis-py types its calls as sync/async unions. Shrink the exclusion as each
|
||||
integration is revisited.
|
||||
- FEAT/API: per-hook shared secrets for `HttpNode` webhooks. They are mounted unauthenticated
|
||||
under `/hooks/{flow}/…` because devices cannot present a JWT.
|
||||
- FEAT/FLOW: state does not survive a restart — the engine runs on `MemoryState` unless
|
||||
`REDIS_HOST` is set, and there is no Redis compose service yet.
|
||||
- CHORE/FLOW: rotating `SECRET_KEY` makes the stored secrets unreadable, since the Fernet key
|
||||
is derived from it. Needs a re-encrypt path before anyone rotates in production.
|
||||
- PERF/FLOW: every save rebuilds the whole pipeline. Fine at the current flow count; rebuild
|
||||
only the touched flow when it starts to show.
|
||||
- FEAT/UI: reintroduce `--chart-*` tokens as one designed sequential scale when the first
|
||||
chart lands. The stock shadcn five were dropped because their light and dark values are
|
||||
unrelated hues.
|
||||
- CHORE/API: `backend/app/models.py` still holds only the template's `User` and `Item`.
|
||||
`Item` should go once flow persistence replaces it.
|
||||
- CHORE/API: `backend/app/models.py` still holds the template's `Item` model. It has no
|
||||
purpose now that flows are persisted and can go with its routes and tests.
|
||||
- PERF/UI: both frontend bundles exceed the 500 kB warning threshold. Revisit chunking
|
||||
once the canvas and editor are in, since they will dominate.
|
||||
|
||||
## Blocked
|
||||
|
||||
- FEAT/INFRA: Redis, MQTT broker and InfluxDB compose services. Blocked on the flow engine
|
||||
being reachable from the API — no runtime path imports them today.
|
||||
- FEAT/INFRA: MQTT broker and InfluxDB compose services for local development. The node types
|
||||
exist; a local broker would make them testable without external hardware.
|
||||
- CHORE/INFRA: `bun install` inside the frontend Docker build intermittently fails with
|
||||
"Fail extracting tarball" for several packages at once, and succeeds on a plain rebuild.
|
||||
It looks like concurrent extraction under memory pressure. Pin down or retry in the
|
||||
|
||||
Reference in New Issue
Block a user