The engine now speaks MCP at /mcp, with a built-in OAuth 2.1 authorization
server in front of it: an agent registers itself, sends a human to the browser
to approve it, and exchanges the resulting code for a token. PKCE is required,
codes are single-use and stored only as hashes, the browser is redirected to
the URI that was registered rather than the one asked for, and refresh tokens
rotate so that replaying a spent one revokes the whole line.

Twenty tools cover reading, building, publishing and running flows, and each
one calls the same REST endpoint the dashboard calls, in-process, carrying the
caller's own token. That keeps one description of what a flow is and what may
be done to it — validation, the draft/publish split, the version check — and
means an agent can do nothing a person could not do in the browser.

Agent tokens are RS256 with a keypair of their own rather than the secret that
signs browser sessions, so deleting the key withdraws every agent without
logging anyone out, and deps.decode_token grew the branch that trusting a
second issuer will need when the hosted login arrives.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-08-16 00:22:41 +02:00

7.7 KiB

Raw Blame History

Roadmap

Component-level breakdown. The milestone-level master (M1–M5, with the vision decisions behind it) is docs/private/roadmap.md in the docs submodule.

Implementation strategy and record of existing/planned features. Completed items are terse checklists — the requirement detail lives in docs/private/vision.md (goals, requirements, decisions) and docs/architecture/structure.canvas (the four-way component split). Remaining tasks keep enough scope to be actionable.

Legend: [x] done · [ ] planned · sub-lists split done vs. remaining for partial items.

Within each phase, remaining [ ] items are listed in rough priority order: making the existing flow engine reachable and persistent precedes new feature breadth.

Phase 0 — Workspace and platform

Root orchestrator repo with app, index and docs as submodules
make init bootstrap: secrets generation, per-stack .env propagation, shared proxy docker network
Layered compose (compose.yml → compose.dev.yml → compose.local.yml) for both stacks, one Traefik serving ${DOMAIN}, app.${DOMAIN}, api.${DOMAIN}
Design token contract: root DESIGN-GUIDELINES.md, per-repo DESIGN.md, byte-identical token blocks verified by make design-check
CI on Codeberg (Forgejo Actions): pre-commit, backend tests, Playwright, compose smoke

Phase 1 — Backend: management

Python, optimised for development speed. Owns the graph structure, persistence and the external interfaces. See docs/architecture/structure.canvas → Backend – Management.

FastAPI + SQLModel + Alembic + Postgres base with JWT auth and user management
Flow engine in backend/app/flow/: Node / Pipeline / StateBackend (memory + Redis) / FlowController
Node types: HTTP, MQTT, InfluxDB, Delay, MLP
app/flow is an importable package with absolute app.flow.* imports
Typed, serializable node I/O: every port declares a DType, messages are JSON on the wire and in Redis, no pickle anywhere. Binary codecs are still open — DType.JSON carries everything non-scalar for now
Message namespacing per flow (flow.message), with several producers per message resolving to real fan-in
Secrets/credentials store for node integrations managed via the API/UI (encrypted at rest, referenced from node params as {"$secret": "name"}); .env bootstrap-only
Connector node contract: ConnectorNode with a declared contract version, a polling coordinator that deduplicates, x-secret parameters the editor renders as a secret picker, and health reporting. Connectors are installed packages found through the fluksio.node_types entry point group; the contract is documented in docs/connectors/ with a working skeleton at connector-skeleton/. The registry follows later
Node lifecycle as a protocol (start/stop/report_health on Node), replacing the controller's per-type isinstance chains — the same hooks a connector implements, validated on the built-in nodes first
Flow persistence: flow.json plus node sources per flow, replacing the watch-directory prototype
REST + WebSocket API over the engine: create/read/update flows, edit node source, run, and stream values, node status and execution events
Dependency-loop detection and graph validation surfaced as API errors
Per-flow start/stop, stored in a runtime.json beside the flow so it survives a restart and stays out of the autosaved document; pause/resume holds a flow's nodes while its values keep arriving
Node log streaming: what a node prints, and the traceback of one that fails, reach the editor as node_log events
MQTT broker / InfluxDB compose services for local development
Git-based versioning of the flow store (one commit per saved change)
Draft/publish split: edits autosave to flow.draft.json / nodes.draft/, the engine runs only the published files, and publishing promotes the draft. Saves carry the version they were based on, so a second client editing the same flow is refused rather than overwritten
Import/export of a flow as human-readable code plus a JSON structure
Per-input/-output discretization interval setting: a port publishes, or wakes its node, at most every n seconds. State keeps the latest value, so only the delivery is skipped
Alert / notification handler
Test nodes: a small node dragged onto an existing one, smoke or unit, blocking deployment on failure
User management scoped per flow and per data set
MCP server over the same API: agents authenticate through a built-in OAuth 2.1 authorization server (dynamic registration, PKCE, rotating refresh tokens) and drive the flow API through 20 tools. Tokens are RS256, signed with their own keypair, so the set can be revoked on its own — and an additional issuer is one branch in deps.decode_token, which is the seam remote access needs later
LLM interface for natural-language flow authoring beyond the MCP tools

Phase 2 — Backend: processing

Rust, optimised for throughput. Executes nodes and distributes them across workers. See docs/architecture/structure.canvas → Backend – Processing.

Parallel invocation of stateless nodes over independent input sets, to keep I/O delay minimal (stateful I/O nodes keep serializing via the synchronous mechanism)
Extract node execution from the Python prototype into a Rust engine
Worker distribution and load balancing across capable devices
Input/output validation at the node boundary
Data aggregation and discretization

Phase 3 — Frontend: admin view

React + Vite, primarily desktop but usable on mobile. See docs/architecture/structure.canvas → Frontend – Admin View.

Dashboard SPA shell: TanStack Router, floating frosted sidebar, auth flows, generated OpenAPI SDK
Node canvas (@xyflow/react) showing nodes and their connections, which are derived from message names rather than stored
Tab-style view of atomic flows, with a floating dock
Embedded code editor (Monaco) for node source
Live values on the edges, with the last payload and its time on click
Validation shown on the node it belongs to, and summarised in the dock
Publish control and draft markers in the flow bar, discard in the flow panel, and a conflict dialog when another client got there first
Marking a node reusable, and placing a shared one from the palette
Secret picker for credential parameters, so a password never lands in flow.json
Dashboard showing which flows run, which are stopped and which have errors, with a switch per flow
Logs panel in the canvas dock, pause/resume beside Run, and replaying an edge's last message from the inspector
Device assignment per node, selectable from compatible devices
Test-node affordance on the canvas
User management screens
Mobile-friendly canvas: touch connect, full-screen node panel
Installable as a PWA (vite-plugin-pwa)

Phase 4 — Frontend: dashboard view

Shares components with the admin view. See docs/architecture/structure.canvas → Frontend – Dashboard View.

User-defined dashboard layout with edit and view modes
Responsive layout targeting wall panels, mobile and desktop
Per-device view

Phase 5 — Website and docs

Marketing site with a live node-graph demo, shared design system
Published documentation site fed from the docs submodule
Umami analytics configured (the site still ships the placeholder script)

Reference in New Issue View Git Blame Copy Permalink

7.7 KiB Raw Blame History Unescape Escape