Melvin StroblandClaude Fable 5
7344eac262
Flows can now be taken off the engine and put back. Stopped state lives in a runtime.json beside the flow, not in the flow document: the canvas autosaves that document, so a stopped flow would otherwise start itself again on the next edit. A stopped flow gets no subscriptions, schedules or webhooks, its nodes are skipped by the scheduler, and running it answers 409. Pausing holds a flow's nodes while its values keep arriving, so the canvas still shows what is coming in. Node code is user code and print is how it says things, so stdout is teed through a contextvar sink active only during a node execution — one event per execution, capped, so a chatty node cannot outrun the stream. A node that fails sends its traceback the same way, trimmed to the author's own frames. The dock gains a logs panel and a pause control; the dashboard replaces its placeholder with what is running, stopped or failing; the edge inspector can send the last message again. Single-stepping is deferred and noted: the scheduler keeps no progress between calls, so a step button would re-run the same node rather than advance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
6.6 KiB
6.6 KiB
Roadmap
Component-level breakdown. The milestone-level master (M1–M5, with the vision
decisions behind it) is docs/private/roadmap.md in the docs submodule.
Implementation strategy and record of existing/planned features. Completed items are
terse checklists — the requirement detail lives in docs/private/vision.md (goals,
requirements, decisions) and docs/architecture/structure.canvas (the four-way component split).
Remaining tasks keep enough scope to be actionable.
Legend: [x] done · [ ] planned · sub-lists split done vs. remaining for partial items.
Within each phase, remaining [ ] items are listed in rough priority order: making the
existing flow engine reachable and persistent precedes new feature breadth.
Phase 0 — Workspace and platform
- Root orchestrator repo with
app,indexanddocsas submodules make initbootstrap: secrets generation, per-stack.envpropagation, sharedproxydocker network- Layered compose (
compose.yml→compose.dev.yml→compose.local.yml) for both stacks, one Traefik serving${DOMAIN},app.${DOMAIN},api.${DOMAIN} - Design token contract: root
DESIGN-GUIDELINES.md, per-repoDESIGN.md, byte-identical token blocks verified bymake design-check - CI on Codeberg (Forgejo Actions): pre-commit, backend tests, Playwright, compose smoke
Phase 1 — Backend: management
Python, optimised for development speed. Owns the graph structure, persistence and the
external interfaces. See docs/architecture/structure.canvas → Backend – Management.
- FastAPI + SQLModel + Alembic + Postgres base with JWT auth and user management
- Flow engine in
backend/app/flow/:Node/Pipeline/StateBackend(memory + Redis) /FlowController - Node types: HTTP, MQTT, InfluxDB, Delay, MLP
app/flowis an importable package with absoluteapp.flow.*imports- Typed, serializable node I/O: every port declares a
DType, messages are JSON on the wire and in Redis, no pickle anywhere. Binary codecs are still open —DType.JSONcarries everything non-scalar for now - Message namespacing per flow (
flow.message), with several producers per message resolving to real fan-in - Secrets/credentials store for node integrations managed via the API/UI
(encrypted at rest, referenced from node params as
{"$secret": "name"});.envbootstrap-only - Connector node contract: declared I/O schemas, UI-driven credential and setup flow, polling/discretization behaviour, health reporting, packaging metadata — plus an authoring guide and skeleton repo. Device-specific nodes are written against it as external plugins; the registry follows later
- Flow persistence:
flow.jsonplus node sources per flow, replacing the watch-directory prototype - REST + WebSocket API over the engine: create/read/update flows, edit node source, run, and stream values, node status and execution events
- Dependency-loop detection and graph validation surfaced as API errors
- Per-flow start/stop, stored in a
runtime.jsonbeside the flow so it survives a restart and stays out of the autosaved document; pause/resume holds a flow's nodes while its values keep arriving - Node log streaming: what a node prints, and the traceback of one that
fails, reach the editor as
node_logevents - MQTT broker / InfluxDB compose services for local development
- Git-based versioning of the flow store (one commit per saved change)
- Draft/publish split: edits autosave to
flow.draft.json/nodes.draft/, the engine runs only the published files, and publishing promotes the draft. Saves carry the version they were based on, so a second client editing the same flow is refused rather than overwritten - Import/export of a flow as human-readable code plus a JSON structure
- Per-input/-output discretization interval setting
- Alert / notification handler
- Test nodes: a small node dragged onto an existing one, smoke or unit, blocking deployment on failure
- User management scoped per flow and per data set
- LLM interface for natural-language flow authoring
Phase 2 — Backend: processing
Rust, optimised for throughput. Executes nodes and distributes them across workers. See
docs/architecture/structure.canvas → Backend – Processing.
- Parallel invocation of stateless nodes over independent input sets, to
keep I/O delay minimal (stateful I/O nodes keep serializing via the
synchronousmechanism) - Extract node execution from the Python prototype into a Rust engine
- Worker distribution and load balancing across capable devices
- Input/output validation at the node boundary
- Data aggregation and discretization
Phase 3 — Frontend: admin view
React + Vite, primarily desktop but usable on mobile. See docs/architecture/structure.canvas →
Frontend – Admin View.
- Dashboard SPA shell: TanStack Router, floating frosted sidebar, auth flows, generated OpenAPI SDK
- Node canvas (
@xyflow/react) showing nodes and their connections, which are derived from message names rather than stored - Tab-style view of atomic flows, with a floating dock
- Embedded code editor (Monaco) for node source
- Live values on the edges, with the last payload and its time on click
- Validation shown on the node it belongs to, and summarised in the dock
- Publish control and draft markers in the flow bar, discard in the flow panel, and a conflict dialog when another client got there first
- Dashboard showing which flows run, which are stopped and which have errors, with a switch per flow
- Logs panel in the canvas dock, pause/resume beside Run, and replaying an edge's last message from the inspector
- Device assignment per node, selectable from compatible devices
- Test-node affordance on the canvas
- User management screens
- Mobile-friendly canvas: touch connect, full-screen node panel
- Installable as a PWA (
vite-plugin-pwa)
Phase 4 — Frontend: dashboard view
Shares components with the admin view. See docs/architecture/structure.canvas →
Frontend – Dashboard View.
- User-defined dashboard layout with edit and view modes
- Responsive layout targeting wall panels, mobile and desktop
- Per-device view
Phase 5 — Website and docs
- Marketing site with a live node-graph demo, shared design system
- Published documentation site fed from the
docssubmodule - Umami analytics configured (the site still ships the placeholder script)