Add a Python SDK: flows declared in your own repository

A data scientist keeps their code where it is and decorates it: `@node`
declares a function's ports beside the function, `Flow(name, nodes=[...])`
says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds
one for a single flow. `fluksio sync` uploads the document plus a generated
import shim per node, so the store still holds a complete, runnable,
git-versioned definition while the code it imports stays theirs.

`fluksio login|run|runs` and `flow.submit().wait()` are the client half, over
the run endpoints that already existed. Runs record the user repository's
commit beside the store's, so "what code produced this number" is answerable
on the side that now holds the code.

- `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim
  generator, an HTTP client and sync. Standard library only at import, so
  `from fluksio import node` in a training script pulls in no engine.
- `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the
  repository's commit; `POST /modules/refresh` retires the workers without an
  install, which every sync calls — a worker holds the imported package in
  memory, so an edit to it is invisible until the process goes.
- The canvas shows a generated body read-only and names the repository to edit
  instead; a body edited there stops the next sync rather than being discarded.
- The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since
  the shim imports a module whose first line declares them.
- `examples/myresearch` is the worked example, `make sync-example` uploads it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU

Makefile

@@ -3,7 +3,7 @@
 # The workspace root delegates to these (see ../Makefile).
 
 .PHONY: dev-utils dev dev-local dev-lan up down update install dev-backend dev-frontend \
-        generate-client seed-example seed-demo seed-house seed-aircon seed-tinyhouse seed-hosted-demo test test-backend test-frontend soak bench-startup lint lint-backend \
+        generate-client seed-example seed-demo sync-example seed-house seed-aircon seed-tinyhouse seed-hosted-demo test test-backend test-frontend soak bench-startup lint lint-backend \
         lint-frontend format-frontend umami build docs docs-serve clean help
 
 COMPOSE_ROOT := $(CURDIR)
@@ -106,6 +106,12 @@ seed-example:  ## Seed the querying-chart example (needs a running stack + Influ
 seed-demo:  ## Seed the training-run example: a batch flow and its dashboard
 	cd backend && uv run python ../scripts/seed_demo_training.py
 
+sync-example:  ## Upload `examples/myresearch` the way a data scientist would
+	# The same command the docs give, against whichever engine `fluksio login`
+	# last talked to. The nodes import the package from this checkout, so the
+	# engine has to be one that can see this path.
+	cd backend && uv run fluksio sync ../examples/myresearch
+
 seed-house:  ## Seed the house write-path rig (needs the real broker reachable)
 	cd backend && uv run python ../scripts/seed_house_control.py
 

NOTEPAD.md

@@ -24,11 +24,14 @@ The apperance should be strictly separated from the functionality (which both se
 All widgets (and the sidebar rail) should follow the appearance and should be reworked to receive animations through the motion library.
 It should be possible to set a background in the dashboard (input, controllable externally via a node. In v1 this can be just an url to an image).
 It should be possible to disable the title of a widget.
-The dashboard setting should contain a "Touch" toggle, which, when activated, makes all widgets more touch friendly
+The dashboard setting should contain a "Touch" toggle, which, when activated, makes all widgets more touch friendly.
+All components should react responsive to the size of the widget.
+Components and animations should be carefully designed and adhere to high-quality design standards and taste.
+A minimal research on dashboard/ component design should be conducted.
 
 Specific changes
 - the nested bar should be changed in a multi-row bar with N inputs (and therefore n bars), color separated as the chart widget
-- the range picker should go to the right (vertical) of the chart widgeta
+- the range picker should go to the right (vertical) of the chart widget to allow for more vertical space of the chart
 - the color picker should be changed to a circular color picker (disk with colors, saturation changes towards the center) and a vertical slider for the brightness
 
 ### To be sorted
@@ -397,6 +400,23 @@ as an em dash.
 
 ## Deferred
 
+- FEAT/SDK: a traced flow body — `@Flow` over a function whose calls to other
+  nodes build the graph, the way Covalent's lattice does. Rejected for the first
+  version with reasons in `docs/private/python-api.md` (2.5× the code, two
+  meanings per decorated function, and several things it cannot express). The
+  registry holds `Flow` objects rather than modules, so a tracer producing the
+  same `FlowDef` would slot in without changing anything stored.
+- CHORE/SDK: two `use()`s of one function in one flow store two identical node
+  bodies. `share_node` already models this — `_lib/<name>.py` plus a
+  `source_ref` — and sync could write the shared shim once. Not done because
+  editing a shared source bypasses draft/publish, which a generated body should
+  not.
+- FEAT/SDK: `fluksio sync` puts the repository on `sys.path` from inside the
+  generated body, by absolute path. Right for the machine that synced, and
+  meaningless in a container — the note in the docs says to install the package
+  where the node runs. A `--repo-as` flag, or reading the installed location
+  back from the engine, would close it.
+
 Open on purpose. Each names what should bring it back.
 
 - PERF/UI: the app's entry chunk exceeds the warning threshold. React Flow and Monaco are already lazy; a manualChunks split measured no better, so this needs route-level work on the shell rather than chunking config.

ROADMAP.md

@@ -184,6 +184,17 @@ external interfaces. See `docs/architecture/structure.canvas` → *Backend – M
       separate from the cascade rollups, which are pruned on a retention window
       and an experiment must not be. `/runs`, `/runs/{id}`, `/runs/flows/{name}`,
       `/sweep`, `/cancel`, `/metrics` and `/series/compare`
+- [x] A Python SDK, so a research repository is the source of a flow rather than
+      a place code is copied from: `@node` declares a function's ports where the
+      function is, `Flow(name, nodes=[...])` says which of them make a flow, and
+      `use(fn, wire=…, **settings)` rebinds one for a single flow. `fluksio
+      sync` uploads the document plus a generated import shim per node — so the
+      store still holds a complete, runnable, git-versioned definition — stamps
+      it with the repository's commit, and retires the workers so the next run
+      imports the code as it is now. `fluksio login|run|runs` and
+      `flow.submit().wait()` are the client half. The decorators return the
+      function untouched, which is the whole point: it is still your code, still
+      callable, and `device="gpu"` on the same decorator is where it runs
 - [x] Streaming outputs: a node that produces values over time is a generator,
       and every `yield` is a dict keyed by output port, published the instant it
       happens; what it returns is its result. A port doing this declares

backend/fluksio/__init__.py

@@ -1,25 +1,48 @@
-"""The Fluksio engine.
+"""The Fluksio engine, and the decorators that declare flows in your own code.
 
-Inside a node, ``import fluksio`` is not this package: the worker installs a
-reporter of its own under that name before any node code runs, so what a node
-gets is :mod:`fluksio_worker.worker_main`'s ``emit`` and ``save_artifact``.
-The stubs below stand in the same place everywhere else, and say so rather
-than failing as a missing attribute.
+Two audiences, one import name. In a research repository, ``from fluksio
+import Port, node, Flow`` is the authoring API: it says which of your existing
+functions are nodes and which nodes make up a flow, and ``fluksio sync``
+uploads what it finds. Those names come from :mod:`fluksio.sdk`, which imports
+nothing but the standard library.
+
+Inside a node, ``import fluksio`` is not this package at all: the worker
+installs a reporter of its own under that name before any node code runs, so
+what a node gets is :mod:`fluksio_worker.worker_main`'s ``emit`` and
+``save_artifact`` — and inert copies of the decorators, since a module the
+node imports may well declare them at its top. The stubs below stand in the
+same place everywhere else, and say so rather than failing as a missing
+attribute.
 
 Nothing is imported from the rest of the package here. Every ``from fluksio.x
 import y`` in the engine passes through this module, and an import cycle or a
 second of start-up cost would both begin here.
 """
 
+import logging
 from importlib.metadata import PackageNotFoundError
 from importlib.metadata import version as _version
 from typing import Any
 
+from fluksio.sdk import Flow, Port, node, use
+
 try:
     __version__ = _version("fluksio")
 except PackageNotFoundError:  # pragma: no cover - a checkout that was never installed
     __version__ = "0.0.0+unknown"
 
+__all__ = [
+    "Flow",
+    "Port",
+    "emit",
+    "load_artifact",
+    "node",
+    "save_artifact",
+    "use",
+]
+
+logger = logging.getLogger(__name__)
+
 _OUTSIDE = (
     "fluksio.{name}() only works inside a node: the worker running it installs "
     "the real one. There is nothing to {verb} out here."
@@ -27,8 +50,14 @@ _OUTSIDE = (
 
 
 def emit(**ports: Any) -> None:
-    """Publish values on a node's declared output ports, mid-run."""
-    raise RuntimeError(_OUTSIDE.format(name="emit", verb="emit to"))
+    """Publish values on a node's declared output ports, mid-run.
+
+    Outside a node there is nowhere to publish to, and this does nothing on
+    purpose: a node function called directly — in a test, in a notebook, under
+    a debugger — is ordinary Python, and having it die on a progress report
+    would defeat the point of the code staying yours.
+    """
+    logger.debug("fluksio.emit(%s) outside a node", ", ".join(ports))
 
 
 def save_artifact(
@@ -38,6 +67,6 @@ def save_artifact(
     raise RuntimeError(_OUTSIDE.format(name="save_artifact", verb="save to"))
 
 
-def load_artifact(ref: dict[str, Any]) -> bytes:
-    """Read back what an artifact reference points at."""
+def load_artifact(ref: dict[str, Any]) -> str:
+    """Fetch what an artifact reference points at, and return a path to it."""
     raise RuntimeError(_OUTSIDE.format(name="load_artifact", verb="load from"))

backend/fluksio/alembic/versions/b7c21f4d9e30_run_origin_commit.py

@@ -0,0 +1,38 @@
+"""run.origin_commit
+
+A flow declared with the Python decorators is stored here as a generated
+import shim, so the store's own commit names the shim rather than the code it
+imports. This column carries the other half: the commit of the repository the
+node function actually lives in.
+
+Revision ID: b7c21f4d9e30
+Revises: ee1b4b4426a3
+Create Date: 2026-08-23
+
+"""
+
+import sqlalchemy as sa
+import sqlmodel.sql.sqltypes
+from alembic import op
+
+# revision identifiers, used by Alembic.
+revision = "b7c21f4d9e30"
+down_revision = "ee1b4b4426a3"
+branch_labels = None
+depends_on = None
+
+
+def upgrade():
+    op.add_column(
+        "run",
+        sa.Column(
+            "origin_commit",
+            sqlmodel.sql.sqltypes.AutoString(length=80),
+            nullable=False,
+            server_default="",
+        ),
+    )
+
+
+def downgrade():
+    op.drop_column("run", "origin_commit")

backend/fluksio/api/routes/modules.py

@@ -21,6 +21,7 @@ from fluksio.api.deps import (
 from fluksio.flow import modules
 from fluksio.flow.events import event_bus
 from fluksio.flow.schemas import ApplyRequest, ApplyResult, ModulesInfo
+from fluksio.models import Message
 
 router = APIRouter(
     prefix="/modules", tags=["modules"], dependencies=[Depends(get_current_user)]
@@ -70,6 +71,38 @@ async def apply_modules(
     # Retire the workers first, so the rebuild compiles every node against the
     # packages that were just installed — a node that could not import one is
     # the reason this was called, and it stays red until it is built again.
+    await _retire(controller, pool)
+    return ApplyResult(ok=True, output=output)
+
+
+@router.post("/refresh", response_model=Message)
+async def refresh_modules(
+    controller: FlowControllerDep,
+    pool: WorkerPoolDep,
+    user: CurrentUser,
+) -> Any:
+    """Retire the workers without installing anything.
+
+    A node that imports the caller's own package holds it in `sys.modules` for
+    as long as the process lives, so editing that package changes nothing a
+    running worker can see — recompiling the node would not help either, since
+    the import returns the module already there. Retiring the processes is the
+    whole of it, and `fluksio sync` asks for it after every upload.
+    """
+    event_bus.publish(
+        {
+            "type": "audit",
+            "action": "refreshed workers",
+            "flow": "",
+            "user": user.email,
+            "ts": time.time(),
+        }
+    )
+    await _retire(controller, pool)
+    return Message(message="Workers retired; the next call imports afresh")
+
+
+async def _retire(controller: Any, pool: Any) -> None:
+    """Send the worker processes away and rebuild whatever was broken."""
     pool.respawn_all()
     await controller.reload_failed_flows()
-    return ApplyResult(ok=True, output=output)

backend/fluksio/api/routes/runs.py

@@ -70,6 +70,10 @@ class RunRow(BaseModel):
     cause: str
     params: dict[str, Any]
     params_digest: str
+    #: The user repository's commit, for a flow declared in code with the
+    #: decorators. Empty for one drawn on the canvas, where `commit` is the
+    #: whole answer to what produced the number.
+    origin_commit: str = ""
     seed: int | None
     group_id: str | None
     labels: list[str]

backend/fluksio/cli.py

@@ -1,4 +1,4 @@
-"""`fluksio serve`, `fluksio enroll`, `fluksio worker`.
+"""`fluksio serve`, `fluksio enroll`, `fluksio worker`, `sync`, `run`, `runs`.
 
 The point of this module is a machine nobody can route to: a node on a cluster
 where ports cannot be opened, or a laptop with no Docker. `fluksio serve`
@@ -268,6 +268,11 @@ def _parser() -> argparse.ArgumentParser:
         help="run nodes for an engine elsewhere (fluksio-worker)",
         add_help=False,
     )
+
+    # The client half: talking to an engine rather than being one.
+    from fluksio.sdk.cli import add_parsers
+
+    add_parsers(subparsers)
     return parser
 
 
@@ -276,7 +281,13 @@ def main(argv: list[str] | None = None) -> int:
     # Everything after `worker` belongs to the agent's own parser.
     if argv and argv[0] == "worker":
         return cmd_worker(argv[1:])
-    args = _parser().parse_args(argv)
+    parser = _parser()
+    if argv and argv[0] == "run":
+        # A flow's inputs are its own, so `--lr 0.05` cannot be declared here:
+        # whatever this parser does not know is typed against the flow.
+        args, rest = parser.parse_known_args(argv)
+        return int(args.func(args, rest))
+    args = parser.parse_args(argv)
     result: int = args.func(args)
     return result
 

backend/fluksio/flow/runs.py

@@ -276,6 +276,18 @@ class MetricSink:
             )
 
 
+def origin_commit(flow: FlowDef) -> str:
+    """The user repository's commit, for a flow declared in code elsewhere.
+
+    Marked dirty when the tree had uncommitted changes at sync, because then
+    the hash names something other than what actually ran.
+    """
+    origin = flow.origin
+    if origin is None or not origin.commit:
+        return ""
+    return f"{origin.commit}-dirty" if origin.dirty else origin.commit
+
+
 class RunService:
     """Accepts runs, drives them, and writes down what they did."""
 
@@ -360,6 +372,7 @@ class RunService:
             flow=flow.name,
             flow_version=flow.version,
             commit=self.controller.store.head(),
+            origin_commit=origin_commit(flow),
             params=params,
             params_digest=digest_of(params, seed),
             seed=seed,

backend/fluksio/flow/schemas.py

@@ -77,6 +77,30 @@ class NodeDef(BaseModel):
         return _validate_name(value)
 
 
+class FlowOrigin(BaseModel):
+    """Where a flow was declared, when that was somewhere other than here.
+
+    A flow drawn on the canvas has no origin: the store is where it lives. One
+    stamped with this was declared with the decorators in somebody's own
+    repository and put here by ``fluksio sync``, so the node bodies below it
+    are generated imports and the code they run is versioned twice — once here
+    and once there. Its presence is what makes a flow code-defined.
+
+    Deliberately no timestamp. The store commits every change it is given, so
+    when a flow was last synced is a fact its own history already holds — and
+    one that would otherwise change on every sync, making an unchanged upload
+    look like a new version of the flow.
+    """
+
+    kind: Literal["python"] = "python"
+    #: The repository root on the machine that ran ``sync``.
+    repo: str = ""
+    #: Its commit, and whether the tree had uncommitted changes at the time —
+    #: a run stamped with a dirty commit names code that was never stored.
+    commit: str = ""
+    dirty: bool = False
+
+
 class FlowInput(BaseModel):
     """A message the flow starts with rather than computes."""
 
@@ -108,6 +132,13 @@ class FlowDef(BaseModel):
             "means every message the flow ends up holding."
         ),
     )
+    origin: FlowOrigin | None = Field(
+        default=None,
+        description=(
+            "Set when the flow was declared in code elsewhere and uploaded by "
+            "`fluksio sync`. Absent for a flow drawn on the canvas."
+        ),
+    )
 
     @field_validator("name")
     @classmethod

backend/fluksio/models.py

@@ -326,6 +326,11 @@ class Run(SQLModel, table=True):
     #: can be traced back to the code that produced it.
     flow_version: int = 1
     commit: str = Field(default="", max_length=64)
+    #: The commit of the *user's* repository, for a flow declared there with
+    #: the decorators. The store's commit names a generated import shim; this
+    #: one names the code it imported. A `-dirty` suffix is git's own way of
+    #: saying the tree had changes that are in no commit at all.
+    origin_commit: str = Field(default="", max_length=80)
     params: dict[str, Any] = Field(sa_column=Column(JSON), default_factory=dict)
     #: sha256 over params and seed. Two runs of the same thing share it, which
     #: is what makes "have I already run this?" a lookup.

backend/fluksio/sdk/__init__.py

@@ -0,0 +1,657 @@
+"""Declare flows and nodes in the repository you already have.
+
+A research repository keeps its functions where they are; the decorators here
+say which of them are nodes and which nodes make up a flow. ``fluksio sync``
+uploads the flow document and, per node, a generated import shim — so the flow
+store still holds a complete, runnable, git-versioned definition, and the code
+it imports stays yours.
+
+Nothing here imports the engine. A ``from fluksio import node`` in a training
+script must not drag FastAPI and SQLAlchemy in behind it, and the same names
+have to resolve inside a worker, where ``fluksio`` is the reporter module.
+"""
+
+from __future__ import annotations
+
+import inspect
+import json
+import re
+import warnings
+from collections.abc import Callable, Sequence
+from pathlib import Path
+from typing import Any, TypeVar
+
+__all__ = [
+    "FLOWS",
+    "Flow",
+    "Port",
+    "SyncError",
+    "build",
+    "node",
+    "shims",
+    "use",
+]
+
+F = TypeVar("F", bound=Callable[..., Any])
+
+#: Mirrors :class:`fluksio.flow.messages.DType`. Mirrored rather than imported:
+#: importing it would pull the engine into a research process.
+DTYPES = frozenset(
+    {"float", "int", "str", "bool", "json", "series", "record", "list", "artifact"}
+)
+#: What a list may hold, mirroring ``messages._ITEM_TYPES``.
+ITEM_TYPES = frozenset({"float", "int", "str", "bool", "json", "record"})
+#: Settings the engine reads itself, mirroring ``nodes.base.RESERVED_SETTINGS``.
+RESERVED_SETTINGS = frozenset({"synchronous"})
+#: Mirrors ``fluksio.flow.schemas.NAME_PATTERN``.
+NAME_PATTERN = re.compile(r"^[a-z][a-z0-9_]*$")
+#: First line of every generated node body. Its absence is how ``sync`` knows a
+#: node's source was edited on the canvas.
+MARKER = "# generated by fluksio sync"
+
+
+class SyncError(Exception):
+    """A declaration the engine would refuse, caught where it was written."""
+
+
+#: Every flow declared in the modules imported so far. ``fluksio sync`` reads it.
+FLOWS: dict[str, Flow] = {}
+
+
+def _name(value: str, what: str) -> str:
+    if not NAME_PATTERN.match(value):
+        raise SyncError(
+            f"{what} '{value}': use lowercase letters, digits and underscores, "
+            "starting with a letter"
+        )
+    return value
+
+
+class Port:
+    """One port of a node, and the message it binds to.
+
+    A thin mirror of :class:`fluksio.flow.messages.MessageSpec`. ``name`` is
+    the wiring — two nodes are connected because one provides the name another
+    requires — and ``port`` is what the function calls its parameter, which
+    defaults to the name.
+    """
+
+    def __init__(
+        self,
+        name: str,
+        dtype: str | None = None,
+        *,
+        port: str = "",
+        stream: bool = False,
+        initial: Any = None,
+        item: str | None = None,
+        interval: float = 0.0,
+        trigger: bool = True,
+    ) -> None:
+        #: Whether the caller said the type or left it to be resolved from
+        #: whoever provides the message. A message has one type, declared where
+        #: it is produced; requiring it does not mean declaring it again.
+        self.explicit = dtype is not None
+        dtype = dtype or "float"
+        if dtype not in DTYPES:
+            raise SyncError(f"port '{name}': unknown type '{dtype}'")
+        if item is not None and item not in ITEM_TYPES:
+            raise SyncError(f"port '{name}': a list cannot hold '{item}' items")
+        if interval < 0:
+            raise SyncError(f"port '{name}': interval cannot be negative")
+        self.name = name
+        self.dtype = dtype
+        self.port = port or name.rsplit(".", 1)[-1]
+        self.stream = stream
+        self.initial = initial
+        self.item = item
+        self.interval = interval
+        self.trigger = trigger
+        if not self.port.isidentifier():
+            raise SyncError(
+                f"port '{name}': '{self.port}' is not a Python identifier, so no "
+                "parameter can carry it — pass port='...'"
+            )
+
+    def spec(self) -> dict[str, Any]:
+        """The ``MessageSpec`` this port serialises to."""
+        out: dict[str, Any] = {
+            "name": self.name,
+            "port": self.port,
+            "dtype": self.dtype,
+            "interval": self.interval,
+            "trigger": self.trigger,
+            "stream": self.stream,
+        }
+        if self.item is not None:
+            out["item"] = self.item
+        return out
+
+    def replace(self, name: str = "") -> Port:
+        """A copy of this port, optionally bound to a different message."""
+        copy = Port(
+            name or self.name,
+            self.dtype,
+            port=self.port,
+            stream=self.stream,
+            initial=self.initial,
+            item=self.item,
+            interval=self.interval,
+            trigger=self.trigger,
+        )
+        copy.explicit = self.explicit
+        return copy
+
+    def __repr__(self) -> str:
+        return f"Port({self.name!r}, {self.dtype!r})"
+
+
+def _ports(value: Sequence[str | Port] | str | Port) -> tuple[list[Port], bool]:
+    """Normalise a ports declaration; the flag says it was a single port.
+
+    A single ``Port`` rather than a list of one is the opt-in to a bare return:
+    the shim wraps the value in the dict the engine expects. It has to be
+    explicit, because an artifact reference is itself a dict and a dict whose
+    keys match no port is dropped in silence.
+    """
+    if isinstance(value, (str, Port)):
+        return [value if isinstance(value, Port) else Port(value)], True
+    return [item if isinstance(item, Port) else Port(item) for item in value], False
+
+
+class NodeSpec:
+    """What ``@node`` recorded about a function."""
+
+    def __init__(
+        self,
+        fn: Callable[..., Any],
+        *,
+        id: str,
+        requires: list[Port],
+        provides: list[Port],
+        single: bool,
+        settings: dict[str, Any],
+        title: str,
+        timeout: float | None,
+        device: str | None,
+        device_policy: str,
+    ) -> None:
+        self.fn = fn
+        self.id = id
+        self.requires = requires
+        self.provides = provides
+        self.single = single
+        self.settings = settings
+        self.title = title
+        self.timeout = timeout
+        self.device = device
+        self.device_policy = device_policy
+
+    def rebind(
+        self, *, id: str = "", wire: dict[str, str] | None = None, **settings: Any
+    ) -> NodeSpec:
+        """A copy of this node wired and configured for one flow."""
+        wire = wire or {}
+        known = {port.port for port in self.requires} | {p.port for p in self.provides}
+        unknown = sorted(set(wire) - known)
+        if unknown:
+            raise SyncError(
+                f"node '{self.id}': wire={{'{unknown[0]}': ...}} names no port of it "
+                f"(it has {', '.join(sorted(known)) or 'none'})"
+            )
+
+        # Copied rather than shared: the same decorated function appears in
+        # several flows, and each resolves its own port types.
+        def rewire(ports: list[Port]) -> list[Port]:
+            return [port.replace(wire.get(port.port, "")) for port in ports]
+
+        return NodeSpec(
+            self.fn,
+            id=_name(id, "node id") if id else self.id,
+            requires=rewire(self.requires),
+            provides=rewire(self.provides),
+            single=self.single,
+            settings={**self.settings, **settings},
+            title=self.title,
+            timeout=self.timeout,
+            device=self.device,
+            device_policy=self.device_policy,
+        )
+
+
+class NodeUse:
+    """One use of a decorated function in one flow — see :func:`use`."""
+
+    def __init__(
+        self,
+        fn: Callable[..., Any],
+        *,
+        id: str = "",
+        wire: dict[str, str] | None = None,
+        settings: dict[str, Any] | None = None,
+    ) -> None:
+        self.fn = fn
+        self.id = id
+        self.wire = wire
+        self.settings = settings or {}
+
+
+def node(
+    *,
+    requires: Sequence[str | Port] = (),
+    provides: Sequence[str | Port] | str | Port = (),
+    id: str = "",
+    settings: dict[str, Any] | None = None,
+    title: str = "",
+    timeout: float | None = None,
+    device: str | None = None,
+    device_policy: str = "require",
+) -> Callable[[F], F]:
+    """Mark a function as a node, declaring its ports.
+
+    The function is returned untouched: it is still callable, testable and
+    importable as what it was. A bare string in ``requires`` or ``provides`` is
+    shorthand for a ``float`` port of that name.
+
+    Parameters with a default that are not ports become the node's settings, so
+    the canvas can tune them without editing code. ``device`` picks the worker
+    the code runs on — ``"gpu"``, say — and ``device_policy="prefer"`` runs it
+    locally when no such worker is attached rather than waiting for one.
+    """
+    if device_policy not in ("require", "prefer"):
+        raise SyncError("device_policy is 'require' or 'prefer'")
+
+    def decorate(fn: F) -> F:
+        required, _ = _ports(requires)
+        provided, single = _ports(provides)
+        spec = NodeSpec(
+            fn,
+            id=_name(id or fn.__name__, "node id"),
+            requires=required,
+            provides=provided,
+            single=single,
+            settings=dict(settings or {}),
+            title=title,
+            timeout=timeout,
+            device=device,
+            device_policy=device_policy,
+        )
+        _check_signature(spec)
+        fn.__fluksio__ = spec  # type: ignore[attr-defined]
+        return fn
+
+    return decorate
+
+
+def use(
+    fn: Callable[..., Any],
+    *,
+    id: str = "",
+    wire: dict[str, str] | None = None,
+    **settings: Any,
+) -> NodeUse:
+    """The same node, wired or configured differently in this flow.
+
+    ``wire={"dataset": "augmented"}`` binds the ``dataset`` port to another
+    message; keyword arguments override settings. Give ``id`` when one function
+    appears twice in one flow.
+    """
+    return NodeUse(fn, id=id, wire=wire, settings=settings)
+
+
+def _spec_of(entry: Callable[..., Any] | NodeUse) -> NodeSpec:
+    fn = entry.fn if isinstance(entry, NodeUse) else entry
+    spec = getattr(fn, "__fluksio__", None)
+    if not isinstance(spec, NodeSpec):
+        raise SyncError(
+            f"{getattr(fn, '__name__', fn)!r} is not a node — decorate it with @node()"
+        )
+    if isinstance(entry, NodeUse):
+        return spec.rebind(id=entry.id, wire=entry.wire, **entry.settings)
+    return spec.rebind()
+
+
+def _check_signature(spec: NodeSpec) -> None:
+    """Whether the declaration matches the function it is on.
+
+    Raised where the decorator is rather than where the flow is: this is a
+    disagreement between a function and its own ports, and the engine would
+    otherwise only find it when the node is first called.
+    """
+    fn, where = spec.fn, f"node '{spec.id}'"
+    if inspect.iscoroutinefunction(fn):
+        raise SyncError(f"{where}: async functions cannot be nodes")
+    if fn.__module__ == "__main__":
+        raise SyncError(
+            f"{where}: {fn.__name__}() is defined in a script run directly, so the "
+            "generated node could not import it — put it in an importable module"
+        )
+    generator = inspect.isgeneratorfunction(fn)
+    if generator and spec.single:
+        raise SyncError(
+            f"{where}: {fn.__name__}() yields, so it says which port each value is "
+            f"on — pass provides=[Port(...)] rather than a single port"
+        )
+
+    signature = inspect.signature(fn)
+    parameters = signature.parameters
+    var_keyword = any(
+        p.kind is inspect.Parameter.VAR_KEYWORD for p in parameters.values()
+    )
+    ports = {port.port for port in spec.requires}
+
+    for port in ports:
+        if port not in parameters and not var_keyword:
+            raise SyncError(
+                f"{where}: port '{port}' has no parameter to arrive in — "
+                f"{fn.__name__}({', '.join(parameters)}) takes no such argument"
+            )
+    for name, parameter in parameters.items():
+        if parameter.kind is inspect.Parameter.VAR_POSITIONAL:
+            raise SyncError(f"{where}: *{name} cannot be filled from ports")
+        if parameter.kind is inspect.Parameter.VAR_KEYWORD or name in ports:
+            continue
+        if parameter.default is inspect.Parameter.empty:
+            raise SyncError(
+                f"{where}: parameter '{name}' is neither a port nor a setting — "
+                f"add Port('{name}', ...) to requires=, or give it a default"
+            )
+
+
+def _check(spec: NodeSpec, mode: str) -> dict[str, Any]:
+    """The node as stored, with the settings and wiring of one flow.
+
+    Mirrors ``FlowController._build_node`` and ``flow.runs.batch_issues``. Kept
+    apart from the signature check because ``use(fn, epochs=3)`` adds settings
+    per flow, so this is the first point at which they are all known.
+    """
+    fn, where = spec.fn, f"node '{spec.id}'"
+    parameters = inspect.signature(fn).parameters
+    var_keyword = any(
+        p.kind is inspect.Parameter.VAR_KEYWORD for p in parameters.values()
+    )
+    ports = {port.port for port in spec.requires}
+
+    params: dict[str, Any] = {}
+    for name, parameter in parameters.items():
+        if name in ports or parameter.default is inspect.Parameter.empty:
+            continue
+        if name in RESERVED_SETTINGS:
+            continue
+        if _serialisable(parameter.default):
+            params[name] = parameter.default
+    for name, value in spec.settings.items():
+        if name in ports:
+            raise SyncError(
+                f"{where}: '{name}' is both a port and a setting — a node reads its "
+                "settings beside its ports, so the names have to differ"
+            )
+        if name in RESERVED_SETTINGS:
+            raise SyncError(f"{where}: '{name}' is a setting the engine reads itself")
+        if name not in parameters and not var_keyword:
+            raise SyncError(
+                f"{where}: setting '{name}' is not a parameter of {fn.__name__}()"
+            )
+        if not _serialisable(value):
+            raise SyncError(
+                f"{where}: setting '{name}' is not JSON — settings are stored with "
+                "the flow, so they have to survive a round trip"
+            )
+        params[name] = value
+
+    if mode == "batch":
+        for port in spec.requires + spec.provides:
+            if port.interval > 0 and not port.stream:
+                raise SyncError(
+                    f"{where}: port '{port.port}' has interval={port.interval} but a "
+                    "batch run delivers once — drop it, or declare stream=True"
+                )
+
+    return {
+        "id": spec.id,
+        "type": "python",
+        "title": spec.title,
+        "params": params,
+        "requires": [port.spec() for port in spec.requires],
+        "provides": [port.spec() for port in spec.provides],
+        "timeout": spec.timeout,
+        "device": spec.device,
+        "device_policy": spec.device_policy,
+    }
+
+
+def _serialisable(value: Any) -> bool:
+    try:
+        json.dumps(value)
+    except (TypeError, ValueError):
+        return False
+    return True
+
+
+class Flow:
+    """A flow: which nodes are in it, what it takes and what it reports."""
+
+    def __init__(
+        self,
+        name: str,
+        *,
+        nodes: Sequence[Callable[..., Any] | NodeUse],
+        mode: str = "batch",
+        inputs: Sequence[Port] = (),
+        outputs: Sequence[str] = (),
+        title: str = "",
+    ) -> None:
+        """Declare a flow over decorated functions, and register it for sync.
+
+        ``nodes`` is the membership: any decorated function, from anywhere in
+        the repository. Wiring is not membership — nodes connect because one
+        provides a message another requires, never because one imported the
+        other. Wrap an entry in :func:`use` to rebind or reconfigure it here
+        without touching the other flows using it.
+        """
+        if mode not in ("batch", "live"):
+            raise SyncError(f"flow '{name}': mode is 'batch' or 'live'")
+        self.name = _name(name, "flow name")
+        self.title = title
+        self.mode = mode
+        self.inputs = list(inputs)
+        self.outputs = list(outputs)
+        self.nodes = [_spec_of(entry) for entry in nodes]
+        self._resolve_types()
+        self._validate()
+        self._nodes = [_check(spec, mode) for spec in self.nodes]
+        self._register()
+
+    def _register(self) -> None:
+        """Put this flow where `fluksio sync` will find it."""
+        existing = FLOWS.get(self.name)
+        if existing is not None and existing is not self:
+            warnings.warn(
+                f"two flows are called '{self.name}'; the later one wins, and only "
+                "it will be synced",
+                stacklevel=3,
+            )
+        FLOWS[self.name] = self
+
+    def _resolve_types(self) -> None:
+        """Give every required port the type of the message it reads.
+
+        A message has one type, declared once by whoever produces it. Requiring
+        it is not a second declaration, so `requires=["dataset"]` takes the
+        artifact type from the node providing `dataset` rather than defaulting
+        to a float the engine would then refuse at run time. Spelling the type
+        out on both sides is still allowed — and a disagreement is an error,
+        because one of the two is wrong about what it is handling.
+        """
+        declared: dict[str, Port] = {}
+        for port in self.inputs:
+            declared.setdefault(port.name, port)
+        for spec in self.nodes:
+            for port in spec.provides:
+                declared.setdefault(port.name, port)
+        for spec in self.nodes:
+            for port in spec.requires:
+                source = declared.get(port.name)
+                if source is None:
+                    continue
+                if not port.explicit:
+                    port.dtype = source.dtype
+                    port.item = source.item
+                elif port.dtype != source.dtype:
+                    raise SyncError(
+                        f"flow '{self.name}': node '{spec.id}' reads '{port.name}' as "
+                        f"{port.dtype}, but it is provided as {source.dtype}"
+                    )
+
+    def _validate(self) -> None:
+        seen: set[str] = set()
+        for spec in self.nodes:
+            if spec.id in seen:
+                raise SyncError(
+                    f"flow '{self.name}': two nodes are called '{spec.id}' — "
+                    "pass use(fn, id='...') to tell them apart"
+                )
+            seen.add(spec.id)
+
+        produced: dict[str, list[str]] = {}
+        for spec in self.nodes:
+            for port in spec.provides:
+                produced.setdefault(port.name, []).append(spec.id)
+        for name, writers in produced.items():
+            if len(writers) > 1:
+                warnings.warn(
+                    f"flow '{self.name}': {', '.join(writers)} all publish "
+                    f"'{name}', so a reader gets whichever ran last",
+                    stacklevel=4,
+                )
+
+        available = set(produced) | {port.name for port in self.inputs}
+        for spec in self.nodes:
+            for port in spec.requires:
+                if "." in port.name or port.name in available:
+                    continue
+                raise SyncError(
+                    f"flow '{self.name}': node '{spec.id}' requires '{port.name}', "
+                    "which no node in it provides and no input declares"
+                )
+        for output in self.outputs:
+            if output not in available:
+                raise SyncError(
+                    f"flow '{self.name}': output '{output}' is not produced by any "
+                    "of its nodes"
+                )
+
+    def document(self, origin: dict[str, Any] | None = None) -> dict[str, Any]:
+        """The flow document to store, without a version — sync sets that."""
+        doc: dict[str, Any] = {
+            "name": self.name,
+            "title": self.title,
+            "mode": self.mode,
+            "nodes": self._nodes,
+            "inputs": [
+                {"spec": port.spec(), "initial": port.initial} for port in self.inputs
+            ],
+            "outputs": self.outputs,
+        }
+        if origin is not None:
+            doc["origin"] = origin
+        return doc
+
+    def shims(self) -> dict[str, str]:
+        """One generated node body per node, keyed by node id."""
+        return {spec.id: _shim(spec) for spec in self.nodes}
+
+    def submit(
+        self, *, seed: int | None = None, client: Any = None, **params: Any
+    ) -> Any:
+        """Start a run of this flow and hand back a handle to it."""
+        from fluksio.sdk.client import Client
+
+        return (client or Client()).submit(self.name, params, seed=seed)
+
+    def runs(self, limit: int = 20, client: Any = None) -> list[dict[str, Any]]:
+        """This flow's runs, newest first."""
+        from fluksio.sdk.client import Client
+
+        rows: list[dict[str, Any]] = (client or Client()).runs(
+            flow=self.name, limit=limit
+        )
+        return rows
+
+    def __repr__(self) -> str:
+        return f"Flow({self.name!r}, nodes={[n.id for n in self.nodes]})"
+
+
+def build(target: Flow, origin: dict[str, Any] | None = None) -> dict[str, Any]:
+    """The flow document for a flow — checked, without a version."""
+    return target.document(origin)
+
+
+def shims(target: Flow) -> dict[str, str]:
+    """The generated node bodies for a flow."""
+    return target.shims()
+
+
+def import_root(fn: Callable[..., Any]) -> str:
+    """The directory that has to be on the path for ``fn`` to be importable.
+
+    Its module's dotted name says how deep in a package it sits, so counting
+    the segments back up from the file lands on the directory the import
+    resolves against — which is not always the repository root, and is what a
+    generated shim actually needs.
+    """
+    source = inspect.getsourcefile(fn)
+    if not source:
+        return ""
+    path = Path(source).resolve()
+    for _ in fn.__module__.split("."):
+        path = path.parent
+    return str(path)
+
+
+def _shim(spec: NodeSpec) -> str:
+    """The node body the store keeps: an import of the real function.
+
+    The store therefore still holds a complete, runnable definition — the body
+    simply happens to be generated, which is why it says so and says where the
+    real thing is.
+    """
+    fn = spec.fn
+    where = inspect.getsourcefile(fn) or fn.__module__
+    repo = import_root(fn)
+    ports = [port.port for port in spec.requires]
+    signature = ", ".join(ports + ["**settings"])
+    arguments = ", ".join([f"{port}={port}" for port in ports] + ["**settings"])
+    call = f"{fn.__name__}({arguments})"
+
+    lines = [
+        f"{MARKER} from {where} — edit that file instead",
+        "import sys",
+        "",
+    ]
+    if repo:
+        # ponytail: the repository is on the path because the shim puts it
+        # there. An editable install (`-e /repo` in the Modules manifest, or a
+        # VCS requirement on a remote worker) makes these three lines a no-op.
+        lines += [
+            f"_REPO = {repo!r}",
+            "if _REPO not in sys.path:",
+            "    sys.path.insert(0, _REPO)",
+            "",
+        ]
+    lines += [
+        f"from {fn.__module__} import {fn.__name__}",
+        "",
+        "",
+        f"def process({signature}):",
+    ]
+    if inspect.isgeneratorfunction(fn):
+        # `return (yield from ...)`, not a bare `yield from`: the delegating
+        # form is what carries the generator's own return value out, and that
+        # return value is everything the node produces at the end.
+        lines.append(f"    return (yield from {call})")
+    elif spec.single:
+        lines.append(f"    return {{{spec.provides[0].port!r}: {call}}}")
+    else:
+        lines.append(f"    return {call}")
+    return "\n".join(lines) + "\n"

backend/fluksio/sdk/cli.py

@@ -0,0 +1,306 @@
+"""The `fluksio sync`, `run`, `runs` and `login` commands.
+
+Kept beside the SDK rather than in `fluksio.cli`: these are the client half of
+the tool, and none of them needs the engine to be importable.
+"""
+
+from __future__ import annotations
+
+import argparse
+import getpass
+import importlib
+import json
+import pkgutil
+import sys
+from pathlib import Path
+from typing import Any
+
+from fluksio.sdk import FLOWS, Flow, SyncError
+from fluksio.sdk.client import (
+    ApiError,
+    Client,
+    config_path,
+    login,
+    origin_of,
+    repo_root,
+    sync,
+)
+
+__all__ = ["add_parsers", "discover"]
+
+
+def _say(message: str = "") -> None:
+    print(message)
+
+
+def _fail(message: str) -> int:
+    print(f"fluksio: {message}", file=sys.stderr)
+    return 1
+
+
+# ---------------------------------------------------------------------------
+# Discovery
+# ---------------------------------------------------------------------------
+
+
+def _package_of(directory: Path) -> tuple[str, str]:
+    """The path root and dotted name of a package directory."""
+    parts = [directory.name]
+    parent = directory.parent
+    while (parent / "__init__.py").exists():
+        parts.append(parent.name)
+        parent = parent.parent
+    return str(parent), ".".join(reversed(parts))
+
+
+def _module_of(path: Path) -> tuple[str, str]:
+    """The path root and dotted name of a module file."""
+    parts = [path.stem]
+    directory = path.parent
+    while (directory / "__init__.py").exists():
+        parts.append(directory.name)
+        directory = directory.parent
+    return str(directory), ".".join(reversed(parts))
+
+
+def _import(root: str, dotted: str) -> None:
+    if root not in sys.path:
+        sys.path.insert(0, root)
+    importlib.import_module(dotted)
+
+
+def discover(targets: list[str]) -> list[Flow]:
+    """Import what was named and hand back the flows it declared.
+
+    Imported by dotted name with its root on the path, never from a file
+    location: the generated node bodies import the same way, and a module
+    loaded under a different name would generate an import that does not
+    resolve.
+    """
+    for target in targets:
+        path = Path(target)
+        if not path.exists():
+            _import(str(Path.cwd()), target)
+            continue
+        path = path.resolve()
+        if path.is_file():
+            _import(*_module_of(path))
+            continue
+        if (path / "__init__.py").exists():
+            root, dotted = _package_of(path)
+            _import(root, dotted)
+            package = sys.modules[dotted]
+            for info in pkgutil.walk_packages(package.__path__, f"{dotted}."):
+                importlib.import_module(info.name)
+            continue
+        for module in sorted(path.glob("*.py")):
+            _import(*_module_of(module))
+    return list(FLOWS.values())
+
+
+# ---------------------------------------------------------------------------
+# Commands
+# ---------------------------------------------------------------------------
+
+
+def cmd_login(args: argparse.Namespace) -> int:
+    email = args.email or input("Email: ")
+    password = args.password or getpass.getpass("Password: ")
+    try:
+        login(args.url, email, password)
+    except ApiError as exc:
+        return _fail(f"could not log in: {exc.detail}")
+    _say(f"Logged in to {args.url}; the token is in {config_path()}.")
+    return 0
+
+
+def cmd_sync(args: argparse.Namespace) -> int:
+    targets = args.targets or ["."]
+    try:
+        flows = discover(targets)
+    except (ImportError, SyncError) as exc:
+        return _fail(str(exc))
+    if not flows:
+        return _fail(
+            f"no flows declared in {', '.join(targets)} — a flow is a `flow(...)` "
+            "call at module level"
+        )
+
+    repo = repo_root(targets[0])
+    origin = origin_of(repo)
+    if origin["dirty"]:
+        _say(f"warning: {repo} has uncommitted changes, so the stamp says -dirty")
+    if not origin["commit"]:
+        _say(f"warning: {repo} is not a git repository, so runs cannot name a commit")
+
+    if args.dry_run:
+        for target in flows:
+            _say(f"=== flow {target.name}")
+            _say(json.dumps(target.document(origin), indent=2))
+            for node_id, code in target.shims().items():
+                _say(f"=== {target.name}.{node_id}")
+                _say(code)
+        return 0
+
+    try:
+        client = Client(url=args.url, token=args.token)
+        reports = sync(
+            flows,
+            client,
+            origin=origin,
+            publish=not args.no_publish,
+            force=args.force,
+        )
+    except (SyncError, ApiError) as exc:
+        return _fail(str(exc))
+
+    for report in reports:
+        if report.unchanged:
+            _say(f"  {report.flow}: unchanged")
+            continue
+        what = "created" if report.created else "updated"
+        detail = ", ".join(report.changed)
+        state = "published" if report.published else "draft"
+        _say(f"  {report.flow}: {what} ({detail}) — {state}")
+    stamp = origin["commit"][:7] + ("-dirty" if origin["dirty"] else "")
+    _say(f"Stamped with {stamp or 'no commit'} from {repo}.")
+    return 0
+
+
+def _coerce(value: str, dtype: str) -> Any:
+    if dtype == "int":
+        return int(value)
+    if dtype == "float":
+        return float(value)
+    if dtype == "bool":
+        return value.lower() in ("true", "1", "yes", "on")
+    if dtype == "str":
+        return value
+    return json.loads(value)
+
+
+def _params(definition: dict[str, Any], rest: list[str]) -> dict[str, Any]:
+    """Turn `--lr 0.05` into a typed parameter, using the flow's own inputs."""
+    types = {
+        str(entry["spec"]["name"]): str(entry["spec"].get("dtype", "float"))
+        for entry in definition.get("inputs") or []
+    }
+    params: dict[str, Any] = {}
+    pending: str | None = None
+    for token in rest:
+        if token.startswith("--"):
+            if pending is not None:
+                # A flag with no value is a flag: `--resume` means true.
+                params[pending] = True
+            name, sep, value = token[2:].partition("=")
+            # Only the name is spelled with dashes; a value may hold one, and
+            # `--lr=1e-4` is the case that says so.
+            pending = name.replace("-", "_")
+            if sep:
+                params[pending] = _coerce(value, types.get(pending, "json"))
+                pending = None
+            continue
+        if pending is None:
+            raise SyncError(f"unexpected argument '{token}'")
+        params[pending] = _coerce(token, types.get(pending, "json"))
+        pending = None
+    if pending is not None:
+        params[pending] = True
+    unknown = sorted(set(params) - set(types))
+    if unknown:
+        raise SyncError(
+            f"'{unknown[0]}' is not an input of this flow (it takes "
+            f"{', '.join(sorted(types)) or 'none'})"
+        )
+    return params
+
+
+def cmd_run(args: argparse.Namespace, rest: list[str]) -> int:
+    try:
+        client = Client(url=args.url, token=args.token)
+        stored = client.get_flow(args.flow)
+        if stored is None:
+            return _fail(f"no flow '{args.flow}' on that engine")
+        params = _params(stored.get("definition") or {}, rest)
+        handle = client.submit(args.flow, params, seed=args.seed)
+    except (SyncError, ApiError) as exc:
+        return _fail(str(exc))
+    _say(f"{handle.id}  queued  {json.dumps(params)}")
+    if not args.wait:
+        return 0
+    handle.wait(timeout=args.timeout)
+    _say(f"{handle.id}  {handle.status}  {json.dumps(handle.result)}")
+    return 0 if handle.status == "ok" else 1
+
+
+def cmd_runs(args: argparse.Namespace) -> int:
+    try:
+        rows = Client(url=args.url, token=args.token).runs(
+            flow=args.flow, limit=args.limit
+        )
+    except (SyncError, ApiError) as exc:
+        return _fail(str(exc))
+    for row in rows:
+        commit = (row.get("origin_commit") or "")[:7]
+        _say(
+            f"{row['id']}  {row['status']:<9} {row['flow']:<16} "
+            f"{row['duration_ms'] / 1000:7.1f}s  {commit:<8} {json.dumps(row['params'])}"
+        )
+    return 0
+
+
+# ---------------------------------------------------------------------------
+# Wiring
+# ---------------------------------------------------------------------------
+
+
+def add_parsers(subparsers: Any) -> None:
+    """Register the client commands on `fluksio`'s parser."""
+
+    def with_engine(sub: argparse.ArgumentParser) -> None:
+        sub.add_argument(
+            "--url", default="", help="the engine (default: the last login)"
+        )
+        sub.add_argument("--token", default="", help="override the stored token")
+
+    parser = subparsers.add_parser("login", help="store a token for an engine")
+    parser.add_argument("--url", default="http://localhost:8000")
+    parser.add_argument("--email", default="")
+    parser.add_argument("--password", default="")
+    parser.set_defaults(func=cmd_login)
+
+    parser = subparsers.add_parser(
+        "sync", help="upload the flows declared in your own code"
+    )
+    parser.add_argument(
+        "targets",
+        nargs="*",
+        help="modules, packages or directories to import (default: .)",
+    )
+    parser.add_argument(
+        "--dry-run", action="store_true", help="print what would be uploaded"
+    )
+    parser.add_argument(
+        "--no-publish", action="store_true", help="leave the changes as a draft"
+    )
+    parser.add_argument(
+        "--force", action="store_true", help="overwrite work done on the canvas"
+    )
+    with_engine(parser)
+    parser.set_defaults(func=cmd_sync)
+
+    parser = subparsers.add_parser(
+        "run", help="start a run, passing the flow's inputs as --name value"
+    )
+    parser.add_argument("flow")
+    parser.add_argument("--seed", type=int, default=None)
+    parser.add_argument("--wait", action="store_true", help="block until it finishes")
+    parser.add_argument("--timeout", type=float, default=0.0)
+    with_engine(parser)
+    parser.set_defaults(func=cmd_run)
+
+    parser = subparsers.add_parser("runs", help="the runs an engine has recorded")
+    parser.add_argument("--flow", default="")
+    parser.add_argument("--limit", type=int, default=20)
+    with_engine(parser)
+    parser.set_defaults(func=cmd_runs)

backend/fluksio/sdk/client.py

@@ -0,0 +1,411 @@
+"""Talking to an engine: uploading declared flows, and running them.
+
+Two halves of one job. :func:`sync` puts what the decorators declared into the
+flow store; :class:`Client` submits runs and reads back what they produced, so
+an experiment is started and inspected from the same script that defines it.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import subprocess
+import time
+from collections.abc import Iterable
+from pathlib import Path
+from typing import Any
+
+from fluksio.sdk import MARKER, Flow, SyncError
+
+__all__ = ["Client", "RunHandle", "SyncReport", "config_path", "login", "sync"]
+
+API = "/api/v1"
+#: A run is over when it reaches one of these.
+DONE = frozenset({"ok", "error", "cancelled", "abandoned"})
+
+
+def config_path() -> Path:
+    """Where ``fluksio login`` leaves the engine it talked to."""
+    base = os.environ.get("XDG_CONFIG_HOME") or str(Path.home() / ".config")
+    return Path(base) / "fluksio" / "client.json"
+
+
+def _stored() -> dict[str, str]:
+    path = config_path()
+    if not path.exists():
+        return {}
+    try:
+        data = json.loads(path.read_text())
+    except ValueError:
+        return {}
+    return data if isinstance(data, dict) else {}
+
+
+class ApiError(Exception):
+    """The engine refused, with what it said."""
+
+    def __init__(self, status: int, detail: Any) -> None:
+        super().__init__(f"{status}: {detail}")
+        self.status = status
+        self.detail = detail
+
+
+class Conflict(ApiError):
+    """Someone else saved in between — the 409 the optimistic lock answers."""
+
+    def __init__(self, detail: Any) -> None:
+        super().__init__(409, detail)
+        self.current_version = 0
+        if isinstance(detail, dict):
+            self.current_version = int(detail.get("current_version") or 0)
+
+
+class Client:
+    """An authenticated engine, addressed over its HTTP API."""
+
+    def __init__(
+        self, url: str = "", token: str = "", http: Any = None, timeout: float = 30.0
+    ) -> None:
+        stored = _stored()
+        self.url = url or os.environ.get("FLUKSIO_URL") or stored.get("url") or ""
+        self.token = (
+            token or os.environ.get("FLUKSIO_TOKEN") or stored.get("token") or ""
+        )
+        if http is None:
+            if not self.url:
+                raise SyncError(
+                    "No engine to talk to. Run `fluksio login --url http://…`, or "
+                    "set FLUKSIO_URL and FLUKSIO_TOKEN."
+                )
+            import httpx
+
+            http = httpx.Client(base_url=self.url, timeout=timeout)
+        self.http = http
+        if self.token:
+            self.http.headers["Authorization"] = f"Bearer {self.token}"
+
+    # -- plumbing ----------------------------------------------------------
+
+    def _call(self, method: str, path: str, **kwargs: Any) -> Any:
+        response = self.http.request(method, f"{API}{path}", **kwargs)
+        if response.status_code == 409:
+            raise Conflict(_detail(response))
+        if response.status_code >= 400:
+            raise ApiError(response.status_code, _detail(response))
+        if response.status_code == 204 or not response.content:
+            return None
+        return response.json()
+
+    # -- flows -------------------------------------------------------------
+
+    def get_flow(self, name: str) -> dict[str, Any] | None:
+        """The stored flow, draft included, or ``None`` if there is none."""
+        try:
+            result: dict[str, Any] = self._call("GET", f"/flows/{name}")
+        except ApiError as exc:
+            if exc.status == 404:
+                return None
+            raise
+        return result
+
+    def put_flow(self, document: dict[str, Any]) -> dict[str, Any]:
+        result: dict[str, Any] = self._call(
+            "PUT", f"/flows/{document['name']}", json=document
+        )
+        return result
+
+    def get_source(self, flow: str, node: str) -> str:
+        result = self._call("GET", f"/flows/{flow}/nodes/{node}/source")
+        return str(result.get("code", ""))
+
+    def put_source(self, flow: str, node: str, code: str) -> dict[str, Any]:
+        result: dict[str, Any] = self._call(
+            "PUT", f"/flows/{flow}/nodes/{node}/source", json={"code": code}
+        )
+        return result
+
+    def publish(self, name: str, version: int) -> dict[str, Any]:
+        result: dict[str, Any] = self._call(
+            "POST", f"/flows/{name}/publish", json={"version": version}
+        )
+        return result
+
+    def refresh_modules(self) -> None:
+        """Retire the engine's workers, so the next run imports the code as it is."""
+        self._call("POST", "/modules/refresh")
+
+    # -- runs --------------------------------------------------------------
+
+    def submit(
+        self, flow: str, params: dict[str, Any] | None = None, seed: int | None = None
+    ) -> RunHandle:
+        row = self._call(
+            "POST", f"/runs/flows/{flow}", json={"params": params or {}, "seed": seed}
+        )
+        return RunHandle(self, row["id"], row)
+
+    def run(self, run_id: str) -> dict[str, Any]:
+        result: dict[str, Any] = self._call("GET", f"/runs/{run_id}")
+        return result
+
+    def runs(self, flow: str = "", limit: int = 20, **filters: Any) -> Any:
+        query = {"limit": limit, **filters}
+        if flow:
+            query["flow"] = flow
+        return self._call("GET", "/runs", params=query)
+
+    def metrics(self, run_id: str, name: str = "", stride: int = 1) -> Any:
+        query: dict[str, Any] = {"stride": stride}
+        if name:
+            query["name"] = name
+        return self._call("GET", f"/runs/{run_id}/metrics", params=query)
+
+    def compare(self, ids: Iterable[str], metric: str) -> Any:
+        return self._call(
+            "GET",
+            "/runs/series/compare",
+            params={"ids": ",".join(ids), "metric": metric},
+        )
+
+    def cancel(self, run_id: str) -> Any:
+        return self._call("POST", f"/runs/{run_id}/cancel")
+
+    def download(self, digest: str) -> bytes:
+        response = self.http.request("GET", f"{API}/artifacts/{digest}")
+        if response.status_code >= 400:
+            raise ApiError(response.status_code, _detail(response))
+        return bytes(response.content)
+
+
+def _detail(response: Any) -> Any:
+    try:
+        body = response.json()
+    except ValueError:
+        return response.text
+    return body.get("detail", body) if isinstance(body, dict) else body
+
+
+class RunHandle:
+    """One run, and the answers it accumulates."""
+
+    def __init__(self, client: Client, run_id: str, row: dict[str, Any]) -> None:
+        self.client = client
+        self.id = run_id
+        self._row = row
+
+    def refresh(self) -> RunHandle:
+        self._row = self.client.run(self.id)
+        return self
+
+    @property
+    def status(self) -> str:
+        return str(self._row.get("status", ""))
+
+    @property
+    def done(self) -> bool:
+        return self.status in DONE
+
+    @property
+    def result(self) -> dict[str, Any]:
+        """What the flow's outputs held when it finished."""
+        result = self._row.get("result")
+        return result if isinstance(result, dict) else {}
+
+    @property
+    def artifacts(self) -> list[dict[str, Any]]:
+        rows = self._row.get("artifacts")
+        return rows if isinstance(rows, list) else []
+
+    def wait(self, timeout: float = 0.0, poll: float = 1.0) -> RunHandle:
+        """Block until the run is over, or ``timeout`` seconds have passed."""
+        deadline = time.monotonic() + timeout if timeout else 0.0
+        while True:
+            self.refresh()
+            if self.done:
+                return self
+            if deadline and time.monotonic() > deadline:
+                raise TimeoutError(f"run {self.id} is still {self.status}")
+            time.sleep(poll)
+
+    def metrics(self, name: str = "", stride: int = 1) -> list[dict[str, Any]]:
+        """A streamed port's whole series — a run's metrics are its outputs."""
+        points: list[dict[str, Any]] = self.client.metrics(self.id, name, stride)
+        return points
+
+    def download(self, name: str) -> bytes:
+        """The bytes of an artifact this run produced."""
+        for row in self.artifacts:
+            if row.get("name") == name:
+                return self.client.download(str(row["digest"]))
+        have = ", ".join(str(row.get("name")) for row in self.artifacts) or "none"
+        raise KeyError(f"run {self.id} has no artifact '{name}' (it has {have})")
+
+    def __getitem__(self, key: str) -> Any:
+        return self._row[key]
+
+    def __repr__(self) -> str:
+        return f"RunHandle({self.id!r}, status={self.status!r})"
+
+
+def login(url: str, email: str, password: str, timeout: float = 30.0) -> str:
+    """Exchange credentials for a token and remember the engine."""
+    import httpx
+
+    response = httpx.post(
+        f"{url.rstrip('/')}{API}/login/access-token",
+        data={"username": email, "password": password},
+        timeout=timeout,
+    )
+    if response.status_code >= 400:
+        raise ApiError(response.status_code, _detail(response))
+    token = str(response.json()["access_token"])
+    path = config_path()
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(json.dumps({"url": url.rstrip("/"), "token": token}, indent=2))
+    path.chmod(0o600)
+    return token
+
+
+# ---------------------------------------------------------------------------
+# sync
+# ---------------------------------------------------------------------------
+
+
+class SyncReport:
+    """What one flow's sync did, for the CLI to print."""
+
+    def __init__(self, flow: str) -> None:
+        self.flow = flow
+        self.created = False
+        self.changed: list[str] = []
+        self.published = False
+
+    @property
+    def unchanged(self) -> bool:
+        return not self.created and not self.changed
+
+    def __repr__(self) -> str:
+        return f"SyncReport({self.flow!r}, changed={self.changed})"
+
+
+def origin_of(repo: str) -> dict[str, Any]:
+    """Where this flow came from, so a run can name the code that produced it."""
+    commit, dirty = "", False
+    if repo:
+        head = _git(repo, "rev-parse", "HEAD")
+        if head is not None:
+            commit = head
+            dirty = bool(_git(repo, "status", "--porcelain"))
+    return {"kind": "python", "repo": repo, "commit": commit, "dirty": dirty}
+
+
+def _git(repo: str, *args: str) -> str | None:
+    try:
+        result = subprocess.run(
+            ["git", "-C", repo, *args], capture_output=True, text=True, timeout=10
+        )
+    except (OSError, subprocess.SubprocessError):
+        return None
+    return result.stdout.strip() if result.returncode == 0 else None
+
+
+def repo_root(start: str | Path) -> str:
+    """The git repository a path sits in, or its directory if there is none."""
+    directory = Path(start).resolve()
+    if directory.is_file():
+        directory = directory.parent
+    found = _git(str(directory), "rev-parse", "--show-toplevel")
+    return found or str(directory)
+
+
+def sync(
+    flows: Iterable[Flow],
+    client: Client,
+    *,
+    repo: str = "",
+    origin: dict[str, Any] | None = None,
+    publish: bool = True,
+    force: bool = False,
+) -> list[SyncReport]:
+    """Put declared flows into the store, refusing to overwrite canvas work.
+
+    The order matters: the document first, so the nodes it names exist, then
+    each node's generated body, then one publish. Both writes no-op on
+    identical content, so a second sync with nothing changed commits nothing.
+    """
+    stamp = origin if origin is not None else origin_of(repo)
+    reports = []
+    for target in flows:
+        reports.append(_sync_one(target, client, stamp, publish=publish, force=force))
+    # Always, even when nothing here changed. A worker holds the imported
+    # package in `sys.modules` for as long as it lives, so an edit to the
+    # caller's own code is invisible until the process is retired — and that
+    # edit is invisible to this function too, since it changes no shim and no
+    # document. Refusing to refresh on a "no-op" sync would skip exactly the
+    # case the refresh exists for.
+    client.refresh_modules()
+    return reports
+
+
+def _sync_one(
+    target: Flow,
+    client: Client,
+    origin: dict[str, Any],
+    *,
+    publish: bool,
+    force: bool,
+) -> SyncReport:
+    report = SyncReport(target.name)
+    stored = client.get_flow(target.name)
+    version = 1
+    if stored is None:
+        report.created = True
+    else:
+        definition = stored.get("definition") or {}
+        version = int(definition.get("version") or 1)
+        if not force:
+            _refuse_on_drift(client, target, definition)
+
+    saved = client.put_flow(target.document(origin) | {"version": version})
+    stored_version = int((saved.get("definition") or {}).get("version") or version)
+    if report.created or stored_version != version:
+        # The store bumps the version only when the content actually changed,
+        # so this is its own no-op detection rather than a second guess at it.
+        report.changed.append("flow")
+    version = stored_version
+
+    for node_id, code in target.shims().items():
+        if not report.created and client.get_source(target.name, node_id) == code:
+            continue
+        client.put_source(target.name, node_id, code)
+        report.changed.append(node_id)
+
+    if publish and (client.get_flow(target.name) or {}).get("has_draft"):
+        client.publish(target.name, version)
+        report.published = True
+    return report
+
+
+def _refuse_on_drift(client: Client, target: Flow, definition: dict[str, Any]) -> None:
+    """Stop before overwriting work that was done somewhere else.
+
+    Two ways a stored flow is not ours to replace: it was drawn on the canvas
+    and has no origin at all, or one of its node bodies no longer carries the
+    line saying it was generated — which means somebody edited the code there.
+    """
+    if not definition.get("origin"):
+        raise SyncError(
+            f"flow '{target.name}' was not created by sync, so replacing it would "
+            "discard whoever drew it. Rename yours, or pass --force."
+        )
+    for stored_node in definition.get("nodes") or []:
+        node_id = str(stored_node.get("id"))
+        try:
+            code = client.get_source(target.name, node_id)
+        except ApiError:
+            continue
+        if code and not code.startswith(MARKER):
+            raise SyncError(
+                f"node '{target.name}.{node_id}' was edited on the canvas, and "
+                "syncing would throw that edit away. Copy it out, or pass --force."
+            )

backend/pyproject.toml

@@ -103,10 +103,13 @@ ignore = [
 "fluksio/__init__.py" = ["ARG001"]
 # It talks to whoever ran it; that is what a command line is.
 "fluksio/cli.py" = ["T201"]
+"fluksio/sdk/cli.py" = ["T201"]
 # Node functions take `params` whether or not they use it — that is the
 # contract the engine calls them with.
 "fluksio/flow/nodes.py" = ["ARG001", "ARG002"]
 "tests/flow/*" = ["ARG001"]
+# Node functions declared for a test take the ports they declare, used or not.
+"tests/sdk/*" = ["ARG001"]
 # Printing is what this one is about: node code is user code, and `print` is
 # how it says things.
 "tests/flow/test_logs.py" = ["ARG001", "T201"]

backend/tests/api/routes/test_sync.py

@@ -0,0 +1,115 @@
+"""`fluksio sync` against a real engine: what it stores, and what it refuses."""
+
+import sys
+from collections.abc import Generator
+from pathlib import Path
+
+import pytest
+from fastapi.testclient import TestClient
+
+from fluksio.core.config import settings
+from fluksio.sdk import FLOWS, MARKER, SyncError
+from fluksio.sdk.client import Client, sync
+
+EXAMPLES = Path(__file__).parents[4] / "examples"
+PREFIX = settings.API_V1_STR
+ORIGIN = {
+    "kind": "python",
+    "repo": str(EXAMPLES),
+    "commit": "a1b2c3d4",
+    "dirty": False,
+}
+
+
+@pytest.fixture(scope="module")
+def flows() -> Generator[dict, None, None]:
+    """The example research package, imported the way `sync` imports it."""
+    sys.path.insert(0, str(EXAMPLES))
+    FLOWS.clear()
+    import myresearch.pipeline  # noqa: F401
+
+    yield dict(FLOWS)
+    FLOWS.clear()
+    sys.path.remove(str(EXAMPLES))
+
+
+@pytest.fixture
+def api(client: TestClient, superuser_token_headers: dict[str, str]) -> Client:
+    return Client(http=client, token=superuser_token_headers["Authorization"][7:])
+
+
+def commits() -> int:
+    """How many commits the flow store has made."""
+    from fluksio.flow.store import FlowStore
+
+    store = FlowStore(settings.FLOWS_DIR)
+    result = store._git("rev-list", "--count", "HEAD")
+    return int(result.stdout.strip()) if result.returncode == 0 else 0
+
+
+def test_sync_stores_a_runnable_flow_with_its_origin(api, flows):
+    reports = sync([flows["train"]], api, origin=ORIGIN)
+
+    assert [r.flow for r in reports] == ["train"]
+    assert reports[0].created and reports[0].published
+
+    stored = api.get_flow("train")
+    definition = stored["definition"]
+    assert definition["origin"]["commit"] == "a1b2c3d4"
+    assert [n["id"] for n in definition["nodes"]] == ["prepare", "fit", "evaluate"]
+    assert not stored["has_draft"]
+    # The store holds a complete definition, and the bodies say where the real
+    # code is rather than being a copy of it.
+    source = api.get_source("train", "fit")
+    assert source.startswith(MARKER)
+    assert "from myresearch.train import fit" in source
+
+
+def test_a_second_sync_changes_nothing_and_commits_nothing(api, flows):
+    sync([flows["train"]], api, origin=ORIGIN)
+    before = commits()
+
+    reports = sync([flows["train"]], api, origin=ORIGIN)
+
+    assert reports[0].unchanged
+    assert commits() == before
+
+
+def test_sync_refuses_to_overwrite_a_canvas_edit(api, flows):
+    sync([flows["train"]], api, origin=ORIGIN)
+    api.put_source("train", "evaluate", "def process(weights):\n    return {}\n")
+
+    with pytest.raises(SyncError, match="edited on the canvas"):
+        sync([flows["train"]], api, origin=ORIGIN)
+
+    reports = sync([flows["train"]], api, origin=ORIGIN, force=True)
+    assert "evaluate" in reports[0].changed
+    assert api.get_source("train", "evaluate").startswith(MARKER)
+
+
+def test_sync_refuses_a_flow_it_did_not_create(api, flows):
+    api.put_flow({"name": "drawn", "version": 1, "nodes": [], "mode": "batch"})
+    api.publish("drawn", 1)
+    theirs = flows["train"]
+    theirs.name = "drawn"
+
+    try:
+        with pytest.raises(SyncError, match="not created by sync"):
+            sync([theirs], api, origin=ORIGIN)
+    finally:
+        theirs.name = "train"
+
+
+def test_use_stores_the_rewiring_it_was_given(api, flows):
+    sync([flows["finetune"]], api, origin=ORIGIN)
+
+    nodes = {n["id"]: n for n in api.get_flow("finetune")["definition"]["nodes"]}
+    assert [p["name"] for p in nodes["fit"]["requires"]] == ["augmented", "lr"]
+    assert nodes["fit"]["params"] == {"epochs": 3}
+
+
+def test_refresh_retires_the_workers(client, superuser_token_headers):
+    response = client.post(f"{PREFIX}/modules/refresh", headers=superuser_token_headers)
+
+    assert response.status_code == 200
+    assert "afresh" in response.json()["message"]

backend/tests/sdk/conftest.py

@@ -0,0 +1,19 @@
+from collections.abc import Generator
+
+import pytest
+
+from fluksio.sdk import FLOWS
+
+
+@pytest.fixture(scope="session", autouse=True)
+def db() -> Generator[None, None, None]:
+    """Declaring a flow touches no database."""
+    yield
+
+
+@pytest.fixture(autouse=True)
+def registry() -> Generator[None, None, None]:
+    """The registry is module-level, so one test's flows are not another's."""
+    FLOWS.clear()
+    yield
+    FLOWS.clear()

backend/tests/sdk/test_build.py

@@ -0,0 +1,238 @@
+"""What the decorators declare, and what the engine would be given."""
+
+import pytest
+
+from fluksio.flow.schemas import FlowDef
+from fluksio.sdk import MARKER, Flow, Port, SyncError, node, use
+
+# The functions live here rather than in each test: a node's module is what the
+# generated body imports, and `__main__` is refused for exactly that reason.
+
+
+@node(provides=[Port("dataset", "artifact"), Port("rows", "int")])
+def prepare(limit=8):
+    return {"dataset": {"digest": "sha256:x", "size": 1}, "rows": limit}
+
+
+@node(
+    requires=["dataset", Port("lr", "float")],
+    provides=[Port("loss", "float", stream=True), Port("weights", "artifact")],
+    device="gpu",
+    device_policy="prefer",
+)
+def fit(dataset, lr, epochs=3):
+    for step in range(epochs):
+        yield {"loss": 1.0 / (step + 1)}
+    return {"weights": {"digest": "sha256:y", "size": 1}}
+
+
+@node(requires=["weights"], provides=Port("score", "float"))
+def evaluate(weights):
+    return 0.5
+
+
+@node(requires=["dataset"], provides=[Port("augmented", "artifact")])
+def augment(dataset):
+    return {"augmented": dataset}
+
+
+def a_flow(name="train", **kwargs):
+    options = {
+        "nodes": [prepare, fit, evaluate],
+        "inputs": [Port("lr", "float", initial=0.01)],
+        "outputs": ["score"],
+    }
+    options.update(kwargs)
+    return Flow(name, **options)
+
+
+def test_document_is_a_flow_the_engine_accepts():
+    document = a_flow().document({"kind": "python", "repo": "/r", "commit": "abc"})
+    definition = FlowDef.model_validate(document)
+
+    assert definition.mode == "batch"
+    assert [n.id for n in definition.nodes] == ["prepare", "fit", "evaluate"]
+    assert definition.origin is not None
+    assert definition.origin.commit == "abc"
+    assert [i.spec.name for i in definition.inputs] == ["lr"]
+    assert definition.outputs == ["score"]
+
+
+def test_defaults_become_settings_and_ports_do_not():
+    definition = FlowDef.model_validate(a_flow().document())
+    nodes = {n.id: n for n in definition.nodes}
+
+    assert nodes["prepare"].params == {"limit": 8}
+    # `dataset` and `lr` are ports, so they are not settings as well.
+    assert nodes["fit"].params == {"epochs": 3}
+    assert {p.port for p in nodes["fit"].requires} == {"dataset", "lr"}
+
+
+def test_device_travels_to_the_node():
+    definition = FlowDef.model_validate(a_flow().document())
+    node_def = next(n for n in definition.nodes if n.id == "fit")
+
+    assert node_def.device == "gpu"
+    assert node_def.device_policy == "prefer"
+
+
+def test_a_required_port_takes_the_type_it_is_provided_as():
+    """`requires=["dataset"]` is not a second declaration of its type."""
+    definition = FlowDef.model_validate(a_flow().document())
+    node_def = next(n for n in definition.nodes if n.id == "fit")
+
+    assert next(p for p in node_def.requires if p.port == "dataset").dtype == "artifact"
+
+
+def test_use_rewires_and_reconfigures_one_flow_only():
+    train = a_flow()
+    finetune = Flow(
+        "finetune",
+        nodes=[prepare, augment, use(fit, wire={"dataset": "augmented"}, epochs=9)],
+        inputs=[Port("lr", "float")],
+    )
+
+    theirs = next(n for n in finetune.document()["nodes"] if n["id"] == "fit")
+    ours = next(n for n in train.document()["nodes"] if n["id"] == "fit")
+    assert [p["name"] for p in theirs["requires"]] == ["augmented", "lr"]
+    assert theirs["params"] == {"epochs": 9}
+    # The same function in another flow is untouched by that.
+    assert [p["name"] for p in ours["requires"]] == ["dataset", "lr"]
+    assert ours["params"] == {"epochs": 3}
+
+
+def unknown_port(a):
+    return a
+
+
+def two_arguments(a, b):
+    return a
+
+
+def one_default(a=1):
+    return a
+
+
+def a_generator():
+    yield {"x": 1}
+
+
+def reads_dataset(dataset):
+    return dataset
+
+
+def test_a_port_with_no_parameter_is_refused():
+    with pytest.raises(SyncError, match="no parameter to arrive in"):
+        node(requires=["nope"])(unknown_port)
+
+
+def test_a_parameter_that_is_neither_port_nor_setting_is_refused():
+    with pytest.raises(SyncError, match="neither a port nor a setting"):
+        node(requires=["a"])(two_arguments)
+
+
+def test_a_name_the_store_would_refuse_is_refused_here():
+    with pytest.raises(SyncError, match="node id"):
+        node(requires=["a"], id="Trainer")(one_default)
+
+
+def test_a_generator_cannot_use_the_bare_return_shorthand():
+    with pytest.raises(SyncError, match="yields"):
+        node(provides=Port("x", "float"))(a_generator)
+
+
+def test_a_setting_that_is_also_a_port_is_refused():
+    with pytest.raises(SyncError, match="both a port and a setting"):
+        Flow("clash", nodes=[use(prepare, limit=2), _sets("dataset")])
+
+
+def test_a_setting_that_is_not_a_parameter_is_refused():
+    with pytest.raises(SyncError, match="not a parameter"):
+        Flow("stray", nodes=[use(prepare, nonesuch=1)])
+
+
+def _sets(name):
+    """A node whose settings clash with its own port, built for the test."""
+    fn = node(requires=["dataset"], settings={name: 1})
+    return fn(reads_dataset)
+
+
+def test_a_message_nobody_provides_is_refused():
+    orphan = node(requires=["missing"], id="orphan")(reads_dataset_missing)
+    with pytest.raises(SyncError, match="which no node in it provides"):
+        Flow("broken", nodes=[orphan])
+
+
+def reads_dataset_missing(missing):
+    return missing
+
+
+def test_an_output_nobody_produces_is_refused():
+    with pytest.raises(SyncError, match="not produced by any"):
+        a_flow("typo", outputs=["scoer"])
+
+
+def test_two_nodes_of_one_name_are_refused():
+    with pytest.raises(SyncError, match="tell them apart"):
+        Flow("twice", nodes=[prepare, prepare])
+
+
+def test_a_type_both_sides_disagree_on_is_refused():
+    wrong = node(requires=[Port("dataset", "float")], id="wrong")(reads_dataset)
+    with pytest.raises(SyncError, match="provided as artifact"):
+        Flow("mistyped", nodes=[prepare, wrong])
+
+
+def test_shim_imports_rather_than_copies():
+    shims = a_flow().shims()
+
+    assert shims["prepare"].startswith(MARKER)
+    assert "from tests.sdk.test_build import prepare" in shims["prepare"]
+    assert "def process(**settings):" in shims["prepare"]
+    assert "return prepare(**settings)" in shims["prepare"]
+
+
+def test_shim_of_a_generator_delegates_and_keeps_its_return_value():
+    """A bare `yield from` streams but drops what the generator returns."""
+    assert (
+        "return (yield from fit(dataset=dataset, lr=lr, **settings))"
+        in a_flow().shims()["fit"]
+    )
+
+
+def test_a_generator_shim_publishes_both_the_stream_and_the_result():
+    """Driven the way the worker drives it: every yield, then the return."""
+    namespace: dict = {}
+    exec(compile(a_flow().shims()["fit"], "<shim>", "exec"), namespace)
+
+    generator = namespace["process"](dataset=None, lr=0.5, epochs=2)
+    streamed = []
+    result = None
+    while True:
+        try:
+            streamed.append(next(generator))
+        except StopIteration as stop:
+            result = stop.value
+            break
+
+    assert [step["loss"] for step in streamed] == [1.0, 0.5]
+    # A bare `yield from` would stream the same and lose this.
+    assert result == {"weights": {"digest": "sha256:y", "size": 1}}
+
+
+def test_shim_of_a_single_port_wraps_the_bare_return():
+    assert "return {'score': evaluate(weights=weights" in a_flow().shims()["evaluate"]
+
+
+def test_every_shim_compiles_and_defines_process():
+    for code in a_flow().shims().values():
+        namespace: dict = {}
+        exec(compile(code, "<shim>", "exec"), namespace)
+        assert callable(namespace["process"])
+
+
+def test_the_decorators_leave_the_function_alone():
+    """The point of the whole feature: it is still your code."""
+    assert prepare(limit=2)["rows"] == 2
+    assert [step["loss"] for step in fit(None, 0.5, epochs=2)] == [1.0, 0.5]
+    assert evaluate(None) == 0.5

backend/tests/test_cli.py

@@ -60,3 +60,29 @@ def test_the_store_works_without_git(tmp_path: Path, monkeypatch) -> None:
     assert store.head() == ""
     store.write_requirements("numpy\n")
     assert store.read_requirements() == "numpy\n"
+
+
+def test_run_arguments_are_typed_by_the_flow_they_are_for() -> None:
+    """`--lr 0.05` is a float because the flow says `lr` is one."""
+    from fluksio.sdk import SyncError
+    from fluksio.sdk.cli import _params
+
+    definition = {
+        "inputs": [
+            {"spec": {"name": "lr", "dtype": "float"}},
+            {"spec": {"name": "epochs", "dtype": "int"}},
+            {"spec": {"name": "resume", "dtype": "bool"}},
+        ]
+    }
+
+    assert _params(definition, ["--lr", "0.05", "--epochs", "3", "--resume"]) == {
+        "lr": 0.05,
+        "epochs": 3,
+        "resume": True,
+    }
+    assert _params(definition, ["--lr=1e-4"]) == {"lr": 0.0001}
+
+    import pytest
+
+    with pytest.raises(SyncError, match="not an input of this flow"):
+        _params(definition, ["--nonesuch", "1"])

docs/code/cli.md

@@ -11,6 +11,10 @@ There is a second, smaller distribution — `fluksio-worker` — for a machine t
 should only *run nodes* for an engine elsewhere. It has none of the engine in
 it. See [Remote workers](workers.md).
 
+The command is two things at once: `serve`, `enroll` and `worker` *are* an
+installation, while `login`, `sync`, `run` and `runs` talk to one that may be
+anywhere.
+
 ## `fluksio serve`
 
 Runs the engine.
@@ -100,6 +104,63 @@ fluksio worker --url wss://api.example.com/api/v1/workers/attach \
 
 See [Remote workers](workers.md).
 
+## Talking to an engine
+
+The four commands below are the client half: they run wherever you work, and
+address an engine over its API rather than being one.
+
+### `fluksio login`
+
+```sh
+fluksio login --url http://127.0.0.1:8000
+```
+
+Asks for an email and password, and keeps the token it gets in
+`~/.config/fluksio/client.json` (`$XDG_CONFIG_HOME` is honoured). Everything
+below reads it from there, or from `FLUKSIO_URL` and `FLUKSIO_TOKEN`, or from
+its own `--url` and `--token`.
+
+### `fluksio sync`
+
+```sh
+fluksio sync [PATH_OR_MODULE ...]        # default: the current directory
+```
+
+Imports what you name, collects the flows the decorators declared, and uploads
+each one with a generated import shim per node. A directory that is a package
+is walked; a dotted name is imported as it stands; nothing is loaded from a
+file path, because the shim has to import the same way.
+
+| Flag | What it does |
+|---|---|
+| `--dry-run` | print the flow documents and shims, upload nothing |
+| `--no-publish` | leave the upload as a draft |
+| `--force` | overwrite a flow, or a node body, that was edited on the canvas |
+
+Every sync retires the engine's workers, including one that had nothing to
+upload — a worker holds your package in memory, so an edit to it is invisible
+until the process goes. See
+[Getting started: data science](../getting-started/data-science.md).
+
+### `fluksio run`
+
+```sh
+fluksio run train --lr 0.05 --seed 7 [--wait]
+```
+
+Submits a run. Flags that are not its own are the flow's inputs, typed by what
+the flow declares them as. `--wait` blocks until the run finishes and exits
+non-zero if it failed.
+
+### `fluksio runs`
+
+```sh
+fluksio runs [--flow train] [--limit 20]
+```
+
+The runs an engine has recorded, newest first: id, status, flow, duration, the
+commit of the repository it came from, and its parameters.
+
 ## What lives in the data directory
 
 ```text

docs/code/nodes.md

@@ -204,6 +204,15 @@ def process(lr, epochs):
 That is the whole of it. The node is three lines, `myresearch` can be as many
 modules as it likes, and nothing was copied.
 
+!!! tip "You can have those three lines written for you"
+
+    Decorate `fit` with `@node(...)` where it is defined, say which nodes make
+    a flow with `Flow(...)`, and `fluksio sync` generates the body above —
+    along with the flow document, so there is nothing to PUT by hand. The
+    declaration lives beside the function it describes and is checked against
+    its signature. See
+    [Getting started: data science](../getting-started/data-science.md).
+
 !!! warning "Editable, but not live"
 
     `-e` means edits reach the venv without reinstalling — but a node's process

docs/getting-started/data-science.md

@@ -44,14 +44,21 @@ notices.
     orchestrator spends before it does anything. Leave it in a `tmux` window,
     or write a small `systemd --user` unit for it.
 
-## Get a token
+## Log in
 
-Everything below is the HTTP API. Grab a token once:
+```sh
+fluksio login --url http://127.0.0.1:8000
+```
+
+It asks for the email and password printed above and keeps the token in
+`~/.config/fluksio/client.json`, so nothing below needs credentials again.
+
+Everything the commands do is the HTTP API, and some of this page shows it
+directly. For that, grab the same token as a shell variable:
 
 ```sh
 export FLUKSIO=http://127.0.0.1:8000/api/v1
-export TOKEN=$(curl -s -X POST $FLUKSIO/login/access-token \
-  -d "username=admin@example.com&password=k3Qm-8vTpLdX" | jq -r .access_token)
+export TOKEN=$(jq -r .token ~/.config/fluksio/client.json)
 ```
 
 While you are experimenting, the interactive schema at
@@ -90,17 +97,18 @@ Adding a package takes effect immediately; nothing restarts.
     Then mark the node `"device": "local"` and it runs on that interpreter. It
     is the same mechanism that sends a node to a GPU box, and it is worth
     knowing about early — see [Remote workers](../code/workers.md).
-
-## Wrap your training script
+## Say which functions are nodes
 
 A **flow** is a graph of nodes. A **batch flow** is one that runs on demand
-from parameters to a result, which is what an experiment is. Your existing
-script becomes the body of a node.
+from parameters to a result, which is what an experiment is. A node is one of
+your own functions — it stays in your repository, imported by its siblings as
+it always was.
 
-Say your script looks roughly like this:
+Say your project looks roughly like this:
 
 ```python
-def train(lr, epochs):
+# myresearch/train.py
+def fit(dataset, lr, epochs=25):
     model = build_model()
     for epoch in range(epochs):
         loss = step(model, lr)
@@ -109,17 +117,27 @@ def train(lr, epochs):
     return loss
 ```
 
-Two changes turn it into a node:
+Two changes turn it into a node, and neither of them moves it:
 
 ```python
-"""Fit the model. A generator, so numbers escape while it is still running."""
-
+# myresearch/train.py
 import fluksio
+from fluksio import Port, node
 
 
-def process(lr, epochs):
-    model = build_model()
-    for epoch in range(int(epochs)):
+@node(
+    requires=["dataset", Port("lr", "float")],
+    provides=[
+        Port("loss", "float", stream=True),
+        Port("weights", "artifact"),
+        Port("final_loss", "float"),
+    ],
+    timeout=600,
+)
+def fit(dataset, lr, epochs=25):
+    """Fit the model. A generator, so numbers escape while it is running."""
+    model = build_model(fluksio.load_artifact(dataset))
+    for epoch in range(epochs):
         loss = step(model, lr)
         yield {"loss": loss}                # ← published now, on the loss port
     torch.save(model.state_dict(), "weights.pt")
@@ -140,48 +158,23 @@ function is invisible to all three.
 checkpoint, a dataset, a plot. It stores the bytes by their hash and returns a
 small reference. Nothing changes about how you write the file.
 
-### If your code does not fit in one file
+The decorator returns the function untouched, so `fit(dataset, 0.05)` in a
+test, a notebook or a debugger is exactly what it was before. Outside a run
+`fluksio.emit` does nothing rather than failing, so a progress report never
+stops your own code from running.
 
-The example above is a single self-contained file, which most real projects are
-not. If the function you want as a node imports half your repository, do not
-move it — install the repository into the venv the nodes run on, by adding one
-line to the manifest you just applied:
+### What is declared, and what is not
 
-```text
--e /home/you/my-research
-```
-
-Now the node is a wrapper over what you already have, and your code stays in
-your own repository, under your own version control, importing its own
-siblings as it always did:
-
-```python
-"""The node. The training lives in the project, where it belongs."""
-
-from myresearch.train import fit
-
-
-def process(lr, epochs):
-    return fit(lr, epochs)
-```
-
-A generator still works through the wrapper — `yield from fit(...)` — so the
-per-epoch metrics arrive exactly as before.
-
-!!! warning "Apply after you edit"
-
-    The engine's workers are long-lived and hold your imported modules in
-    memory, so a change to `myresearch` is picked up when they are retired —
-    which is what **Apply** does. If you are editing many times an hour, attach
-    your own interpreter as a worker instead: it starts a process per call and
-    therefore reads your code fresh every run.
-
-    ```sh
-    fluksio worker --url ws://127.0.0.1:8000/api/v1/workers/attach \
-      --token "$WORKER_TOKEN" --labels local --python "$(which python)"
-    ```
-
-    Then mark the node `"device": "local"`.
+- **Ports are declared, never inferred.** `requires` names the messages the
+  node reads; each arrives as the parameter of the same name. A bare string is
+  the shorthand — `"dataset"` takes its type from whoever provides it, so a
+  message's type is written down exactly once.
+- **Parameters with a default that are not ports become settings.** `epochs`
+  above is stored with the flow and tunable on the canvas without touching
+  this file.
+- **The declaration is checked against the function.** A port with no matching
+  parameter, or a parameter that is neither port nor setting, is an error when
+  the module is imported — not when the node is first called.
 
 !!! note "Where a `yield` cannot reach"
 
@@ -195,92 +188,172 @@ per-epoch metrics arrive exactly as before.
     )])
     ```
 
-## Create the flow
+## Say which nodes make a flow
 
-There is no scaffolding command yet, so a flow is created by PUTting its
-definition. That is a fifteen-line script you run once:
+Membership is a list, not a directory layout: the functions can live wherever
+they already do.
 
 ```python
-"""Create the `train` flow. Run once; edit it in the canvas afterwards."""
+# myresearch/pipeline.py
+from fluksio import Flow, Port, use
 
-import httpx
+from myresearch.data import augment, prepare
+from myresearch.evaluate import evaluate
+from myresearch.train import fit
 
-API = "http://127.0.0.1:8000/api/v1"
-api = httpx.Client(base_url=API, timeout=60)
-token = api.post(
-    "/login/access-token",
-    data={"username": "admin@example.com", "password": "k3Qm-8vTpLdX"},
-).json()["access_token"]
-api.headers["Authorization"] = f"Bearer {token}"
-
-api.put("/flows/train", json={
-    "name": "train",
-    "title": "Model training",
-    # Batch: nothing is activated, nothing fires until a run asks.
-    "mode": "batch",
-    # Its inputs are the run's parameters, with the values a run gets when it
-    # names none.
-    "inputs": [
-        {"spec": {"name": "lr", "dtype": "float"}, "initial": 0.01},
-        {"spec": {"name": "epochs", "dtype": "int"}, "initial": 50},
-    ],
-    # What a run reports as its result.
-    "outputs": ["final_loss", "weights"],
-    "nodes": [{
-        "id": "train",
-        "type": "python",
-        "title": "Fit the model",
-        # An *idle* timeout once the node streams: this is how long it may go
-        # quiet, not how long it may run.
-        "timeout": 600,
-        "requires": [
-            {"name": "lr", "dtype": "float"},
-            {"name": "epochs", "dtype": "int"},
-        ],
-        "provides": [
-            # `stream` says this port publishes repeatedly during one execution.
-            {"name": "loss", "dtype": "float", "stream": True},
-            {"name": "final_loss", "dtype": "float"},
-            {"name": "weights", "dtype": "artifact"},
-        ],
-    }],
-}).raise_for_status()
-
-api.put("/flows/train/nodes/train/source",
-        json={"code": open("train_node.py").read()}).raise_for_status()
-
-version = api.get("/flows/train", params={"draft": True}).json()["definition"]["version"]
-api.post("/flows/train/publish", json={"version": version}).raise_for_status()
-print("published")
+train = Flow(
+    "train",
+    nodes=[prepare, fit, evaluate],
+    inputs=[Port("lr", "float", initial=0.01)],
+    outputs=["score", "final_loss"],
+)
 ```
 
-Two things worth noticing. Ports are declared, not inferred — `process(lr,
-epochs)` gets its arguments from the ports of the same name, and the types are
-checked on every value. And saving writes a *draft*; `publish` is what the
-engine picks up. That separation is what lets you edit a flow that is running.
+`inputs` are the run's parameters, with the value a run gets when it names
+none; `outputs` are what a run reports as its result.
+
+Nodes are connected because one **provides** a message another **requires** —
+never because one imported the other. Importing `fit` into a second flow means
+"the same code", not "wired to it".
+
+Which is how the same function serves two flows, rewired and reconfigured for
+each:
+
+```python
+finetune = Flow(
+    "finetune",
+    nodes=[
+        prepare,
+        augment,
+        use(fit, wire={"dataset": "augmented"}, epochs=3),
+        evaluate,
+    ],
+    inputs=[Port("lr", "float", initial=1e-4)],
+    outputs=["score"],
+)
+```
+
+`use(fn, ...)` is one use of a node in one flow: `wire` binds a port to a
+different message, keyword arguments override settings, and `id=` tells two
+uses of one function apart. `train` above is unaffected by any of it.
+
+## Sync it
+
+```sh
+fluksio sync myresearch
+```
+
+That imports the package, checks every declaration, and uploads each flow with
+a generated body per node:
+
+```python
+# generated by fluksio sync from myresearch/train.py — edit that file instead
+import sys
+
+_REPO = '/home/you/my-research'
+if _REPO not in sys.path:
+    sys.path.insert(0, _REPO)
+
+from myresearch.train import fit
+
+
+def process(dataset, lr, **settings):
+    return (yield from fit(dataset=dataset, lr=lr, **settings))
+```
+
+So the flow store still holds a complete, runnable, git-versioned definition —
+the body simply happens to import rather than duplicate. Your code stays in
+your repository, under your version control.
+
+The upload is stamped with that repository's commit, and every run records it
+alongside the store's own. `--dry-run` prints all of this and uploads nothing;
+`--no-publish` leaves it as a draft.
+
+!!! warning "Sync after you edit"
+
+    The engine's workers are long-lived and hold your imported modules in
+    memory, so an edit to `myresearch` is invisible until they are retired —
+    which is what every `fluksio sync` does, including one that has nothing to
+    upload. If you are editing many times an hour, attach your own interpreter
+    as a worker instead: it starts a process per call and therefore reads your
+    code fresh every run.
+
+    ```sh
+    fluksio worker --url ws://127.0.0.1:8000/api/v1/workers/attach \
+      --token "$WORKER_TOKEN" --labels local --python "$(which python)"
+    ```
+
+    Then give the node `device="local"`.
+
+!!! note "The repository path is a deployment detail"
+
+    The generated body puts your repository on `sys.path` by absolute path,
+    which is right for the machine you synced from and meaningless in a
+    container. For anything else, install the package where the node runs —
+    `-e /home/you/my-research` in the module manifest, or
+    `myresearch @ git+ssh://…@a1b2c3d`, which travels where a path does not.
+
+### Which hardware a node runs on
+
+`device` picks the worker, the same way an executor does elsewhere:
+
+```python
+@node(requires=["dataset"], provides=[...], device="gpu", device_policy="prefer")
+def fit(dataset, lr, epochs=25):
+    ...
+```
+
+`device_policy="require"` (the default) waits for a worker carrying that label;
+`"prefer"` runs it locally when none is attached, which is what you want while
+the GPU box is not switched on. See [Remote workers](../code/workers.md).
+
+### What the canvas does with a synced flow
+
+Its node bodies are generated, so the editor shows them read-only and says
+where the real code is. Everything else behaves as usual — but ports and
+settings changed there are overwritten by the next sync, which is the point of
+the repository being the source of truth. A node whose code you edit on the
+canvas makes the next sync stop and say so rather than discarding your edit;
+`--force` overrides that.
 
 ## Run it
 
 ```sh
-curl -X POST $FLUKSIO/runs/flows/train -H "Authorization: Bearer $TOKEN" \
-  -H 'Content-Type: application/json' \
-  -d '{"params": {"lr": 0.003, "epochs": 200}, "seed": 7}'
+fluksio run train --lr 0.003 --seed 7
+fluksio runs --flow train
 ```
 
-It answers immediately with a queued run — training is measured in hours, so
-nothing waits for it. A parameter you did not declare, or one of the wrong
-type, is refused with a 422 before anything executes.
+`--lr` is typed by the flow's own inputs, so `0.003` arrives as a float. A
+parameter you did not declare, or one of the wrong type, is refused before
+anything executes. It answers immediately with a queued run — training is
+measured in hours, so nothing waits for it unless you pass `--wait`.
 
-Then, whenever you like:
+From Python, the flow you declared is also the handle to its runs:
+
+```python
+from myresearch.pipeline import train
+
+run = train.submit(lr=0.003, seed=7).wait()
+print(run.status, run.result)
+print(run.metrics("train.loss")[-1])       # the whole series is kept
+open("weights.pt", "wb").write(run.download("train.weights"))
+```
+
+Or over HTTP, which is what both of those are:
 
 ```sh
+curl -X POST $FLUKSIO/runs/flows/train -H "Authorization: Bearer $TOKEN" \
+  -H 'Content-Type: application/json' \
+  -d '{"params": {"lr": 0.003}, "seed": 7}'
+
 curl -s $FLUKSIO/runs/<id> -H "Authorization: Bearer $TOKEN" | jq
 curl -s "$FLUKSIO/runs/<id>/metrics?name=train.loss" -H "Authorization: Bearer $TOKEN" | jq
 ```
 
 The run carries its parameters, a digest of them, the seed, its result, how
-long each node took, what it logged, and every artifact it produced. That is
-the answer to "what was the learning rate on the run that got 94%?".
+long each node took, what it logged, every artifact it produced, and both
+commits — the flow store's and your repository's. That is the answer to "what
+was the learning rate on the run that got 94%?".
 
 ## Sweep it
 
@@ -309,6 +382,71 @@ curl -s "$FLUKSIO/runs/series/compare?ids=$A,$B,$C&metric=train.loss" \
 
 which answers in exactly the shape a chart widget draws.
 
+## What sync does, at the API level
+
+Nothing here is privileged: `sync` is a client, and a flow is a document you
+can PUT yourself. The whole of it, for the one-node case:
+
+```python
+"""Create the `train` flow by hand. What `fluksio sync` automates."""
+
+import httpx
+
+API = "http://127.0.0.1:8000/api/v1"
+api = httpx.Client(base_url=API, timeout=60)
+token = api.post(
+    "/login/access-token",
+    data={"username": "admin@example.com", "password": "k3Qm-8vTpLdX"},
+).json()["access_token"]
+api.headers["Authorization"] = f"Bearer {token}"
+
+api.put("/flows/train", json={
+    "name": "train",
+    "title": "Model training",
+    # Batch: nothing is activated, nothing fires until a run asks.
+    "mode": "batch",
+    "inputs": [
+        {"spec": {"name": "lr", "dtype": "float"}, "initial": 0.01},
+        {"spec": {"name": "epochs", "dtype": "int"}, "initial": 50},
+    ],
+    "outputs": ["final_loss", "weights"],
+    "nodes": [{
+        "id": "train",
+        "type": "python",
+        "title": "Fit the model",
+        # An *idle* timeout once the node streams: this is how long it may go
+        # quiet, not how long it may run.
+        "timeout": 600,
+        "requires": [
+            {"name": "lr", "dtype": "float"},
+            {"name": "epochs", "dtype": "int"},
+        ],
+        "provides": [
+            # `stream` says this port publishes repeatedly during one execution.
+            {"name": "loss", "dtype": "float", "stream": True},
+            {"name": "final_loss", "dtype": "float"},
+            {"name": "weights", "dtype": "artifact"},
+        ],
+    }],
+}).raise_for_status()
+
+api.put("/flows/train/nodes/train/source",
+        json={"code": open("train_node.py").read()}).raise_for_status()
+
+version = api.get("/flows/train").json()["definition"]["version"]
+api.post("/flows/train/publish", json={"version": version}).raise_for_status()
+```
+
+Two things worth noticing, because the decorators only move where they are
+said. Ports are declared, not inferred — `process(lr, epochs)` gets its
+arguments from the ports of the same name, and the types are checked on every
+value. And saving writes a *draft*; `publish` is what the engine picks up.
+That separation is what lets you edit a flow that is running.
+
+A flow uploaded this way carries no `origin`, which is what tells the canvas —
+and the next `fluksio sync` — that it was not generated. See
+[The API](../code/api.md).
+
 ## Small scripts you are just playing with
 
 The same machinery, minus the ceremony. If what you want is "keep a record of
@@ -328,9 +466,10 @@ For quick iteration, keep the flow small (one node is fine), keep the engine
 running, and submit from wherever you are working:
 
 ```python
-import httpx
-run = httpx.post(f"{API}/runs/flows/train", json={"params": {"lr": lr}},
-                 headers=auth).json()
+from myresearch.pipeline import train
+
+for lr in (0.001, 0.003, 0.01):
+    train.submit(lr=lr)
 ```
 
 A submit is around 15 ms, so calling that in a loop is a reasonable thing to do.
@@ -360,6 +499,7 @@ is the same stack.
   artifacts, sweeps, durability, what happens when your engine dies mid-training
 - [Writing node code](../code/nodes.md) — generators, settings, what a node may
   and may not do
+- [The command line](../code/cli.md) — `login`, `sync`, `run`, `runs` in full
 - [Remote workers](../code/workers.md) — send the training node to the GPU box
   and keep the rest on your laptop
 - [The flow editor](../interface/flow-editor.md) — once you have a portal, this

examples/myresearch/__init__.py

@@ -0,0 +1,7 @@
+"""A research package that knows nothing about Fluksio, except where it says so.
+
+Everything here is ordinary Python: `prepare`, `fit` and `evaluate` are called
+directly by `python -m myresearch.pipeline`, and the decorators on them only
+say what a node of each would look like. `fluksio sync examples/myresearch` is
+what turns that into a flow.
+"""

examples/myresearch/data.py

@@ -0,0 +1,34 @@
+"""Where the data comes from."""
+
+from __future__ import annotations
+
+import json
+
+import fluksio
+from fluksio import Port, node
+
+
+@node(provides=[Port("dataset", "artifact"), Port("rows", "int")])
+def prepare(source: str = "builtin", limit: int = 512):
+    """Make the training set and store it, returning a reference to it.
+
+    `source` and `limit` have defaults and are not ports, so they become the
+    node's settings: the canvas can tune them without touching this file.
+    """
+    rows = [{"x": index / limit, "y": (index % 7) / 7} for index in range(limit)]
+    payload = json.dumps({"source": source, "rows": rows}).encode()
+    return {
+        "dataset": fluksio.save_artifact(payload, "dataset.json", "application/json"),
+        "rows": len(rows),
+    }
+
+
+@node(requires=["dataset"], provides=[Port("augmented", "artifact")])
+def augment(dataset, factor: int = 2):
+    """Grow the training set, so `finetune` has something of its own."""
+    payload = json.loads(open(fluksio.load_artifact(dataset)).read())
+    payload["rows"] = payload["rows"] * factor
+    blob = json.dumps(payload).encode()
+    return {
+        "augmented": fluksio.save_artifact(blob, "augmented.json", "application/json")
+    }

examples/myresearch/evaluate.py

@@ -0,0 +1,19 @@
+"""Scoring what was trained."""
+
+from __future__ import annotations
+
+import json
+
+import fluksio
+from fluksio import Port, node
+
+
+@node(requires=["weights"], provides=Port("score", "float"))
+def evaluate(weights):
+    """Score the model, returning the number rather than a dict.
+
+    `provides=Port(...)` — one port rather than a list of them — is the opt-in
+    to a bare return: the generated node wraps it in the message it belongs to.
+    """
+    trained = json.loads(open(fluksio.load_artifact(weights)).read())
+    return round(1 - trained["lr"] / (1 + trained["epochs"]), 4)

examples/myresearch/pipeline.py

@@ -0,0 +1,48 @@
+"""Which nodes make up which flow.
+
+Membership is this list, not the file a function sits in: `fit` is declared in
+`train.py` and used by both flows below, once as itself and once rewired and
+reconfigured. Wiring is not membership either — nodes connect because one
+provides a message another requires, never because one imported the other.
+
+    fluksio sync examples/myresearch
+    fluksio run train --lr 0.05 --wait
+"""
+
+from __future__ import annotations
+
+from fluksio import Flow, Port, use
+from myresearch.data import augment, prepare
+from myresearch.evaluate import evaluate
+from myresearch.train import fit
+
+train = Flow(
+    "train",
+    title="Train",
+    nodes=[prepare, fit, evaluate],
+    inputs=[Port("lr", "float", initial=0.01)],
+    outputs=["score", "final_loss"],
+)
+
+finetune = Flow(
+    "finetune",
+    title="Finetune",
+    nodes=[
+        prepare,
+        augment,
+        # The same function, reading `augmented` instead of `dataset` and with
+        # a shorter schedule. `train` is unaffected.
+        use(fit, wire={"dataset": "augmented"}, epochs=3),
+        evaluate,
+    ],
+    inputs=[Port("lr", "float", initial=0.0001)],
+    outputs=["score"],
+)
+
+
+if __name__ == "__main__":
+    # Run it here, with no engine involved: the decorators changed nothing
+    # about calling these functions.
+    dataset = prepare(limit=64)
+    losses = list(fit(dataset["dataset"], lr=0.05, epochs=5))
+    print("losses:", [round(step["loss"], 4) for step in losses])

examples/myresearch/train.py

@@ -0,0 +1,41 @@
+"""The part that would be on the GPU."""
+
+from __future__ import annotations
+
+import json
+import math
+
+import fluksio
+from fluksio import Port, node
+
+
+@node(
+    requires=["dataset", Port("lr", "float")],
+    provides=[
+        Port("loss", "float", stream=True),
+        Port("weights", "artifact"),
+        Port("final_loss", "float"),
+    ],
+    device="gpu",
+    device_policy="prefer",
+    timeout=600,
+)
+def fit(dataset, lr, epochs=25):
+    """Train, reporting the loss as it goes.
+
+    Yielding is the reporting: each one publishes on the `loss` port the
+    instant it happens, and the run keeps every value as a series — which is
+    why there is no `log_metric()` to call. `device="gpu"` with
+    `device_policy="prefer"` sends this to a worker carrying that label when
+    one is attached, and runs it here when none is.
+    """
+    rows = json.loads(open(fluksio.load_artifact(dataset)).read())["rows"]
+    loss = 1.0
+    for epoch in range(epochs):
+        loss = math.exp(-lr * epoch * 10) * (1 + 0.05 * (epoch % 3)) / (1 + lr)
+        yield {"loss": loss}
+    weights = json.dumps({"lr": lr, "epochs": epochs, "n": len(rows)}).encode()
+    return {
+        "weights": fluksio.save_artifact(weights, "weights.json", "application/json"),
+        "final_loss": loss,
+    }

frontend/src/client/schemas.gen.ts

@@ -764,6 +764,17 @@ export const FlowDef_InputSchema = {
             type: 'array',
             title: 'Outputs',
             description: 'Messages a batch run reports as its result, unqualified. Empty means every message the flow ends up holding.'
+        },
+        origin: {
+            anyOf: [
+                {
+                    '$ref': '#/components/schemas/FlowOrigin'
+                },
+                {
+                    type: 'null'
+                }
+            ],
+            description: 'Set when the flow was declared in code elsewhere and uploaded by `fluksio sync`. Absent for a flow drawn on the canvas.'
         }
     },
     type: 'object',
@@ -816,6 +827,17 @@ export const FlowDef_OutputSchema = {
             type: 'array',
             title: 'Outputs',
             description: 'Messages a batch run reports as its result, unqualified. Empty means every message the flow ends up holding.'
+        },
+        origin: {
+            anyOf: [
+                {
+                    '$ref': '#/components/schemas/FlowOrigin'
+                },
+                {
+                    type: 'null'
+                }
+            ],
+            description: 'Set when the flow was declared in code elsewhere and uploaded by `fluksio sync`. Absent for a flow drawn on the canvas.'
         }
     },
     type: 'object',
@@ -921,6 +943,46 @@ export const FlowInput_OutputSchema = {
     description: 'A message the flow starts with rather than computes.'
 } as const;
 
+export const FlowOriginSchema = {
+    properties: {
+        kind: {
+            type: 'string',
+            const: 'python',
+            title: 'Kind',
+            default: 'python'
+        },
+        repo: {
+            type: 'string',
+            title: 'Repo',
+            default: ''
+        },
+        commit: {
+            type: 'string',
+            title: 'Commit',
+            default: ''
+        },
+        dirty: {
+            type: 'boolean',
+            title: 'Dirty',
+            default: false
+        }
+    },
+    type: 'object',
+    title: 'FlowOrigin',
+    description: `Where a flow was declared, when that was somewhere other than here.
+
+A flow drawn on the canvas has no origin: the store is where it lives. One
+stamped with this was declared with the decorators in somebody's own
+repository and put here by \`\`fluksio sync\`\`, so the node bodies below it
+are generated imports and the code they run is versioned twice — once here
+and once there. Its presence is what makes a flow code-defined.
+
+Deliberately no timestamp. The store commits every change it is given, so
+when a flow was last synced is a fact its own history already holds — and
+one that would otherwise change on every sync, making an unchanged upload
+look like a new version of the flow.`
+} as const;
+
 export const FlowRollupSchema = {
     properties: {
         flow: {
@@ -2369,6 +2431,11 @@ export const RunDetailSchema = {
             type: 'string',
             title: 'Params Digest'
         },
+        origin_commit: {
+            type: 'string',
+            title: 'Origin Commit',
+            default: ''
+        },
         seed: {
             anyOf: [
                 {
@@ -3475,6 +3542,11 @@ export const fluksio__api__routes__runs__RunRowSchema = {
             type: 'string',
             title: 'Params Digest'
         },
+        origin_commit: {
+            type: 'string',
+            title: 'Origin Commit',
+            default: ''
+        },
         seed: {
             anyOf: [
                 {

frontend/src/client/sdk.gen.ts

@@ -3,7 +3,7 @@
 import type { CancelablePromise } from './core/CancelablePromise';
 import { OpenAPI } from './core/OpenAPI';
 import { request as __request } from './core/request';
-import type { AlertsReadAlertsConfigResponse, AlertsSaveAlertsConfigData, AlertsSaveAlertsConfigResponse, AlertsTestChannelData, AlertsTestChannelResponse, ArtifactsPutArtifactData, ArtifactsPutArtifactResponse, ArtifactsGetArtifactData, ArtifactsGetArtifactResponse, CloudReadStatusResponse, CloudEnrollData, CloudEnrollResponse, CloudAddRemoteUserData, CloudAddRemoteUserResponse, CloudDisconnectResponse, DashboardsReadDashboardsResponse, DashboardsReadDashboardData, DashboardsReadDashboardResponse, DashboardsCreateDashboardData, DashboardsCreateDashboardResponse, DashboardsSaveDashboardData, DashboardsSaveDashboardResponse, DashboardsDeleteDashboardData, DashboardsDeleteDashboardResponse, DashboardsPublishDashboardData, DashboardsPublishDashboardResponse, DashboardsDiscardDashboardDraftData, DashboardsDiscardDashboardDraftResponse, DashboardsRenameDashboardData, DashboardsRenameDashboardResponse, FlowsReadFlowsResponse, FlowsReadNodeTypesResponse, FlowsReadGraphResponse, FlowsReadLibraryResponse, FlowsDeleteSharedNodeData, FlowsDeleteSharedNodeResponse, FlowsReadFlowData, FlowsReadFlowResponse, FlowsSaveFlowData, FlowsSaveFlowResponse, FlowsDeleteFlowData, FlowsDeleteFlowResponse, FlowsPublishFlowData, FlowsPublishFlowResponse, FlowsDiscardDraftData, FlowsDiscardDraftResponse, FlowsRenameFlowData, FlowsRenameFlowResponse, FlowsReadNodeSourceData, FlowsReadNodeSourceResponse, FlowsSaveNodeSourceData, FlowsSaveNodeSourceResponse, FlowsShareNodeData, FlowsShareNodeResponse, FlowsUnshareNodeData, FlowsUnshareNodeResponse, FlowsStartFlowData, FlowsStartFlowResponse, FlowsStopFlowData, FlowsStopFlowResponse, FlowsPauseFlowData, FlowsPauseFlowResponse, FlowsResumeFlowData, FlowsResumeFlowResponse, FlowsStepFlowData, FlowsStepFlowResponse, FlowsValidateFlowData, FlowsValidateFlowResponse, FlowsRunFlowData, FlowsRunFlowResponse, FlowsTriggerNodeData, FlowsTriggerNodeResponse, FlowsCancelNodeData, FlowsCancelNodeResponse, FlowsAcknowledgeNodeErrorData, FlowsAcknowledgeNodeErrorResponse, FlowsReadFlowStateData, FlowsReadFlowStateResponse, FlowsReadMessageHistoryData, FlowsReadMessageHistoryResponse, LoginLoginAccessTokenData, LoginLoginAccessTokenResponse, LoginTestTokenResponse, LoginRecoverPasswordData, LoginRecoverPasswordResponse, LoginResetPasswordData, LoginResetPasswordResponse, LoginRecoverPasswordHtmlContentData, LoginRecoverPasswordHtmlContentResponse, MessagesReadMessagesResponse, MessagesPublishMessageData, MessagesPublishMessageResponse, MessagesReadMessageHistoryData, MessagesReadMessageHistoryResponse, ModulesReadModulesResponse, ModulesApplyModulesData, ModulesApplyModulesResponse, OauthRegisterClientData, OauthRegisterClientResponse, OauthAuthorizeValidateData, OauthAuthorizeValidateResponse, OauthAuthorizeData, OauthAuthorizeResponse, OauthTokenData, OauthTokenResponse, OauthReadClientsResponse, OauthRevokeClientData, OauthRevokeClientResponse, ObservabilityReadSummaryResponse, ObservabilityReadTimeseriesData, ObservabilityReadTimeseriesResponse, ObservabilityReadFlowRollupsData, ObservabilityReadFlowRollupsResponse, ObservabilityReadRunsData, ObservabilityReadRunsResponse, ObservabilityReadEventsData, ObservabilityReadEventsResponse, ObservabilityReadDeadLettersData, ObservabilityReadDeadLettersResponse, PanelsReadPanelsResponse, PanelsSavePanelsData, PanelsSavePanelsResponse, PanelsStartPairingResponse, PanelsPollPairingData, PanelsPollPairingResponse, PanelsPendingDeviceData, PanelsPendingDeviceResponse, PanelsApprovePairingData, PanelsApprovePairingResponse, PanelsUnpairPanelData, PanelsUnpairPanelResponse, PanelsReadPanelData, PanelsReadPanelResponse, PrivateCreateUserData, PrivateCreateUserResponse, RunsCreateRunData, RunsCreateRunResponse, RunsCreateSweepData, RunsCreateSweepResponse, RunsReadRunsData, RunsReadRunsResponse, RunsReadRunData, RunsReadRunResponse, RunsCancelRunData, RunsCancelRunResponse, RunsReadMetricsData, RunsReadMetricsResponse, RunsCompareMetricData, RunsCompareMetricResponse, SecretsReadSecretsResponse, SecretsSaveSecretData, SecretsSaveSecretResponse, SecretsDeleteSecretData, SecretsDeleteSecretResponse, UsersReadUsersData, UsersReadUsersResponse, UsersCreateUserData, UsersCreateUserResponse, UsersReadUserMeResponse, UsersDeleteUserMeResponse, UsersUpdateUserMeData, UsersUpdateUserMeResponse, UsersUpdatePasswordMeData, UsersUpdatePasswordMeResponse, UsersRegisterUserData, UsersRegisterUserResponse, UsersReadUserByIdData, UsersReadUserByIdResponse, UsersUpdateUserData, UsersUpdateUserResponse, UsersDeleteUserData, UsersDeleteUserResponse, UtilsTestEmailData, UtilsTestEmailResponse, UtilsHealthCheckResponse, UtilsHealthResponse, WorkersReadWorkersResponse, WorkersIssueTokenData, WorkersIssueTokenResponse, WorkersReadRuntimeResponse } from './types.gen';
+import type { AlertsReadAlertsConfigResponse, AlertsSaveAlertsConfigData, AlertsSaveAlertsConfigResponse, AlertsTestChannelData, AlertsTestChannelResponse, ArtifactsPutArtifactData, ArtifactsPutArtifactResponse, ArtifactsGetArtifactData, ArtifactsGetArtifactResponse, CloudReadStatusResponse, CloudEnrollData, CloudEnrollResponse, CloudAddRemoteUserData, CloudAddRemoteUserResponse, CloudDisconnectResponse, DashboardsReadDashboardsResponse, DashboardsReadDashboardData, DashboardsReadDashboardResponse, DashboardsCreateDashboardData, DashboardsCreateDashboardResponse, DashboardsSaveDashboardData, DashboardsSaveDashboardResponse, DashboardsDeleteDashboardData, DashboardsDeleteDashboardResponse, DashboardsPublishDashboardData, DashboardsPublishDashboardResponse, DashboardsDiscardDashboardDraftData, DashboardsDiscardDashboardDraftResponse, DashboardsRenameDashboardData, DashboardsRenameDashboardResponse, FlowsReadFlowsResponse, FlowsReadNodeTypesResponse, FlowsReadGraphResponse, FlowsReadLibraryResponse, FlowsDeleteSharedNodeData, FlowsDeleteSharedNodeResponse, FlowsReadFlowData, FlowsReadFlowResponse, FlowsSaveFlowData, FlowsSaveFlowResponse, FlowsDeleteFlowData, FlowsDeleteFlowResponse, FlowsPublishFlowData, FlowsPublishFlowResponse, FlowsDiscardDraftData, FlowsDiscardDraftResponse, FlowsRenameFlowData, FlowsRenameFlowResponse, FlowsReadNodeSourceData, FlowsReadNodeSourceResponse, FlowsSaveNodeSourceData, FlowsSaveNodeSourceResponse, FlowsShareNodeData, FlowsShareNodeResponse, FlowsUnshareNodeData, FlowsUnshareNodeResponse, FlowsStartFlowData, FlowsStartFlowResponse, FlowsStopFlowData, FlowsStopFlowResponse, FlowsPauseFlowData, FlowsPauseFlowResponse, FlowsResumeFlowData, FlowsResumeFlowResponse, FlowsStepFlowData, FlowsStepFlowResponse, FlowsValidateFlowData, FlowsValidateFlowResponse, FlowsRunFlowData, FlowsRunFlowResponse, FlowsTriggerNodeData, FlowsTriggerNodeResponse, FlowsCancelNodeData, FlowsCancelNodeResponse, FlowsAcknowledgeNodeErrorData, FlowsAcknowledgeNodeErrorResponse, FlowsReadFlowStateData, FlowsReadFlowStateResponse, FlowsReadMessageHistoryData, FlowsReadMessageHistoryResponse, LoginLoginAccessTokenData, LoginLoginAccessTokenResponse, LoginTestTokenResponse, LoginRecoverPasswordData, LoginRecoverPasswordResponse, LoginResetPasswordData, LoginResetPasswordResponse, LoginRecoverPasswordHtmlContentData, LoginRecoverPasswordHtmlContentResponse, MessagesReadMessagesResponse, MessagesPublishMessageData, MessagesPublishMessageResponse, MessagesReadMessageHistoryData, MessagesReadMessageHistoryResponse, ModulesReadModulesResponse, ModulesApplyModulesData, ModulesApplyModulesResponse, ModulesRefreshModulesResponse, OauthRegisterClientData, OauthRegisterClientResponse, OauthAuthorizeValidateData, OauthAuthorizeValidateResponse, OauthAuthorizeData, OauthAuthorizeResponse, OauthTokenData, OauthTokenResponse, OauthReadClientsResponse, OauthRevokeClientData, OauthRevokeClientResponse, ObservabilityReadSummaryResponse, ObservabilityReadTimeseriesData, ObservabilityReadTimeseriesResponse, ObservabilityReadFlowRollupsData, ObservabilityReadFlowRollupsResponse, ObservabilityReadRunsData, ObservabilityReadRunsResponse, ObservabilityReadEventsData, ObservabilityReadEventsResponse, ObservabilityReadDeadLettersData, ObservabilityReadDeadLettersResponse, PanelsReadPanelsResponse, PanelsSavePanelsData, PanelsSavePanelsResponse, PanelsStartPairingResponse, PanelsPollPairingData, PanelsPollPairingResponse, PanelsPendingDeviceData, PanelsPendingDeviceResponse, PanelsApprovePairingData, PanelsApprovePairingResponse, PanelsUnpairPanelData, PanelsUnpairPanelResponse, PanelsReadPanelData, PanelsReadPanelResponse, PrivateCreateUserData, PrivateCreateUserResponse, RunsCreateRunData, RunsCreateRunResponse, RunsCreateSweepData, RunsCreateSweepResponse, RunsReadRunsData, RunsReadRunsResponse, RunsReadRunData, RunsReadRunResponse, RunsCancelRunData, RunsCancelRunResponse, RunsReadMetricsData, RunsReadMetricsResponse, RunsCompareMetricData, RunsCompareMetricResponse, SecretsReadSecretsResponse, SecretsSaveSecretData, SecretsSaveSecretResponse, SecretsDeleteSecretData, SecretsDeleteSecretResponse, UsersReadUsersData, UsersReadUsersResponse, UsersCreateUserData, UsersCreateUserResponse, UsersReadUserMeResponse, UsersDeleteUserMeResponse, UsersUpdateUserMeData, UsersUpdateUserMeResponse, UsersUpdatePasswordMeData, UsersUpdatePasswordMeResponse, UsersRegisterUserData, UsersRegisterUserResponse, UsersReadUserByIdData, UsersReadUserByIdResponse, UsersUpdateUserData, UsersUpdateUserResponse, UsersDeleteUserData, UsersDeleteUserResponse, UtilsTestEmailData, UtilsTestEmailResponse, UtilsHealthCheckResponse, UtilsHealthResponse, WorkersReadWorkersResponse, WorkersIssueTokenData, WorkersIssueTokenResponse, WorkersReadRuntimeResponse } from './types.gen';
 
 export class AlertsService {
     /**
@@ -1160,6 +1160,12 @@ export class ModulesService {
      *
      * A manifest that does not resolve changes nothing: the venv is left as it
      * was and the stored manifest is only written once the install succeeded.
+     *
+     * Only the flows already holding a node that would not load are rebuilt,
+     * because those are the ones an install is called to fix. A flow that this
+     * install *breaks* — a package taken back out from under it — is still green
+     * and fails at call time with the node author's own import error, until
+     * something rebuilds it.
      * @param data The data for the request.
      * @param data.requestBody
      * @returns ApplyResult Successful Response
@@ -1176,6 +1182,25 @@ export class ModulesService {
             }
         });
     }
+    
+    /**
+     * Refresh Modules
+     * Retire the workers without installing anything.
+     *
+     * A node that imports the caller's own package holds it in `sys.modules` for
+     * as long as the process lives, so editing that package changes nothing a
+     * running worker can see — recompiling the node would not help either, since
+     * the import returns the module already there. Retiring the processes is the
+     * whole of it, and `fluksio sync` asks for it after every upload.
+     * @returns Message Successful Response
+     * @throws ApiError
+     */
+    public static refreshModules(): CancelablePromise<ModulesRefreshModulesResponse> {
+        return __request(OpenAPI, {
+            method: 'POST',
+            url: '/api/v1/modules/refresh'
+        });
+    }
 }
 
 export class OauthService {

frontend/src/client/types.gen.ts

@@ -235,6 +235,10 @@ export type FlowDef_Input = {
      * Messages a batch run reports as its result, unqualified. Empty means every message the flow ends up holding.
      */
     outputs?: Array<(string)>;
+    /**
+     * Set when the flow was declared in code elsewhere and uploaded by `fluksio sync`. Absent for a flow drawn on the canvas.
+     */
+    origin?: (FlowOrigin | null);
 };
 
 /**
@@ -259,6 +263,10 @@ export type FlowDef_Output = {
      * Messages a batch run reports as its result, unqualified. Empty means every message the flow ends up holding.
      */
     outputs?: Array<(string)>;
+    /**
+     * Set when the flow was declared in code elsewhere and uploaded by `fluksio sync`. Absent for a flow drawn on the canvas.
+     */
+    origin?: (FlowOrigin | null);
 };
 
 /**
@@ -294,6 +302,27 @@ export type FlowInput_Output = {
     initial?: (unknown | null);
 };
 
+/**
+ * Where a flow was declared, when that was somewhere other than here.
+ *
+ * A flow drawn on the canvas has no origin: the store is where it lives. One
+ * stamped with this was declared with the decorators in somebody's own
+ * repository and put here by ``fluksio sync``, so the node bodies below it
+ * are generated imports and the code they run is versioned twice — once here
+ * and once there. Its presence is what makes a flow code-defined.
+ *
+ * Deliberately no timestamp. The store commits every change it is given, so
+ * when a flow was last synced is a fact its own history already holds — and
+ * one that would otherwise change on every sync, making an unchanged upload
+ * look like a new version of the flow.
+ */
+export type FlowOrigin = {
+    kind?: "python";
+    repo?: string;
+    commit?: string;
+    dirty?: boolean;
+};
+
 export type FlowRollup = {
     flow: string;
     executions: number;
@@ -385,6 +414,7 @@ export type fluksio__api__routes__runs__RunRow = {
         [key: string]: unknown;
     };
     params_digest: string;
+    origin_commit?: string;
     seed: (number | null);
     group_id: (string | null);
     labels: Array<(string)>;
@@ -845,6 +875,7 @@ export type RunDetail = {
         [key: string]: unknown;
     };
     params_digest: string;
+    origin_commit?: string;
     seed: (number | null);
     group_id: (string | null);
     labels: Array<(string)>;
@@ -1407,6 +1438,8 @@ export type ModulesApplyModulesData = {
 
 export type ModulesApplyModulesResponse = (ApplyResult);
 
+export type ModulesRefreshModulesResponse = (Message);
+
 export type OauthRegisterClientData = {
     requestBody: OAuthClientRegister;
 };

frontend/src/components/Flow/FlowEditor.tsx

@@ -1123,6 +1123,20 @@ function FlowEditorInner({
           <span className="truncate px-3 py-1.5 text-sm font-medium">
             {flowDoc.title || flowName}
           </span>
+          {/* Declared in code somewhere else, at a commit that names what
+              actually ran. A dirty tree says so, because then it does not. */}
+          {flowDoc.origin ? (
+            <span
+              className="shrink-0 rounded-full bg-muted px-2 py-0.5 font-mono text-xs text-muted-foreground"
+              title={flowDoc.origin.repo || "a repository"}
+              data-testid="flow-origin"
+            >
+              {flowDoc.origin.commit
+                ? flowDoc.origin.commit.slice(0, 7)
+                : "no commit"}
+              {flowDoc.origin.dirty ? "*" : ""}
+            </span>
+          ) : null}
         </CanvasTitle>
 
         <FlowDock
@@ -1198,6 +1212,7 @@ function FlowEditorInner({
         node={selected}
         flow={flowName}
         nodeTypes={nodeTypeInfo ?? []}
+        origin={flowDoc.origin}
         suggestions={suggestions}
         expanded={editorExpanded}
         onToggleExpand={() => setEditorExpanded((wide) => !wide)}

frontend/src/components/Flow/NodeEditor.tsx

@@ -13,9 +13,12 @@ import { monacoFontFamily, setupMonaco } from "./monacoSetup"
 export default function NodeEditor({
   value,
   onChange,
+  readOnly = false,
 }: {
   value: string
   onChange: (next: string) => void
+  /** Generated code: readable, and not this panel's to change. */
+  readOnly?: boolean
 }) {
   const { resolvedTheme } = useTheme()
   const [ready, setReady] = useState(false)
@@ -37,6 +40,7 @@ export default function NodeEditor({
       onChange={(next) => onChange(next ?? "")}
       loading={<Skeleton className="h-full w-full rounded-md" />}
       options={{
+        readOnly,
         fontFamily,
         fontSize: 13,
         minimap: { enabled: false },

frontend/src/components/Flow/NodePanel.tsx

@@ -11,6 +11,7 @@ import {
 
 import {
   type DType,
+  type FlowOrigin,
   FlowsService,
   type MessageSpec,
   type NodeDef_Input,
@@ -997,6 +998,7 @@ function PanelBody({
   node,
   flow,
   nodeType,
+  origin,
   suggestions,
   expanded,
   onChange,
@@ -1008,6 +1010,7 @@ function PanelBody({
   node: NodeDef_Input
   flow: string
   nodeType: NodeTypeInfo | undefined
+  origin: FlowOrigin | null | undefined
   suggestions: PortSuggestions
   expanded: boolean
   onChange: (next: NodeDef_Input) => void
@@ -1060,7 +1063,7 @@ function PanelBody({
   const editNode = (next: NodeDef_Input) => {
     onChange(next)
     const current = code ?? source?.code
-    if (!hasSource || next.source_ref || current === undefined) return
+    if (!hasSource || next.source_ref || origin || current === undefined) return
     const wanted = scaffoldFor(next)
     if (current !== wanted && SCAFFOLD_SHAPE.test(current)) editCode(wanted)
   }
@@ -1224,6 +1227,19 @@ function PanelBody({
               {expanded ? <Minimize2 /> : <Maximize2 />}
             </Button>
           </div>
+          {origin ? (
+            /* The body below is an import of the real function, and the real
+               function is somewhere else. Editing it here would be undone by
+               the next sync, so it is read-only and says where to go. */
+            <p
+              className="text-xs text-muted-foreground"
+              data-testid="node-source-generated"
+            >
+              Generated by <span className="font-mono">fluksio sync</span> from{" "}
+              <span className="font-mono">{origin.repo || "a repository"}</span>{" "}
+              — edit it there and sync again.
+            </p>
+          ) : null}
           <div className="min-h-0 flex-1 overflow-hidden rounded-md border border-border">
             <Suspense
               fallback={
@@ -1233,6 +1249,7 @@ function PanelBody({
               <NodeEditor
                 value={code ?? source?.code ?? ""}
                 onChange={editCode}
+                readOnly={Boolean(origin)}
               />
             </Suspense>
           </div>
@@ -1253,6 +1270,7 @@ export function NodePanel({
   node,
   flow,
   nodeTypes,
+  origin,
   suggestions,
   expanded,
   onChange,
@@ -1266,6 +1284,8 @@ export function NodePanel({
   node: NodeDef_Input | null
   flow: string
   nodeTypes: NodeTypeInfo[]
+  /** Set when the flow was declared in code elsewhere; its bodies are generated. */
+  origin: FlowOrigin | null | undefined
   suggestions: PortSuggestions
   expanded: boolean
   onChange: (next: NodeDef_Input) => void
@@ -1318,6 +1338,7 @@ export function NodePanel({
           node={node}
           flow={flow}
           nodeType={nodeType}
+          origin={origin}
           suggestions={suggestions}
           expanded={expanded}
           onChange={onChange}

worker/fluksio_worker/worker_main.py

@@ -124,6 +124,52 @@ class _Reporter(ModuleType):
             raise ValueError("not an artifact reference")
         return _fetch(digest)
 
+    # ---------------------------------------------------------------------
+    # The authoring API, inert.
+    #
+    # A node generated by `fluksio sync` imports the caller's own module, and
+    # that module says `from fluksio import Port, node, Flow` at the top —
+    # which, in here, is this. The declarations were read at sync time and are
+    # already in the flow document, so what they have to do now is import
+    # without doing anything: the decorators hand the function back, and `flow`
+    # builds nothing.
+    # ---------------------------------------------------------------------
+
+    def Port(self, *args: Any, **kwargs: Any) -> Any:  # noqa: N802
+        """A port declaration, already read by `fluksio sync`."""
+        return _Declared()
+
+    def node(self, *args: Any, **kwargs: Any) -> Any:
+        """The decorator, which here gives the function straight back."""
+        return lambda fn: fn
+
+    def use(self, *args: Any, **kwargs: Any) -> Any:
+        """One use of a node in a flow, already read by `fluksio sync`."""
+        return _Declared()
+
+    def Flow(self, *args: Any, **kwargs: Any) -> Any:  # noqa: N802
+        """A flow declaration, already read by `fluksio sync`."""
+        return _Declared()
+
+
+class _Declared:
+    """Stands in for a declaration whose work was done before the run.
+
+    Tolerant on purpose: a module may keep one at module level and touch it in
+    ways a node never exercises, and none of that should fail an import.
+    """
+
+    def __getattr__(self, name: str) -> Any:
+        if name.startswith("__"):
+            raise AttributeError(name)
+        return _Declared()
+
+    def __call__(self, *args: Any, **kwargs: Any) -> Any:
+        return _Declared()
+
+    def __repr__(self) -> str:
+        return "<fluksio declaration>"
+
 
 def _store_bytes(data: bytes, name: str, media_type: str) -> dict[str, Any]:
     """Write to the artifact store, whichever end of it this worker can see.