Add a Python SDK: flows declared in your own repository

A data scientist keeps their code where it is and decorates it: `@node`
declares a function's ports beside the function, `Flow(name, nodes=[...])`
says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds
one for a single flow. `fluksio sync` uploads the document plus a generated
import shim per node, so the store still holds a complete, runnable,
git-versioned definition while the code it imports stays theirs.

`fluksio login|run|runs` and `flow.submit().wait()` are the client half, over
the run endpoints that already existed. Runs record the user repository's
commit beside the store's, so "what code produced this number" is answerable
on the side that now holds the code.

- `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim
  generator, an HTTP client and sync. Standard library only at import, so
  `from fluksio import node` in a training script pulls in no engine.
- `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the
  repository's commit; `POST /modules/refresh` retires the workers without an
  install, which every sync calls — a worker holds the imported package in
  memory, so an edit to it is invisible until the process goes.
- The canvas shows a generated body read-only and names the repository to edit
  instead; a body edited there stops the next sync rather than being discarded.
- The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since
  the shim imports a module whose first line declares them.
- `examples/myresearch` is the worked example, `make sync-example` uploads it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
This commit is contained in:
2026-08-23 20:16:08 +02:00
co-authored by Claude Fable 5
parent 0a49b6e947
commit 19bc2810cf
35 changed files with 2693 additions and 142 deletions
+46
View File
@@ -124,6 +124,52 @@ class _Reporter(ModuleType):
raise ValueError("not an artifact reference")
return _fetch(digest)
# ---------------------------------------------------------------------
# The authoring API, inert.
#
# A node generated by `fluksio sync` imports the caller's own module, and
# that module says `from fluksio import Port, node, Flow` at the top —
# which, in here, is this. The declarations were read at sync time and are
# already in the flow document, so what they have to do now is import
# without doing anything: the decorators hand the function back, and `flow`
# builds nothing.
# ---------------------------------------------------------------------
def Port(self, *args: Any, **kwargs: Any) -> Any: # noqa: N802
"""A port declaration, already read by `fluksio sync`."""
return _Declared()
def node(self, *args: Any, **kwargs: Any) -> Any:
"""The decorator, which here gives the function straight back."""
return lambda fn: fn
def use(self, *args: Any, **kwargs: Any) -> Any:
"""One use of a node in a flow, already read by `fluksio sync`."""
return _Declared()
def Flow(self, *args: Any, **kwargs: Any) -> Any: # noqa: N802
"""A flow declaration, already read by `fluksio sync`."""
return _Declared()
class _Declared:
"""Stands in for a declaration whose work was done before the run.
Tolerant on purpose: a module may keep one at module level and touch it in
ways a node never exercises, and none of that should fail an import.
"""
def __getattr__(self, name: str) -> Any:
if name.startswith("__"):
raise AttributeError(name)
return _Declared()
def __call__(self, *args: Any, **kwargs: Any) -> Any:
return _Declared()
def __repr__(self) -> str:
return "<fluksio declaration>"
def _store_bytes(data: bytes, name: str, media_type: str) -> dict[str, Any]:
"""Write to the artifact store, whichever end of it this worker can see.