Add a Python SDK: flows declared in your own repository

A data scientist keeps their code where it is and decorates it: `@node`
declares a function's ports beside the function, `Flow(name, nodes=[...])`
says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds
one for a single flow. `fluksio sync` uploads the document plus a generated
import shim per node, so the store still holds a complete, runnable,
git-versioned definition while the code it imports stays theirs.

`fluksio login|run|runs` and `flow.submit().wait()` are the client half, over
the run endpoints that already existed. Runs record the user repository's
commit beside the store's, so "what code produced this number" is answerable
on the side that now holds the code.

- `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim
  generator, an HTTP client and sync. Standard library only at import, so
  `from fluksio import node` in a training script pulls in no engine.
- `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the
  repository's commit; `POST /modules/refresh` retires the workers without an
  install, which every sync calls — a worker holds the imported package in
  memory, so an edit to it is invisible until the process goes.
- The canvas shows a generated body read-only and names the repository to edit
  instead; a body edited there stops the next sync rather than being discarded.
- The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since
  the shim imports a module whose first line declares them.
- `examples/myresearch` is the worked example, `make sync-example` uploads it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
This commit is contained in:
2026-08-23 20:16:08 +02:00
co-authored by Claude Fable 5
parent 775d151307
commit a38e2745eb
35 changed files with 2693 additions and 142 deletions
+61
View File
@@ -11,6 +11,10 @@ There is a second, smaller distribution — `fluksio-worker` — for a machine t
should only *run nodes* for an engine elsewhere. It has none of the engine in
it. See [Remote workers](workers.md).
The command is two things at once: `serve`, `enroll` and `worker` *are* an
installation, while `login`, `sync`, `run` and `runs` talk to one that may be
anywhere.
## `fluksio serve`
Runs the engine.
@@ -100,6 +104,63 @@ fluksio worker --url wss://api.example.com/api/v1/workers/attach \
See [Remote workers](workers.md).
## Talking to an engine
The four commands below are the client half: they run wherever you work, and
address an engine over its API rather than being one.
### `fluksio login`
```sh
fluksio login --url http://127.0.0.1:8000
```
Asks for an email and password, and keeps the token it gets in
`~/.config/fluksio/client.json` (`$XDG_CONFIG_HOME` is honoured). Everything
below reads it from there, or from `FLUKSIO_URL` and `FLUKSIO_TOKEN`, or from
its own `--url` and `--token`.
### `fluksio sync`
```sh
fluksio sync [PATH_OR_MODULE ...] # default: the current directory
```
Imports what you name, collects the flows the decorators declared, and uploads
each one with a generated import shim per node. A directory that is a package
is walked; a dotted name is imported as it stands; nothing is loaded from a
file path, because the shim has to import the same way.
| Flag | What it does |
|---|---|
| `--dry-run` | print the flow documents and shims, upload nothing |
| `--no-publish` | leave the upload as a draft |
| `--force` | overwrite a flow, or a node body, that was edited on the canvas |
Every sync retires the engine's workers, including one that had nothing to
upload — a worker holds your package in memory, so an edit to it is invisible
until the process goes. See
[Getting started: data science](../getting-started/data-science.md).
### `fluksio run`
```sh
fluksio run train --lr 0.05 --seed 7 [--wait]
```
Submits a run. Flags that are not its own are the flow's inputs, typed by what
the flow declares them as. `--wait` blocks until the run finishes and exits
non-zero if it failed.
### `fluksio runs`
```sh
fluksio runs [--flow train] [--limit 20]
```
The runs an engine has recorded, newest first: id, status, flow, duration, the
commit of the repository it came from, and its parameters.
## What lives in the data directory
```text
+9
View File
@@ -204,6 +204,15 @@ def process(lr, epochs):
That is the whole of it. The node is three lines, `myresearch` can be as many
modules as it likes, and nothing was copied.
!!! tip "You can have those three lines written for you"
Decorate `fit` with `@node(...)` where it is defined, say which nodes make
a flow with `Flow(...)`, and `fluksio sync` generates the body above —
along with the flow document, so there is nothing to PUT by hand. The
declaration lives beside the function it describes and is checked against
its signature. See
[Getting started: data science](../getting-started/data-science.md).
!!! warning "Editable, but not live"
`-e` means edits reach the venv without reinstalling — but a node's process