Docs: a node can import the project you already have

"What it can import is what the Modules screen installed" was true and
read as a wall: every example is a self-contained file, and a function
that imports half a repository looks unsupported. It is not — the
manifest is handed to `uv pip sync` verbatim, so `-e /home/you/repo`
installs the project you already have and a node becomes a three-line
wrapper over it. The code stays in your own repository, under your own
version control, importing its own siblings.

Two caveats that are easy to lose an afternoon to, so both are written
down: an editable install reaches the venv without a reinstall but not
into a worker that already imported it — the engine's workers are
long-lived, and Apply is what retires them, while an attached worker
starts a process per call and is always fresh. And the path is resolved
on whichever machine runs the node, while the manifest is committed to
the flow repository, so a laptop path means nothing in a container.

Verified rather than assumed: `uv pip sync` takes `-e`, the shim body
loads through the real `load_function`, an edit is live in a fresh
process and stale in a reused one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-22 10:31:14 +02:00
co-authored by Claude Opus 5
parent 3f3585ed55
commit e5be1880fd
2 changed files with 90 additions and 1 deletions
+43
View File
@@ -140,6 +140,49 @@ function is invisible to all three.
checkpoint, a dataset, a plot. It stores the bytes by their hash and returns a
small reference. Nothing changes about how you write the file.
### If your code does not fit in one file
The example above is a single self-contained file, which most real projects are
not. If the function you want as a node imports half your repository, do not
move it — install the repository into the venv the nodes run on, by adding one
line to the manifest you just applied:
```text
-e /home/you/my-research
```
Now the node is a wrapper over what you already have, and your code stays in
your own repository, under your own version control, importing its own
siblings as it always did:
```python
"""The node. The training lives in the project, where it belongs."""
from myresearch.train import fit
def process(lr, epochs):
return fit(lr, epochs)
```
A generator still works through the wrapper — `yield from fit(...)` — so the
per-epoch metrics arrive exactly as before.
!!! warning "Apply after you edit"
The engine's workers are long-lived and hold your imported modules in
memory, so a change to `myresearch` is picked up when they are retired —
which is what **Apply** does. If you are editing many times an hour, attach
your own interpreter as a worker instead: it starts a process per call and
therefore reads your code fresh every run.
```sh
fluksio worker --url ws://127.0.0.1:8000/api/v1/workers/attach \
--token "$WORKER_TOKEN" --labels local --python "$(which python)"
```
Then mark the node `"device": "local"`.
!!! note "Where a `yield` cannot reach"
If the number comes from inside somebody else's callback — Keras, Lightning,