Files
app/docs/code/nodes.md
T
stroblmeandClaude Opus 5 11e032386b Publish the documentation site: docs.fluksio.com
A zensical site under docs/, served by a new `docs` compose service behind
Traefik, built with --strict in CI. Same pattern the sibling n3xd workspace
uses.

Getting started splits the way the landing page does — one path is
`pip install fluksio` and a training script, the other is a Docker stack and
an afternoon in the browser — because the two audiences will not spend the same
amount of time. Everything after that is shared: the concepts, the web
interface (app and portal), the CLI and the API, and a reference for node types,
payload types and configuration.

The three flow guides move here from the docs submodule rather than being
copied, so there is one version of them.

Styling mirrors DESIGN-GUIDELINES.md: the app's token palette remapped onto
Material's variables in both schemes, Inter, the 16px panel radius, and the one
terracotta accent spent on the facility lane of the audience split.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M7Xv3cJEW5c8AXxn2hoojV
2026-08-22 05:55:34 +02:00

203 lines
7.1 KiB
Markdown

# Writing node code
A Function node is a Python file. That is all it is — no base class, no
decorator, no framework import unless you want one.
```python
def process(temperature, setpoint=21.0):
"""Ask for heat when the room is below the comfort point."""
return {"heat": temperature < setpoint}
```
## The rules
**One function called `process`.** If the file defines exactly one public
function under another name, that one is used instead. Two, and the node
refuses to load rather than guessing.
**Arguments come from ports and settings, by name.** `temperature` above is an
input port; `setpoint` is a setting typed into the node's panel. Both arrive as
keyword arguments, which is why a setting may not share a name with a port.
See [Where a node's values come from](../concepts/values.md).
**The return value is a dict keyed by output ports.** Every value is checked
against the port's declared type before it is published. A key that is not a
declared port is an error, not a silent drop — nothing leaves a node except
through a port it declared.
**Nothing else is importable from the engine.** Node code runs in a separate
process, on a separate interpreter, with none of Fluksio's own modules on its
path. What it can import is what the [Modules](../interface/operations.md)
screen installed.
**A node is a pure function of its inputs.** No context object, no global
store, no handle to reach for. A running total or a debounce timer has a
specific shape — see [Keeping state in a flow](../concepts/state.md).
## Producing values over time
A node that produces values *during* its execution is a generator. Every
`yield` is a dict keyed by output port, published the instant it happens:
```python
def process(lr, steps):
loss = 1.0
for _ in range(steps):
loss = train_one_step(lr)
yield {"loss": loss} # published now
return {"final_loss": loss}
```
Whatever the generator `return`s at the end is the node's result — what
downstream nodes read. If you never `return`, the last thing you yield is the
result instead.
Mark the port so the flow says what it does:
```json
{"name": "loss", "dtype": "float", "stream": true}
```
Two consequences. In a [run](../concepts/runs.md), the whole series is kept as
that run's metrics — this is why there is no `log_metric()` anywhere in the
API. And **the node's timeout starts measuring silence rather than duration**:
each emission resets the deadline, so a node yielding every few seconds can run
for hours under a timeout of 300.
### `fluksio.emit`
Where a `yield` cannot reach — the value comes from inside somebody else's
callback, and they call you rather than the other way round:
```python
import fluksio
def process():
model.fit(callbacks=[LambdaCallback(
on_epoch_end=lambda epoch, logs: fluksio.emit(loss=logs["loss"])
)])
return {"weights": ...}
```
Same ports, same type checking, same publication. Prefer `yield` where you can
reach it; `emit` where you cannot.
## Bytes: artifacts
Messages are JSON, which is what lets the same value pass through Redis, the
work queue and the worker protocol unchanged. A checkpoint is not that.
```python
import fluksio
def process(dataset):
path = fluksio.load_artifact(dataset) # → a local path to read
...
return {
"weights": fluksio.save_artifact("model.pt", media_type="application/octet-stream"),
"score": 0.94,
}
```
`save_artifact` takes bytes or a path, stores them by their SHA-256 digest, and
returns a small reference — digest, size, media type, name — which is what an
`artifact`-typed port carries.
Because the address is the content's hash, a sweep whose fifty configs share
one preprocessed input stores it once, and a reference stays valid wherever the
store is reachable from — including on another machine.
## Printing
`print` works and is captured. The first 16 KB per call is kept and shown in
the flow editor's log panel and on the run's per-node record; the rest is
dropped, so a node printing in a loop cannot fill anything up.
Use it to debug. Do not use it to record results — a number worth keeping is an
output port, not a line of text.
## Errors
An exception fails that node's execution, not the flow. The message you see is
one line from the frame in *your* code, not a stack through the engine — that
is a deliberate choice about what is actionable.
The node keeps its last error visible after it recovers, so a failure that
fired an alert at 03:00 still says what it was at 09:00. It can also be
acknowledged from the canvas.
## Timeouts
`timeout` on a node is how many seconds its code may run before it is stopped.
The default is 30, and it covers the *first* call's imports, which can be much
slower than the body — a node importing torch is not being slow, it is loading.
Above 60 seconds, a live flow may deliver the same work again while the node is
still running. In a batch run, which never redelivers, it is an idle timeout
instead: silence this long is a kill.
## Running a node somewhere else
A node declares the label of the machine it needs:
```json
{"id": "train", "device": "gpu", "device_policy": "require", "timeout": 7200}
```
`require` (the default) waits for a worker carrying that label; `prefer` runs
locally when none is attached. A node bound to a device is compiled *on that
machine* — a node importing `torch` is correct on the GPU box and a missing
module on the engine, so checking it here would fail something that is fine.
See [Remote workers](workers.md).
## Sharing code between flows
A node's source can be promoted to the shared library from its panel, and other
flows can then use it by reference. One copy, one place to edit — and every
flow using it runs the edit, which is the point and also the caution.
Shared sources live in `_lib/` in the flow repository, so they are versioned
with everything else.
## Packages
Node code runs in a virtual environment of its own, on the installation's data
volume — deliberately separate from the one Fluksio itself runs on.
Declare what you import in [Modules](../interface/operations.md), or over the
API:
```sh
curl -X POST $FLUKSIO/modules/apply -H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' \
-d '{"requirements": "numpy>=2\npandas\n"}'
```
It is a pip manifest installed with `uv pip sync`, versioned alongside your
flows. An install takes effect immediately; nothing restarts.
## A worked example
The repository ships a small supervised fit as a seedable demo — three nodes,
a batch flow, streaming metrics, artifacts between stages, and a GPU-labelled
node that falls back to the engine when no worker is attached. It is the
shortest complete thing to read:
```text
prepare ──dataset(artifact)──▶ train ──weights(artifact)──▶ evaluate
└── loss (streaming float) ──▶ chart
```
`make seed-demo` builds it against a running stack.
## See also
- [Where a node's values come from](../concepts/values.md)
- [Keeping state in a flow](../concepts/state.md)
- [Runs: pipelines that finish](../concepts/runs.md)
- [Node types](../reference/node-types.md) — the ones you do not have to write