Make the docs state things rather than argue them
Docs / docs (push) Successful in 37s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m35s
Playwright Tests / test-playwright (2, 2) (push) Failing after 17s
pre-commit / pre-commit (push) Failing after 2m8s
Test Backend / test-backend (push) Failing after 2m48s
Compose Smoke Test / test-compose (push) Failing after 13s
Playwright Tests / merge-reports (push) Failing after 2m25s
Docs / docs (push) Successful in 37s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m35s
Playwright Tests / test-playwright (2, 2) (push) Failing after 17s
pre-commit / pre-commit (push) Failing after 2m8s
Test Backend / test-backend (push) Failing after 2m48s
Compose Smoke Test / test-compose (push) Failing after 13s
Playwright Tests / merge-reports (push) Failing after 2m25s
The site read as a design journal: rationale paragraphs, hedges
("deliberately", "on purpose", "genuinely"), meta-commentary about the docs
themselves, and one em-dash every ten lines carrying an aside.
Roughly twenty rationale blocks are gone or reduced to what a reader needs
in order to use the thing. Em-dashes go from 507 to 135, and what is left is
structural rather than prose: list and definition separators, table cells,
and four inside code blocks that quote what the CLI actually prints.
Also: api.example.com becomes api.fluksio.com (the emails stay, since
bootstrap.py really defaults to admin@example.com and RFC 2606 reserves it);
the mqtt table gains the two settings it had drifted behind on and inject's
wording matches the engine; llms.txt lists the two connector pages that were
in the nav but not in it; and the two device/device_policy notes now agree.
Builds clean under `zensical build --strict`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YrQnKV3bnQd4K342y8tKj
This commit is contained in:
+30
-31
@@ -1,7 +1,7 @@
|
||||
# Writing node code
|
||||
|
||||
A Function node is a Python file. That is all it is — no base class, no
|
||||
decorator, no framework import unless you want one.
|
||||
A Function node is a Python file: no base class, no decorator, no framework
|
||||
import unless you want one.
|
||||
|
||||
```python
|
||||
def process(temperature, setpoint=21.0):
|
||||
@@ -22,18 +22,18 @@ See [Where a node's values come from](../concepts/values.md).
|
||||
|
||||
**The return value is a dict keyed by output ports.** Every value is checked
|
||||
against the port's declared type before it is published. A key that is not a
|
||||
declared port is an error, not a silent drop — nothing leaves a node except
|
||||
declared port is an error, not a silent drop. Nothing leaves a node except
|
||||
through a port it declared.
|
||||
|
||||
**Nothing else is importable from the engine.** Node code runs in a separate
|
||||
process, on a separate interpreter, with none of Fluksio's own modules on its
|
||||
path. What it can import is what the [Modules](../interface/operations.md)
|
||||
screen installed — which can include [your own project](#your-own-code-as-a-package),
|
||||
screen installed, which can include [your own project](#your-own-code-as-a-package),
|
||||
so a node need not be a self-contained file.
|
||||
|
||||
**A node is a pure function of its inputs.** No context object, no global
|
||||
store, no handle to reach for. A running total or a debounce timer has a
|
||||
specific shape — see [Keeping state in a flow](../concepts/state.md).
|
||||
specific shape; see [Keeping state in a flow](../concepts/state.md).
|
||||
|
||||
## Producing values over time
|
||||
|
||||
@@ -49,7 +49,7 @@ def process(lr, steps):
|
||||
return {"final_loss": loss}
|
||||
```
|
||||
|
||||
Whatever the generator `return`s at the end is the node's result — what
|
||||
Whatever the generator `return`s at the end is the node's result, which is what
|
||||
downstream nodes read. If you never `return`, the last thing you yield is the
|
||||
result instead.
|
||||
|
||||
@@ -60,15 +60,15 @@ Mark the port so the flow says what it does:
|
||||
```
|
||||
|
||||
Two consequences. In a [run](../concepts/runs.md), the whole series is kept as
|
||||
that run's metrics — this is why there is no `log_metric()` anywhere in the
|
||||
that run's metrics, which is why there is no `log_metric()` anywhere in the
|
||||
API. And **the node's timeout starts measuring silence rather than duration**:
|
||||
each emission resets the deadline, so a node yielding every few seconds can run
|
||||
for hours under a timeout of 300.
|
||||
|
||||
### `fluksio.emit`
|
||||
|
||||
Where a `yield` cannot reach — the value comes from inside somebody else's
|
||||
callback, and they call you rather than the other way round:
|
||||
Where a `yield` cannot reach, because the value comes from inside somebody
|
||||
else's callback and they call you rather than the other way round:
|
||||
|
||||
```python
|
||||
import fluksio
|
||||
@@ -103,19 +103,19 @@ def process(dataset):
|
||||
```
|
||||
|
||||
`save_artifact` takes bytes or a path, stores them by their SHA-256 digest, and
|
||||
returns a small reference — digest, size, media type, name — which is what an
|
||||
returns a small reference (digest, size, media type, name) which is what an
|
||||
`artifact`-typed port carries.
|
||||
|
||||
Because the address is the content's hash, a sweep whose fifty configs share
|
||||
one preprocessed input stores it once, and a reference stays valid wherever the
|
||||
store is reachable from — including on another machine.
|
||||
store is reachable from, including on another machine.
|
||||
|
||||
`fluksio artifacts RUN NAME` is how one comes back out at a terminal.
|
||||
|
||||
### `fluksio.logger`
|
||||
|
||||
A node's `print` is kept as that node's logs, and so is anything on
|
||||
`fluksio.logger` — the same logger the SDK exports at top level, so code that
|
||||
`fluksio.logger` is the same logger the SDK exports at top level, so code that
|
||||
runs both inside a node and outside one says it the same way:
|
||||
|
||||
```python
|
||||
@@ -152,7 +152,7 @@ each carries and what rates are realistic.
|
||||
|
||||
Only what a node *returns* is recorded against its run. Frames yielded
|
||||
along the way are replaced in state by the next one, and the artifact sweep
|
||||
removes bytes nothing refers to any more — which is what stops a camera
|
||||
removes bytes nothing refers to any more, which is what stops a camera
|
||||
filling the disk. If a particular frame matters, return it.
|
||||
|
||||
## Printing
|
||||
@@ -161,14 +161,13 @@ each carries and what rates are realistic.
|
||||
the flow editor's log panel and on the run's per-node record; the rest is
|
||||
dropped, so a node printing in a loop cannot fill anything up.
|
||||
|
||||
Use it to debug. Do not use it to record results — a number worth keeping is an
|
||||
output port, not a line of text.
|
||||
Use it to debug, not to record results. A number worth keeping is an output
|
||||
port, not a line of text.
|
||||
|
||||
## Errors
|
||||
|
||||
An exception fails that node's execution, not the flow. The message you see is
|
||||
one line from the frame in *your* code, not a stack through the engine — that
|
||||
is a deliberate choice about what is actionable.
|
||||
one line from the frame in *your* code, not a stack through the engine.
|
||||
|
||||
The node keeps its last error visible after it recovers, so a failure that
|
||||
fired an alert at 03:00 still says what it was at 09:00. It can also be
|
||||
@@ -178,13 +177,13 @@ acknowledged from the canvas.
|
||||
|
||||
`timeout` on a node is how many seconds its code may be *silent* before it is
|
||||
stopped. A yield or an `emit` resets the clock, and the first call's imports are
|
||||
not charged to it — a node importing torch is not being slow, it is loading.
|
||||
not charged to it: a node importing torch is not being slow, it is loading.
|
||||
|
||||
There is no timeout by default. Training runs for hours and a node that reports
|
||||
nothing is usually working, so the engine waits: what fails a call is the worker
|
||||
dying, which arrives at once rather than after a deadline. Set a timeout on the
|
||||
nodes where silence means stuck — an HTTP call that should answer in seconds,
|
||||
a loop that can spin — either per node or, for all of them,
|
||||
nodes where silence means stuck (an HTTP call that should answer in seconds, a
|
||||
loop that can spin) either per node or, for all of them,
|
||||
with `FLOW_NODE_TIMEOUT`. `timeout = 0` says explicitly that this node has none.
|
||||
|
||||
## Running a node somewhere else
|
||||
@@ -197,7 +196,7 @@ A node declares the label of the machine it needs:
|
||||
|
||||
`require` (the default) waits for a worker carrying that label; `prefer` runs
|
||||
locally when none is attached. A node bound to a device is compiled *on that
|
||||
machine* — a node importing `torch` is correct on the GPU box and a missing
|
||||
machine*: a node importing `torch` is correct on the GPU box and a missing
|
||||
module on the engine, so checking it here would fail something that is fine.
|
||||
|
||||
See [Remote workers](workers.md).
|
||||
@@ -205,8 +204,8 @@ See [Remote workers](workers.md).
|
||||
## Sharing code between flows
|
||||
|
||||
A node's source can be promoted to the shared library from its panel, and other
|
||||
flows can then use it by reference. One copy, one place to edit — and every
|
||||
flow using it runs the edit, which is the point and also the caution.
|
||||
flows can then use it by reference. There is one copy and one place to edit, and
|
||||
every flow using it runs the edit.
|
||||
|
||||
Shared sources live in `_lib/` in the flow repository, so they are versioned
|
||||
with everything else.
|
||||
@@ -214,7 +213,7 @@ with everything else.
|
||||
## Packages
|
||||
|
||||
Node code runs in a virtual environment of its own, on the instance's data
|
||||
volume — deliberately separate from the one Fluksio itself runs on.
|
||||
volume, separate from the one Fluksio itself runs on.
|
||||
|
||||
Declare what you import in [Modules](../interface/operations.md), or over the
|
||||
API:
|
||||
@@ -233,7 +232,7 @@ flows. An install takes effect immediately; nothing restarts.
|
||||
!!! tip "If Fluksio is installed in the venv you work in, skip this"
|
||||
|
||||
Node code then runs on that environment, so your project and everything it
|
||||
imports are already importable — see
|
||||
imports are already importable; see
|
||||
[Getting started: data science](../getting-started/data-science.md). What
|
||||
follows is for a Fluksio with a venv of its own, which is what a container
|
||||
always has.
|
||||
@@ -246,7 +245,7 @@ installable like any other dependency:
|
||||
numpy>=2
|
||||
```
|
||||
|
||||
A node body then imports it, and the logic stays where it already lives — in
|
||||
A node body then imports it, and the logic stays where it already lives: in
|
||||
your repository, under your own version control, importing its own siblings:
|
||||
|
||||
```python
|
||||
@@ -263,7 +262,7 @@ modules as it likes, and nothing was copied.
|
||||
!!! tip "You can have those three lines written for you"
|
||||
|
||||
Decorate `fit` with `@node(...)` where it is defined, say which nodes make
|
||||
a flow with `Flow(...)`, and `fluksio sync` generates the body above —
|
||||
a flow with `Flow(...)`, and `fluksio sync` generates the body above,
|
||||
along with the flow document, so there is nothing to PUT by hand. The
|
||||
declaration lives beside the function it describes and is checked against
|
||||
its signature. See
|
||||
@@ -271,7 +270,7 @@ modules as it likes, and nothing was copied.
|
||||
|
||||
!!! warning "Editable, but not live"
|
||||
|
||||
`-e` means edits reach the venv without reinstalling — but a node's process
|
||||
`-e` means edits reach the venv without reinstalling, but a node's process
|
||||
already holds the imported module in memory. The engine's workers are
|
||||
long-lived, so a change to your code is picked up when they are retired,
|
||||
which is what **Apply** does. Pressing it after an edit is the loop.
|
||||
@@ -279,20 +278,20 @@ modules as it likes, and nothing was copied.
|
||||
A [worker](workers.md) you attach yourself is the exception: it starts a
|
||||
process per call, so it reads your code fresh every run. If you are
|
||||
iterating on the code many times an hour, point one at your own
|
||||
interpreter — `fluksio-worker --python "$(which python)"` — and mark the
|
||||
interpreter (`fluksio-worker --python "$(which python)"`) and mark the
|
||||
node with its label.
|
||||
|
||||
!!! note "The path is a deployment detail"
|
||||
|
||||
It is resolved on whichever machine runs the node, and the manifest is
|
||||
committed to the flow repository — so an absolute path from your laptop
|
||||
committed to the flow repository, so an absolute path from your laptop
|
||||
means nothing inside a container or on a GPU box. Those need their own
|
||||
install of the same project; a VCS requirement
|
||||
(`myresearch @ git+ssh://…@a1b2c3d`) travels where a path does not.
|
||||
|
||||
## A worked example
|
||||
|
||||
The repository ships a small supervised fit as a seedable demo — three nodes,
|
||||
The repository ships a small supervised fit as a seedable demo: three nodes,
|
||||
a batch flow, streaming metrics, artifacts between stages, and a GPU-labelled
|
||||
node that falls back to the engine when no worker is attached. It is the
|
||||
shortest complete thing to read:
|
||||
|
||||
Reference in New Issue
Block a user