Publish the documentation site: docs.fluksio.com

A zensical site under docs/, served by a new `docs` compose service behind
Traefik, built with --strict in CI. Same pattern the sibling n3xd workspace
uses.

Getting started splits the way the landing page does — one path is
`pip install fluksio` and a training script, the other is a Docker stack and
an afternoon in the browser — because the two audiences will not spend the same
amount of time. Everything after that is shared: the concepts, the web
interface (app and portal), the CLI and the API, and a reference for node types,
payload types and configuration.

The three flow guides move here from the docs submodule rather than being
copied, so there is one version of them.

Styling mirrors DESIGN-GUIDELINES.md: the app's token palette remapped onto
Material's variables in both schemes, Inter, the 16px panel radius, and the one
terracotta accent spent on the facility lane of the audience split.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M7Xv3cJEW5c8AXxn2hoojV
This commit is contained in:
2026-08-22 05:55:34 +02:00
co-authored by Claude Opus 5
parent c3cbbbc962
commit d12c81c8a0
34 changed files with 4167 additions and 2 deletions
+137
View File
@@ -0,0 +1,137 @@
# Remote workers
The engine runs where the automations are. The GPU is somewhere else, the
Raspberry Pi with the relays is in a shed, and neither of them is on the same
network as the other.
A **worker** is a process that runs the code of nodes marked for it. It dials
*out* to the engine over one authenticated websocket, so nothing on that
machine has to be reachable — and nothing has to expose the engine's state
backend across hosts, which it never should.
## Install and attach
```sh
pip install fluksio-worker
fluksio-worker \
--url wss://api.example.com/api/v1/workers/attach \
--token "$FLUKSIO_WORKER_TOKEN" \
--labels gpu,cuda12 \
--python /opt/torch-venv/bin/python
```
`fluksio-worker` is its own distribution — the agent, the node runner, and
`websockets`. Nothing of the engine, so a GPU box does not install a database
driver in order to run a training step. An engine host already has it, and
`fluksio worker …` is the same program.
| Option | Default | What it does |
|---|---|---|
| `--url` | **required** | `wss://…/api/v1/workers/attach` |
| `--token` | `$FLUKSIO_WORKER_TOKEN` | the credential, minted on the engine |
| `--name` | this host's name | how it shows up in the worker list |
| `--labels` | none | comma-separated; what a node's `device` matches |
| `--python` | this interpreter | the interpreter node code runs on |
| `--parallel` | `1` | how many node calls it will take at once |
| `--artifact-url` | derived from `--url` | where the artifact store is, if not beside the socket |
`--python` is the important one. It is how this machine keeps its own wheels —
the CUDA build, the vendor SDK, the thing that will not install anywhere else —
without the engine ever installing them or knowing about them.
## Mint the token
On the engine, as a superuser:
```sh
curl -X POST $FLUKSIO/workers/tokens -H "Authorization: Bearer $TOKEN" \
-H 'Content-Type: application/json' -d '{"name": "gpu-dev"}'
```
Shown once, valid for a year — a worker is a machine somebody sets up and
leaves running. It is signed with the same keypair agent tokens use, so
rotating that key revokes every worker along with them.
??? note "A host where pip is not an option"
The two files work copied into one directory and run with `python agent.py
…`. The engine serves the runner itself at `GET /api/v1/workers/runtime`
it is the same module its own local workers run, deliberately standard
library only.
## Send a node to it
A node declares the label of the machine it needs:
```json
{
"id": "train",
"device": "gpu",
"device_policy": "require",
"timeout": 7200
}
```
| `device_policy` | Behaviour when nothing carrying the label is attached |
|---|---|
| `require` (default) | the run stays `queued` and says what it is waiting for |
| `prefer` | it runs on the engine instead |
`prefer` is what makes a flow work before the GPU box exists. `require` is what
you want once it does.
!!! note "Set from the API"
`device` and `device_policy` are not yet fields in the node panel. Set them
with `PUT /flows/{name}`.
## What follows from this
- **The node's source travels with every call.** Nothing has to be deployed to
the worker, and changing a node's code takes effect on the next execution.
- **A node bound to a device is compiled on that machine.** A node importing
`torch` is correct on the GPU box and a missing module on the engine, so
checking it here would fail something that is fine.
- **`import fluksio` inside a node is the worker's own reporter.** `emit`,
`save_artifact`, `load_artifact` — installed before your code runs, so an
installed `fluksio` package on that box never shadows it.
- **Cancelling a run kills what it is executing**, there or here, and leaves
other runs of the same node alone.
- **If the worker disappears mid-call**, the run fails in seconds with
`worker went away mid-call` rather than waiting out its timeout.
- **A worker sends a heartbeat while it executes**, so a long node is
distinguishable from a dead socket. Ninety seconds of silence is gone.
## Artifacts across machines
An artifact reference names content by its hash, not a location, so it stays
valid wherever the store is reachable from. A worker that shares the engine's
filesystem writes to it directly; one that does not fetches and uploads over
HTTP, using the artifact endpoint beside the socket it already has. Either way
your node code is the same two calls.
## Seeing what is attached
```sh
curl -s $FLUKSIO/workers -H "Authorization: Bearer $TOKEN" | jq
```
Name, labels, how many calls it will take at once, how many are in flight, when
it attached, when it was last seen, its Python version, and a digest of its
environment.
## What a worker is not
It is not a second engine. Subscriptions, schedules, webhooks, the dashboards
and the run queue all stay in one process — that is what keeps a message having
one definition and a cron tick happening once. A worker executes node bodies.
Running two engines against one data directory is not supported. Distribute
work with workers.
## See also
- [Runs: pipelines that finish](../concepts/runs.md#running-a-node-somewhere-else)
- [Writing node code](nodes.md)
- [Getting started: data science](../getting-started/data-science.md)