Make the docs state things rather than argue them
Docs / docs (push) Successful in 37s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m35s
Playwright Tests / test-playwright (2, 2) (push) Failing after 17s
pre-commit / pre-commit (push) Failing after 2m8s
Test Backend / test-backend (push) Failing after 2m48s
Compose Smoke Test / test-compose (push) Failing after 13s
Playwright Tests / merge-reports (push) Failing after 2m25s
Docs / docs (push) Successful in 37s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m35s
Playwright Tests / test-playwright (2, 2) (push) Failing after 17s
pre-commit / pre-commit (push) Failing after 2m8s
Test Backend / test-backend (push) Failing after 2m48s
Compose Smoke Test / test-compose (push) Failing after 13s
Playwright Tests / merge-reports (push) Failing after 2m25s
The site read as a design journal: rationale paragraphs, hedges
("deliberately", "on purpose", "genuinely"), meta-commentary about the docs
themselves, and one em-dash every ten lines carrying an aside.
Roughly twenty rationale blocks are gone or reduced to what a reader needs
in order to use the thing. Em-dashes go from 507 to 135, and what is left is
structural rather than prose: list and definition separators, table cells,
and four inside code blocks that quote what the CLI actually prints.
Also: api.example.com becomes api.fluksio.com (the emails stay, since
bootstrap.py really defaults to admin@example.com and RFC 2606 reserves it);
the mqtt table gains the two settings it had drifted behind on and inject's
wording matches the engine; llms.txt lists the two connector pages that were
in the nav but not in it; and the two device/device_policy notes now agree.
Builds clean under `zensical build --strict`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YrQnKV3bnQd4K342y8tKj
This commit is contained in:
@@ -32,8 +32,8 @@ Fluksio 0.1.0 — data in /home/you/my-research/.fluksio
|
||||
```
|
||||
|
||||
Read the last two lines: you are already signed in. Signing in to your own
|
||||
machine is a formality — the password was printed by the same process that
|
||||
would have checked it — so `serve` writes the token itself and every command
|
||||
machine is a formality, since the password was printed by the same process that
|
||||
would have checked it, so `serve` writes the token itself and every command
|
||||
below just works. `fluksio login` is for an engine somewhere *else*.
|
||||
|
||||
**Write that password down** anyway. It is shown once, and it is what the
|
||||
@@ -41,7 +41,7 @@ dashboard asks for.
|
||||
|
||||
### One instance per project
|
||||
|
||||
`.fluksio` sits beside your code, and is found the way `.git` is — from the
|
||||
`.fluksio` sits beside your code, and is found the way `.git` is: from the
|
||||
directory you are standing in, or any directory above it. So two repositories
|
||||
on one machine are two engines: separate flows, separate run history,
|
||||
separate token, and no chance of one experiment's graph turning up in the
|
||||
@@ -59,7 +59,7 @@ other's.
|
||||
climate/
|
||||
```
|
||||
|
||||
It holds a database and a credential, so it ignores itself from within — a
|
||||
It holds a database and a credential, so it ignores itself from within with a
|
||||
`.gitignore` of `*`, the same thing `uv` writes into `.venv`. Nothing to add
|
||||
to your project's own.
|
||||
|
||||
@@ -67,7 +67,7 @@ Give them different ports (`--port`) if you want two running at once.
|
||||
|
||||
!!! tip "One engine for the machine instead"
|
||||
|
||||
`fluksio serve --global` uses `~/.fluksio` — shared by every directory,
|
||||
`fluksio serve --global` uses `~/.fluksio`, shared by every directory,
|
||||
which is what you want for a personal server rather than a project. When
|
||||
both exist, the banner says which one you are looking at and how to reach
|
||||
the other. `--data-dir` still names any directory outright, which is worth
|
||||
@@ -77,10 +77,9 @@ Give them different ports (`--port`) if you want two running at once.
|
||||
|
||||
!!! tip "Keep it running"
|
||||
|
||||
The engine is meant to be resident. That is the whole reason submitting a
|
||||
run costs about 15 ms instead of the second a project-bootstrapping
|
||||
orchestrator spends before it does anything. Leave it in a `tmux` window,
|
||||
or write a small `systemd --user` unit for it.
|
||||
The engine is meant to be resident, which is what makes submitting a run
|
||||
cost about 15 ms. Leave it in a `tmux` window, or write a small
|
||||
`systemd --user` unit for it.
|
||||
|
||||
## Talking to it over HTTP
|
||||
|
||||
@@ -96,13 +95,13 @@ While you are experimenting, the interactive schema at
|
||||
<http://127.0.0.1:8000/docs> is the fastest way to see what is available.
|
||||
|
||||
For an engine on another machine, `fluksio login --url https://…` asks for a
|
||||
password and stores the token the same way — in this project's `.fluksio`, or
|
||||
password and stores the token the same way: in this project's `.fluksio`, or
|
||||
with `--global` in `~/.fluksio`.
|
||||
|
||||
## Your packages are already there
|
||||
|
||||
If you installed Fluksio into the environment you work in — the venv that
|
||||
already has torch or numpy in it — that is the environment your nodes run on.
|
||||
If you installed Fluksio into the environment you work in (the venv that
|
||||
already has torch or numpy in it) that is the environment your nodes run on.
|
||||
Nothing to declare, nothing to install twice:
|
||||
|
||||
```sh
|
||||
@@ -119,8 +118,8 @@ fluksio serve
|
||||
your environment, adopted. Add packages with pip.
|
||||
```
|
||||
|
||||
That venv is yours. Add a package the way you added the rest — `pip install
|
||||
scikit-learn` — and `fluksio sync` (or a restart) retires the workers so they
|
||||
That venv is yours. Add a package the way you added the rest (`pip install
|
||||
scikit-learn`) and `fluksio sync` (or a restart) retires the workers so they
|
||||
pick it up. The Modules screen lists what is installed and stays read-only,
|
||||
because the alternative would be Fluksio deciding what belongs in an
|
||||
environment it did not make.
|
||||
@@ -130,7 +129,7 @@ environment it did not make.
|
||||
The cost of not having two environments: a package the engine depends on
|
||||
is one you can now upgrade out from under it. In practice this is what
|
||||
everybody does with every other tool in the venv, and the answer when it
|
||||
bites is the same — pin it back, or keep Fluksio somewhere separate with
|
||||
bites is the same: pin it back, or keep Fluksio somewhere separate with
|
||||
the venv of its own below.
|
||||
|
||||
### A venv of Fluksio's own
|
||||
@@ -144,7 +143,7 @@ or an environment too precious to let a node's dependency near. Set
|
||||
a venv of its own; the Modules screen installs into it.
|
||||
```
|
||||
|
||||
Then the Modules screen is how packages get in — a pip manifest, installed
|
||||
Then the Modules screen is how packages get in: a pip manifest, installed
|
||||
with `uv pip sync` and versioned alongside your flows, so what a run imported
|
||||
is recorded with what it ran:
|
||||
|
||||
@@ -156,7 +155,7 @@ curl -X POST $FLUKSIO/modules/apply -H "Authorization: Bearer $TOKEN" \
|
||||
|
||||
Adding a package takes effect immediately; nothing restarts. The Docker image
|
||||
sets `NODE_VENV=managed` for itself, because the venv in it holds the app and
|
||||
nothing of yours — so a container is always this case.
|
||||
nothing of yours, so a container is always this case.
|
||||
|
||||
!!! tip "A GPU box works the same way"
|
||||
|
||||
@@ -169,13 +168,13 @@ nothing of yours — so a container is always this case.
|
||||
--token "$WORKER_TOKEN" --labels gpu
|
||||
```
|
||||
|
||||
Then give the node `device="gpu"` — see [Remote workers](../code/workers.md).
|
||||
Then give the node `device="gpu"`; see [Remote workers](../code/workers.md).
|
||||
|
||||
## Say which functions are nodes
|
||||
|
||||
A **flow** is a graph of nodes. A **batch flow** is one that runs on demand
|
||||
from parameters to a result, which is what an experiment is. A node is one of
|
||||
your own functions — it stays in your repository, imported by its siblings as
|
||||
your own functions. It stays in your repository, imported by its siblings as
|
||||
it always was.
|
||||
|
||||
Say your project looks roughly like this:
|
||||
@@ -222,13 +221,13 @@ def fit(dataset, lr, epochs=25):
|
||||
```
|
||||
|
||||
`yield` replaces `print`. Each one publishes on the node's `loss` port the
|
||||
instant it happens, and the run keeps every value as a series — which is why
|
||||
instant it happens, and the run keeps every value as a series, which is why
|
||||
Fluksio has no `log_metric()` call. A metric that leaves through a port is a
|
||||
message like any other: a chart can bind to it, a downstream node can consume
|
||||
it, and it shows up on the canvas. A metric that escapes through a logging
|
||||
function is invisible to all three.
|
||||
|
||||
`fluksio.save_artifact` handles the things too big to be messages — a
|
||||
`fluksio.save_artifact` handles the things too big to be messages: a
|
||||
checkpoint, a dataset, a plot. It stores the bytes by their hash and returns a
|
||||
small reference. Nothing changes about how you write the file.
|
||||
|
||||
@@ -241,24 +240,24 @@ stops your own code from running.
|
||||
|
||||
- **Ports are declared, never inferred.** `requires` names the messages the
|
||||
node reads; each arrives as the parameter of the same name. A bare string is
|
||||
the shorthand — `"dataset"` takes its type from whoever provides it, so a
|
||||
the shorthand: `"dataset"` takes its type from whoever provides it, so a
|
||||
message's type is written down exactly once.
|
||||
- **Parameters with a default that are not ports become settings.** `epochs`
|
||||
above is stored with the flow and tunable on the canvas without touching
|
||||
this file.
|
||||
- **The declaration is checked against the function.** A port with no matching
|
||||
parameter, or a parameter that is neither port nor setting, is an error when
|
||||
the module is imported — not when the node is first called.
|
||||
the module is imported, not when the node is first called.
|
||||
- **The decorator hands the function back untouched.** `fit(dataset, lr=0.05)`
|
||||
is an ordinary call, and the module is an ordinary script — so a
|
||||
is an ordinary call, and the module is an ordinary script, so a
|
||||
`if __name__ == "__main__":` check beside the nodes runs. Syncing one
|
||||
defined in a script *run as* `__main__` is what cannot work, since the
|
||||
generated body would have no name to import it by.
|
||||
|
||||
!!! note "Where a `yield` cannot reach"
|
||||
|
||||
If the number comes from inside somebody else's callback — Keras, Lightning,
|
||||
HuggingFace `Trainer` — `fluksio.emit(loss=...)` writes the same port the
|
||||
If the number comes from inside somebody else's callback (Keras, Lightning,
|
||||
HuggingFace `Trainer`) `fluksio.emit(loss=...)` writes the same port the
|
||||
same way:
|
||||
|
||||
```python
|
||||
@@ -271,7 +270,7 @@ stops your own code from running.
|
||||
|
||||
A training node is not like the rest of a flow. NumPy, JAX and PyTorch each
|
||||
size their thread pool to every core they can see, so a few of them at once
|
||||
oversubscribe the machine badly enough to starve the engine itself — the API
|
||||
oversubscribe the machine badly enough to starve the engine itself: the API
|
||||
stops answering and every client waiting on a run dies with it. On a GPU the
|
||||
same shape deadlocks: two processes each preallocating most of the card sit at
|
||||
zero utilisation with nothing failing and nothing to read.
|
||||
@@ -289,7 +288,7 @@ def finetune(checkpoint):
|
||||
```
|
||||
|
||||
Two things follow. The node waits its turn rather than starting alongside more
|
||||
of itself than the machine has room for — the same waiting it already does for
|
||||
of itself than the machine has room for, the same waiting it already does for
|
||||
a worker. And the worker it runs in is *started* with thread limits matching
|
||||
what it was given, because that is the only moment a numerical library reads
|
||||
them; a GPU node is told which card is its through `CUDA_VISIBLE_DEVICES`, and
|
||||
@@ -299,8 +298,8 @@ nothing else is given that card while it runs.
|
||||
string whose contents depend on the version you have installed, so writing it
|
||||
for you would silently replace whatever you had put there.
|
||||
|
||||
The engine has to be told how many cards it has — `fluksio serve --gpus 1`, or
|
||||
`FLOW_GPUS` — because detecting them would mean depending on a vendor's
|
||||
The engine has to be told how many cards it has (`fluksio serve --gpus 1`, or
|
||||
`FLOW_GPUS`) because detecting them would mean depending on a vendor's
|
||||
tooling. Until it is, a node asking for one is quietly given zero and runs
|
||||
alongside every other; the log says so the first time it happens.
|
||||
|
||||
@@ -313,9 +312,9 @@ the next GPU run paying for its imports again.
|
||||
imports jax without `resources={"gpus": 1}` runs on the shared pool, is never
|
||||
given `CUDA_VISIBLE_DEVICES`, and leaves a warm worker holding whatever it
|
||||
preallocated. Declaring the card is what makes it a worker the engine knows
|
||||
to retire — and what stops two such nodes running at once in the first place.
|
||||
to retire, and what stops two such nodes running at once in the first place.
|
||||
|
||||
Declaring nothing is the default and is right for most nodes — a poll, a
|
||||
Declaring nothing is the default and is right for most nodes: a poll, a
|
||||
threshold, a message on its way somewhere. Those share the engine's worker pool
|
||||
and are given a fair share of `FLOW_CPUS` as a thread cap, which is what stops
|
||||
several of them at once from each sizing to the whole box.
|
||||
@@ -339,13 +338,13 @@ The flavor is read every time the node is built, so editing it changes what the
|
||||
next run gets. `fluksio flavors` lists them; the Workers screen edits them.
|
||||
|
||||
`ram` takes `"2G"` and `duration_s` takes `"30m"`, and a flavor already says how
|
||||
much — pass one or the numbers, not both. `duration_s` is a statement about the
|
||||
much, so pass one or the numbers, not both. `duration_s` is a statement about the
|
||||
node for whoever is planning around it, not a limit; the limit is `timeout`.
|
||||
|
||||
### Where it runs
|
||||
|
||||
This is one decision, not two. Of every machine attached — this engine and each
|
||||
worker — the node goes to one that can grant what it asked for and has it free.
|
||||
This is one decision, not two. Of every machine attached (this engine and each
|
||||
worker) the node goes to one that can grant what it asked for and has it free.
|
||||
So a node asking for a card finds the box that has one, without naming it:
|
||||
|
||||
```python
|
||||
@@ -354,7 +353,7 @@ So a node asking for a card finds the box that has one, without naming it:
|
||||
```
|
||||
|
||||
If nothing attached can ever grant the ask, it is cut down to what is here and
|
||||
runs anyway — a flow written on a cluster still has to work on a laptop. If
|
||||
runs anyway, since a flow written on a cluster still has to work on a laptop. If
|
||||
something *could* but is busy, the node waits and says so.
|
||||
|
||||
Ask what is free, and what is waiting for it, at
|
||||
@@ -363,8 +362,8 @@ Ask what is free, and what is waiting for it, at
|
||||
!!! note "Accounted, not enforced"
|
||||
|
||||
Nothing stops a node that ignores its declaration; the numbers are
|
||||
bookkeeping plus the environment its libraries read. Real enforcement —
|
||||
cgroups, rlimits — is a next step, not this one.
|
||||
bookkeeping plus the environment its libraries read. Real enforcement
|
||||
(cgroups, rlimits) is a next step, not this one.
|
||||
|
||||
## Say which nodes make a flow
|
||||
|
||||
@@ -390,7 +389,7 @@ train = Flow(
|
||||
`inputs` are the run's parameters, with the value a run gets when it names
|
||||
none; `outputs` are what a run reports as its result.
|
||||
|
||||
Nodes are connected because one **provides** a message another **requires** —
|
||||
Nodes are connected because one **provides** a message another **requires**,
|
||||
never because one imported the other. Importing `fit` into a second flow means
|
||||
"the same code", not "wired to it".
|
||||
|
||||
@@ -439,7 +438,7 @@ def process(dataset, lr, **settings):
|
||||
return (yield from fit(dataset=dataset, lr=lr, **settings))
|
||||
```
|
||||
|
||||
So the flow store still holds a complete, runnable, git-versioned definition —
|
||||
So the flow store still holds a complete, runnable, git-versioned definition;
|
||||
the body simply happens to import rather than duplicate. Your code stays in
|
||||
your repository, under your version control.
|
||||
|
||||
@@ -450,7 +449,7 @@ alongside the store's own. `--dry-run` prints all of this and uploads nothing;
|
||||
!!! warning "Sync after you edit"
|
||||
|
||||
The engine's workers are long-lived and hold your imported modules in
|
||||
memory, so an edit to `myresearch` is invisible until they are retired —
|
||||
memory, so an edit to `myresearch` is invisible until they are retired,
|
||||
which is what every `fluksio sync` does, including one that has nothing to
|
||||
upload. If you are editing many times an hour, attach your own interpreter
|
||||
as a worker instead: it starts a process per call and therefore reads your
|
||||
@@ -467,7 +466,7 @@ alongside the store's own. `--dry-run` prints all of this and uploads nothing;
|
||||
|
||||
The generated body puts your repository on `sys.path` by absolute path,
|
||||
which is right for the machine you synced from and meaningless in a
|
||||
container. For anything else, install the package where the node runs —
|
||||
container. For anything else, install the package where the node runs:
|
||||
`-e /home/you/my-research` in the module manifest, or
|
||||
`myresearch @ git+ssh://…@a1b2c3d`, which travels where a path does not.
|
||||
|
||||
@@ -488,7 +487,7 @@ the GPU box is not switched on. See [Remote workers](../code/workers.md).
|
||||
### What the canvas does with a synced flow
|
||||
|
||||
Its node bodies are generated, so the editor shows them read-only and says
|
||||
where the real code is. Everything else behaves as usual — but ports and
|
||||
where the real code is. Everything else behaves as usual, but ports and
|
||||
settings changed there are overwritten by the next sync, which is the point of
|
||||
the repository being the source of truth. A node whose code you edit on the
|
||||
canvas makes the next sync stop and say so rather than discarding your edit;
|
||||
@@ -501,19 +500,19 @@ fluksio run train --lr 0.003 --seed 7
|
||||
fluksio runs --flow train
|
||||
```
|
||||
|
||||
`run` syncs first, so after an edit the command is just `fluksio run` — there
|
||||
is no step to forget. `--no-sync` skips it for a tight loop where nothing
|
||||
`run` syncs first, so after an edit the command is just `fluksio run`, with no
|
||||
step to forget. `--no-sync` skips it for a tight loop where nothing
|
||||
changed, since syncing retires the workers and the next call pays its imports
|
||||
again.
|
||||
|
||||
`--lr` is typed by the flow's own inputs, so `0.003` arrives as a float. A
|
||||
parameter you did not declare, or one of the wrong type, is refused before
|
||||
anything executes. It answers immediately with a queued run — training is
|
||||
anything executes. It answers immediately with a queued run, since training is
|
||||
measured in hours, so nothing waits for it unless you pass `--wait`, or
|
||||
`--follow`, which waits and prints the numbers as the run reports them.
|
||||
|
||||
No engine has to be up for any of this: `fluksio run --local` boots the
|
||||
engine inside the command instead, on the same `.fluksio` — the same database,
|
||||
engine inside the command instead, on the same `.fluksio`: the same database,
|
||||
artifacts and history a served one would use. It costs a few seconds of
|
||||
startup per invocation against the ~15 ms of submitting to an engine that is
|
||||
already up, so it is for the run you want now rather than the loop you are
|
||||
@@ -521,10 +520,10 @@ iterating in.
|
||||
|
||||
A second run of a flow you did not change mostly does not execute. Each of
|
||||
your nodes is fingerprinted by its settings, the ports it declares, the values
|
||||
it reads, and a digest of your repository's Python files — so editing anything
|
||||
it reads, and a digest of your repository's Python files, so editing anything
|
||||
your node calls into invalidates it, not only the decorated function itself.
|
||||
One an earlier run already answered is restored from that run rather than run
|
||||
again — reported as `cached`, so a flow with nothing left to do finishes as
|
||||
again, reported as `cached`, so a flow with nothing left to do finishes as
|
||||
`(3/3 cached)`. Change `--lr` and only the nodes downstream of it run. The
|
||||
digest is blunt on purpose: an edit anywhere in the repository re-runs
|
||||
everything. `--no-cache` turns caching off for one run, `@node(...,
|
||||
@@ -555,7 +554,7 @@ curl -s "$FLUKSIO/runs/<id>/metrics?name=train.loss" -H "Authorization: Bearer $
|
||||
|
||||
The run carries its parameters, a digest of them, the seed, its result, how
|
||||
long each node took, what it logged, every artifact it produced, and both
|
||||
commits — the flow store's and your repository's. That is the answer to "what
|
||||
commits: the flow store's and your repository's. That is the answer to "what
|
||||
was the learning rate on the run that got 94%?". When a run failed,
|
||||
`run.failures` is the node that did it, with its traceback and its logs.
|
||||
|
||||
@@ -563,9 +562,9 @@ Beside your commit is `code_digest`, and `fluksio runs` prints the pair as
|
||||
`a1b2c3d-dirty+9f0e1a2`. The commit alone cannot identify what ran: your node
|
||||
bodies are imports, so the engine executes whatever is on disk when the worker
|
||||
starts, and an uncommitted tree stamps `-dirty` for every run it ever produces.
|
||||
The digest is over the modules this flow's nodes actually reach — not the
|
||||
The digest is over the modules this flow's nodes actually reach, not the
|
||||
whole repository, so it moves when the code behind a number moves and stays
|
||||
put when a notebook beside it changes — and it is read at the moment the run
|
||||
put when a notebook beside it changes. It is read at the moment the run
|
||||
starts, so in a sweep whose runs queue for hours each one records the code
|
||||
that actually executed it rather than the code that was there when you
|
||||
submitted. It is what makes the column worth joining an
|
||||
@@ -594,7 +593,7 @@ and a blip is seconds:
|
||||
|
||||
A command that still cannot reach the engine says
|
||||
`fluksio: engine not answering (…)` and names the run it started, which is
|
||||
running on the engine regardless — `fluksio runs` finds it again.
|
||||
running on the engine regardless, and `fluksio runs` finds it again.
|
||||
|
||||
## Sweep it
|
||||
|
||||
@@ -618,7 +617,7 @@ curl -X POST $FLUKSIO/runs/flows/train/sweep -H "Authorization: Bearer $TOKEN" \
|
||||
```
|
||||
|
||||
They share a `group_id`, so the sweep is `GET /runs?group=…`. Running them
|
||||
concurrently is safe because each run gets a state backend of its own — two
|
||||
concurrently is safe because each run gets a state backend of its own: two
|
||||
runs of one flow cannot overwrite each other's values.
|
||||
|
||||
Compare the curves in one call:
|
||||
@@ -633,7 +632,7 @@ which answers in exactly the shape a chart widget draws.
|
||||
## Take it into a dataframe
|
||||
|
||||
The analysis itself wants a table, and `export` writes the two an analysis
|
||||
asks for — the curves long, and one row per run with the parameters that
|
||||
asks for: the curves long, and one row per run with the parameters that
|
||||
varied beside its final numbers:
|
||||
|
||||
```sh
|
||||
@@ -643,8 +642,8 @@ fluksio export runs --flow train --status ok -o arms.csv
|
||||
|
||||
Both carry the run id on every row, and `arms.csv` carries the commit and the
|
||||
code digest beside it, so an exported file still says what produced its
|
||||
numbers. Numbers inside a record are columns of their own —
|
||||
`metric.final_metrics.train_loss` — and `--metrics` and `--params` take those
|
||||
numbers. Numbers inside a record are columns of their own
|
||||
(`metric.final_metrics.train_loss`) and `--metrics` and `--params` take those
|
||||
dotted paths to narrow the table. Metric names are flow-qualified, so
|
||||
`--list` prints the ones a selection carries. `Client.export_metrics()` and
|
||||
`Client.export_runs()` answer the same rows to a notebook, ready for
|
||||
@@ -653,8 +652,8 @@ dotted paths to narrow the table. Metric names are flow-qualified, so
|
||||
## Looking at them in the portal
|
||||
|
||||
None of the above needs a browser, but there is one. **Runs** is the table of
|
||||
everything that has run: filter it to a flow, to a status, or to one sweep —
|
||||
where it grows a column per parameter that actually varied — and tick two runs
|
||||
everything that has run: filter it to a flow, to a status, or to one sweep,
|
||||
where it grows a column per parameter that actually varied, and tick two runs
|
||||
to put their curves side by side.
|
||||
|
||||
A run records its metrics under the same names a dashboard binds to, so a
|
||||
@@ -721,13 +720,13 @@ api.post("/flows/train/publish", json={"version": version}).raise_for_status()
|
||||
```
|
||||
|
||||
Two things worth noticing, because the decorators only move where they are
|
||||
said. Ports are declared, not inferred — `process(lr, epochs)` gets its
|
||||
said. Ports are declared, not inferred: `process(lr, epochs)` gets its
|
||||
arguments from the ports of the same name, and the types are checked on every
|
||||
value. And saving writes a *draft*; `publish` is what the engine picks up.
|
||||
That separation is what lets you edit a flow that is running.
|
||||
|
||||
A flow uploaded this way carries no `origin`, which is what tells the canvas —
|
||||
and the next `fluksio sync` — that it was not generated. See
|
||||
A flow uploaded this way carries no `origin`, which is what tells the canvas,
|
||||
and the next `fluksio sync`, that it was not generated. See
|
||||
[The API](../code/api.md).
|
||||
|
||||
## Small scripts you are just playing with
|
||||
@@ -741,7 +740,7 @@ cd .fluksio/flows
|
||||
git log --oneline
|
||||
```
|
||||
|
||||
Every save is a commit — the node source, the parameters, the graph. A run
|
||||
Every save is a commit: the node source, the parameters, the graph. A run
|
||||
records the commit it ran at, so `git show` on that hash is literally the code
|
||||
that produced the number. You get the versioning without adopting anything.
|
||||
|
||||
@@ -759,8 +758,8 @@ A submit is around 15 ms, so calling that in a loop is a reasonable thing to do.
|
||||
|
||||
## Get a browser onto it
|
||||
|
||||
The pip install gives you the engine and the API, not a web interface — a
|
||||
machine with no inbound route cannot serve one usefully anyway. To see the
|
||||
The pip install gives you the engine and the API, not a web interface, since a
|
||||
machine with no inbound route cannot serve one usefully. To see the
|
||||
canvas, the run history and live loss curves, pair the instance with a
|
||||
portal, which serves the dashboard from its side:
|
||||
|
||||
@@ -773,7 +772,7 @@ Nothing needs to be exposed: your machine dials out and holds the connection
|
||||
open. See [Accounts and the portal](../interface/portal.md).
|
||||
|
||||
If you would rather stay entirely offline, run the dashboard SPA yourself from
|
||||
the app's Docker image — see the [facility path](facility-automation.md), which
|
||||
the app's Docker image; see the [facility path](facility-automation.md), which
|
||||
is the same stack.
|
||||
|
||||
## Where to go next
|
||||
|
||||
@@ -20,7 +20,7 @@ in this interface a lot, so it is worth learning it properly.
|
||||
└── workers on other boxes (optional)
|
||||
```
|
||||
|
||||
One engine holds every flow. Nothing here is a plugin you install separately —
|
||||
One engine holds every flow. Nothing here is a plugin you install separately:
|
||||
the broker client, the time-series writer, the dashboards and the alerting are
|
||||
all part of the same process, editing the same graph.
|
||||
|
||||
@@ -28,7 +28,7 @@ all part of the same process, editing the same graph.
|
||||
|
||||
- Docker and Compose v2 on the host
|
||||
- The address you will type into a browser. On a home network that is the
|
||||
server's own address — `192.168.1.50` — and nothing else is needed. A
|
||||
server's own address, `192.168.1.50`, and nothing else is needed. A
|
||||
hostname is optional, and covered below
|
||||
- Optionally: an MQTT broker and an InfluxDB you already run. If not, the stack
|
||||
can start both for you
|
||||
@@ -53,15 +53,15 @@ $EDITOR .env # FIRST_SUPERUSER, ENVIRONMENT=production
|
||||
| `FIRST_SUPERUSER_PASSWORD` | leave it as `changethis` and one is generated for you |
|
||||
| `ENVIRONMENT` | `production` closes the interactive API schema; `local` leaves it open |
|
||||
|
||||
Everything the instance owns — the database, your flows, secrets,
|
||||
artifacts, the packages your node code imports — is on one Docker volume.
|
||||
Everything the instance owns (the database, your flows, secrets, artifacts,
|
||||
the packages your node code imports) is on one Docker volume.
|
||||
Backing that volume up is backing up the instance.
|
||||
|
||||
??? note "Even smaller: no Docker at all"
|
||||
|
||||
`pip install fluksio && fluksio serve` gives you the same engine with no
|
||||
containers, keeping its data in `~/.fluksio`. What it does not give you is
|
||||
the web interface, which the SPA container serves — so you would drive it
|
||||
the web interface, which the SPA container serves, so you would drive it
|
||||
from the API, or [pair it with a portal](../interface/portal.md) that
|
||||
serves the dashboard for you. Good for a Raspberry Pi that only runs flows;
|
||||
less good as your main instance.
|
||||
@@ -82,7 +82,7 @@ APP_PORT=8080 docker compose -p fluksio-app --env-file .env \
|
||||
-f docker/compose.yml -f docker/compose.lan.yml up -d --build
|
||||
```
|
||||
|
||||
Open `http://192.168.1.50:8080`. That one address is the whole application —
|
||||
Open `http://192.168.1.50:8080`. That one address is the whole application:
|
||||
the interface, the API under `/api`, and the websocket that puts live values on
|
||||
the canvas.
|
||||
|
||||
@@ -96,13 +96,13 @@ maintaining: the browser is talking to one origin.
|
||||
|
||||
A published port is plain HTTP with no certificate. Between your own
|
||||
machines that is fine, and it is not something to forward from a router. To
|
||||
reach the house from outside, put a reverse proxy with TLS in front of it —
|
||||
reach the house from outside, put a reverse proxy with TLS in front of it,
|
||||
or [pair it with a portal](../interface/portal.md), which needs no inbound
|
||||
route at all.
|
||||
|
||||
### With a hostname, behind a reverse proxy
|
||||
|
||||
If you have DNS — or want certificates — set `DOMAIN` and run `make up`
|
||||
If you have DNS, or want certificates, set `DOMAIN` and run `make up`
|
||||
instead. The interface is then served on `app.${DOMAIN}` and the API on
|
||||
`api.${DOMAIN}`, and nothing is published on a host port.
|
||||
|
||||
@@ -113,7 +113,7 @@ attach it to the `proxy` network and forward `app.${DOMAIN}` →
|
||||
|
||||
!!! tip "Developing on the same machine"
|
||||
|
||||
`make dev-lan` is the same idea for the integrated dev stack — the app on
|
||||
`make dev-lan` is the same idea for the integrated dev stack: the app on
|
||||
`http://<host-ip>:8080`, `APP_PORT` to move it. `make dev` rebuilds without
|
||||
the published port, so re-run `make dev-lan` after one.
|
||||
|
||||
@@ -128,9 +128,8 @@ Go to **Flows → New flow** and call it `house`. You land on the canvas.
|
||||
|
||||
A flow is a set of nodes that talk to each other through **named messages**.
|
||||
You do not draw wires: a node says which messages it needs and which it
|
||||
produces, and the canvas draws the graph that follows from those names. That
|
||||
sounds like a small difference and turns out to be a large one — renaming is
|
||||
safe, fan-in is free, and two flows can share a value by naming it.
|
||||
produces, and the canvas draws the graph that follows. Renaming is safe, fan-in
|
||||
is free, and two flows share a value by naming it.
|
||||
|
||||
### Read a sensor
|
||||
|
||||
@@ -150,8 +149,8 @@ broker sends one. Click the wire to see the last payload and its history.
|
||||
|
||||
### Do something with it
|
||||
|
||||
Add a **Function** node. This is a Python node — the code editor opens in its
|
||||
panel:
|
||||
Add a **Function** node. This is a Python node, and the code editor opens in
|
||||
its panel:
|
||||
|
||||
```python
|
||||
def process(living_temperature, comfortable=21.0):
|
||||
@@ -160,7 +159,7 @@ def process(living_temperature, comfortable=21.0):
|
||||
```
|
||||
|
||||
Declare `living_temperature` as an input (type `float`) and `heat_wanted` as an
|
||||
output (type `bool`). `comfortable` is not a port — it is a **setting**,
|
||||
output (type `bool`). `comfortable` is not a port but a **setting**,
|
||||
because it is a constant of this node rather than something the graph carries.
|
||||
It shows up as a field in the node's Settings section.
|
||||
|
||||
@@ -178,8 +177,8 @@ Publish the flow. You have a thermostat.
|
||||
!!! warning "Test before it touches a relay"
|
||||
|
||||
A flow can be **paused** (it holds messages instead of running them) and
|
||||
**stepped** (release exactly one). Together with the run button — which
|
||||
injects a value by hand — that is how you convince yourself the logic is
|
||||
**stepped** (release exactly one). Together with the run button, which
|
||||
injects a value by hand, that is how you convince yourself the logic is
|
||||
right before the contactor finds out. Both live on the dock at the bottom
|
||||
of the canvas.
|
||||
|
||||
@@ -212,11 +211,10 @@ point.
|
||||
Note the token. Credentials never sit in a flow: `{"$secret": "influx-token"}`
|
||||
is a reference into an encrypted store, and the editor renders those fields as
|
||||
a secret picker. Add the actual value once under **Secrets**. Flows are a git
|
||||
repository you might well push somewhere — this is what keeps a password out of
|
||||
it.
|
||||
repository you might well push somewhere, so this keeps a password out of it.
|
||||
|
||||
Reading back is the same node with `queries` instead of `writes`, or — for
|
||||
anything a chart asks for — a pair of small Python nodes on either side that
|
||||
Reading back is the same node with `queries` instead of `writes`, or, for
|
||||
anything a chart asks for, a pair of small Python nodes on either side that
|
||||
build a Flux query and shape its rows. That indirection is deliberate: the
|
||||
database node holds the connection and nothing else, so a dashboard widget
|
||||
never learns which database answered it.
|
||||
@@ -231,7 +229,7 @@ Widgets are typed the same way ports are: a switch binds to a `bool`, a gauge
|
||||
to a number, an agenda to a `list`. Bind it wrong and the editor says so rather
|
||||
than drawing nothing.
|
||||
|
||||
Controls work in the other direction — a switch on a dashboard publishes the
|
||||
Controls work in the other direction: a switch on a dashboard publishes the
|
||||
message it is bound to, exactly as a node would. The canvas draws it as a
|
||||
labelled endpoint feeding the nodes that read it, so a value never appears from
|
||||
nowhere.
|
||||
@@ -247,7 +245,7 @@ lives on a different network, or something needs hardware the engine's host
|
||||
does not have.
|
||||
|
||||
The unit of distribution is the **worker**. It runs the code of nodes you mark
|
||||
for it, and it dials *out* to the engine — so the Pi in the shed does not need
|
||||
for it, and it dials *out* to the engine, so the Pi in the shed does not need
|
||||
an inbound route, and the engine does not need to reach it.
|
||||
|
||||
On the engine, mint a token:
|
||||
@@ -276,7 +274,7 @@ flow before the hardware arrives.
|
||||
!!! note "What a worker is not"
|
||||
|
||||
It is not a second engine. Subscriptions, schedules, webhooks and the
|
||||
dashboards all stay in one process — that is what keeps a value having one
|
||||
dashboards all stay in one process, which keeps a value having one
|
||||
definition. A worker executes node bodies, nothing else. Scaling *the
|
||||
engine* to several processes is not supported: run one.
|
||||
|
||||
@@ -291,9 +289,9 @@ What you probably want on day one is everything, to ntfy:
|
||||
- add an ntfy channel with your server and topic
|
||||
- add a rule with no events ticked, which means *all of them*
|
||||
|
||||
The engine deduplicates aggressively — the same node failing every second is
|
||||
one alert, not thirty-six thousand — and caps the total at ten an hour however
|
||||
bad it gets. See [Secrets, modules and alerts](../interface/operations.md).
|
||||
The engine deduplicates aggressively, so the same node failing every second is
|
||||
one alert rather than thirty-six thousand, and caps the total at ten an hour
|
||||
however bad it gets. See [Secrets, modules and alerts](../interface/operations.md).
|
||||
|
||||
## Keep it alive
|
||||
|
||||
|
||||
@@ -1,8 +1,6 @@
|
||||
# Pick your starting point
|
||||
|
||||
People arrive at Fluksio from two directions, and the honest answer to "how do
|
||||
I set this up?" is different for each — not just in the commands, but in how
|
||||
much of an afternoon it is reasonable to spend.
|
||||
Setup differs depending on what you are here for. Pick the path that matches.
|
||||
|
||||
Pick the one that sounds like you. Everything past this section is the same for
|
||||
both.
|
||||
@@ -52,8 +50,8 @@ Some rough tells:
|
||||
| **Data lives in** | SQLite beside the flows | InfluxDB, usually |
|
||||
| **The thing you look at** | run history and loss curves | a dashboard, maybe on a wall |
|
||||
|
||||
If both describe you — a lab with instruments to drive *and* models to
|
||||
fit — start with the data-science path. It is the smaller instance, and it
|
||||
If both describe you (a lab with instruments to drive *and* models to fit),
|
||||
start with the data-science path. It is the smaller instance, and it
|
||||
grows into the other one without being reinstalled: the same engine, the same
|
||||
flows, just more of them running all the time.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user