app/docs/code/cli.md

# The `fluksio` command

```sh
pip install fluksio
```

Installs the engine and the `fluksio` command. Python 3.12 or newer, Linux or
macOS.

The MQTT and InfluxDB connectors, outbound mail and error reporting are
`pip install 'fluksio[server]'`: a deployment talking to devices wants them,
and a laptop waiting on them to install does not. Everything else, the engine
and every python node included, is in the plain install; a node type whose
library is missing says which extra to add when one is actually built.

There is a second, smaller distribution, `fluksio-worker`, for a machine that
should only *run nodes* for an engine elsewhere. It has none of the engine in
it. See [Remote workers](workers.md).

The command is two things at once: `serve`, `enroll` and `worker` *are* an
instance, while `login`, `sync`, `run`, `runs`, `artifacts`, `sweep` and
`status` talk to one that may be anywhere.

## Where an instance lives

`.fluksio` beside your code, found the way `.git` is: from the working
directory, or any directory above it. Two repositories on one machine are
therefore two engines, with their own flows, runs and token. `fluksio serve`
makes one where there is none, and it ignores itself from within with a
`.gitignore` of `*`, so a database and a credential cannot be committed by
accident.

`--global` uses `~/.fluksio` instead, shared by every directory. `--data-dir`
(or `FLUKSIO_HOME`) names any directory outright and wins over both.

## `fluksio serve`

Runs the engine.

```sh
fluksio serve
```

On the first start it creates an admin account and prints its password **once**.
Nothing else has to be running: no database server, no message broker, no
Docker.

At a terminal this opens a dashboard with the engine running under it; see
[below](#the-dashboard). `--plain` prints the log stream instead, which is
also what happens with no terminal: in a container, under systemd, or in CI.

The default port moves out of the way when something already has it (8001,
8002, and so on) and says which one it took; the URL written to
`client.json` is the one it is actually on. A port you *asked* for is never
moved off: `--port 9000` on a taken 9000 fails, because something else is
there and you named it.

What it will *not* do is start a second engine for the same instance. If
the port is held by an engine already serving this directory, it says so and
stops, since one SQLite database wants one engine. Another instance's Fluksio
on that port is named, and the move happens as usual.

| Option | Default | What it does |
|---|---|---|
| `--data-dir PATH` | `./.fluksio` (or `$FLUKSIO_HOME`) | where this instance keeps everything |
| `--host HOST` | `127.0.0.1` | what to bind |
| `--port PORT` | `8000`, or the next free one | what to listen on |
| `--plain` | off at a terminal | the log stream rather than the dashboard |
| `--log-level LEVEL` | `info` | uvicorn's log level |
| `--admin-email ADDR` | `admin@example.com` | the account created on first run |
| `--admin-password PW` | generated | set it instead of having one generated |
| `--enroll CODE` | — | pair with a portal as part of coming up |
| `--portal URL` | — | the portal `--enroll` redeems at |
| `--max-runs N` | 4 | batch runs driven at once (`FLOW_MAX_RUNS`) |
| `--max-cascades N` | 4 | cascades in flight at once (`FLOW_MAX_CASCADES`) |
| `--max-workers N` | 4 | python worker processes (`FLOW_MAX_WORKERS`) |
| `--gpus N` | 0 | GPUs on this machine a node may be given (`FLOW_GPUS`) |

Cards are declared rather than detected, since asking a vendor's tooling would
make one dependency two, so a machine with a GPU reports none until `--gpus` says
otherwise, and a node asking for one is clamped to zero and runs alongside
every other. `--gpus 1` is what serialises them.

`--enroll` with `--portal` is the one-command setup: it pairs before the engine
starts, so the connection is dialled as part of coming up rather than needing a
restart. It is skipped if the instance is already enrolled.

!!! warning "One process"

    `fluksio serve` holds the flow engine. A second one is a *second engine*:
    duplicated subscriptions, duplicated cron ticks, two webhooks answering the
    same path. Run one, and distribute work with
    [workers](workers.md) instead.

!!! note "`$HOME` on a cluster"

    A login node's home directory is often NFS, where SQLite's write-ahead log
    does not work, since the database would be locked or corrupt. `fluksio serve`
    warns when it notices; point `--data-dir` at local disk.

### What it prints

```text
Created the admin account admin@example.com
  password: k3Qm-8vTpLdX
  Shown once. Change it from the dashboard.
Fluksio 0.1.0 — data in /home/you/.fluksio
  API      http://127.0.0.1:8000/api/v1
  No portal. Pair this instance with:
    fluksio enroll <code>
```

An enrolled instance says which portal it is on instead, and notes that the
dashboard is served from there rather than here.

### The dashboard

At a terminal, `serve` draws the health overview, the recent runs, and the
engine's own log in a pane below, so the output above is in there rather than
replaced by it.

| Key | What it does |
|---|---|
| `q` | close the dashboard. **The engine keeps running**, and the pid is printed |
| `s` | stop the engine, or start it again |
| `r` | restart it |
| `c` | cancel the run the cursor is on |
| `space` | tick the run under the cursor into a comparison |
| `enter` | compare the ticked runs, or draw the one under the cursor |
| `a` | list what the run left behind, and fetch it |
| `e` | pair with a portal, without leaving the screen |

The engine is a child process rather than a thread, which is what makes those
possible, and what makes `q` a way out of the screen rather than a way to
stop the engine. Running `fluksio serve` again reattaches to it.

An engine started elsewhere is adopted rather than duplicated, and can be
stopped from here only when it is this instance's own: both the pidfile
beside the data and a token this directory's key signed have to agree. Another
instance's engine is named and left alone.

The screen subscribes to the engine's event bus over the same websocket a
browser uses, so a run appears the moment it starts. It reconnects on its own,
and an engine stopped from this screen is a normal state rather than an error.

#### Comparing runs

`enter` opens a comparison of the ticked runs: one metric's curve for each,
over a table of what differs between them.

The curves are drawn in braille, five to a chart, in five distinct colours: a
departure from the browser's chart palette, where the five series
step one hue by lightness. Hue is what survives a terminal. Every curve is
named in the legend either way.

Two pickers sit above the chart. The first is the metric, named as the run
records it (`train.loss` rather than `loss`, since a name is qualified by the
node that published it). The second is what it is plotted against: the step,
`time (s)` measured from each run's own first reading so runs started hours
apart lie on top of each other, or another metric of the same runs.

The table below keeps only what actually differs (the inputs whose values are
not the same in every run, and the seed when it varies) beside each run's
status, duration and final numbers. A parameter every run shared is not a
column worth scanning.

A run that is still going has its curve re-read once a second, and the whole
comparison refreshes when the engine says a run finished.

## `fluksio enroll`

Pairs an existing instance with a portal.

```sh
fluksio enroll ABCD-1234
```

| Option | What it does |
|---|---|
| `--portal URL` | a portal of your own, instead of `https://hub.fluksio.com` |
| `--as EMAIL` | the local account a portal session arrives as |
| `--data-dir PATH` | which instance, if not the one this directory is in |

Get the code from the portal under **Instances → Add instance**. It is
single-use and expires in fifteen minutes. `--as` matters when the instance
has several superusers. Without it, enrolment refuses rather than guessing.

Afterwards, `fluksio serve` dials the portal as it comes up, and keeps dialling:
a portal that restarts, a wifi that changes, a laptop that suspends and wakes
somewhere else all end the same connection, and the link is put back up without
anybody noticing. A connection that stood up and then dropped is retried at
once; one that never stood up waits a little longer each time, up to half a
minute. See [Accounts and the portal](../interface/portal.md).

## `fluksio worker`

Runs nodes for an engine elsewhere. Everything after `worker` belongs to the
agent's own parser: it is the same program `fluksio-worker` installs, so the
two are interchangeable:

```sh
fluksio worker --url wss://api.fluksio.com/api/v1/workers/attach \
               --token "$FLUKSIO_WORKER_TOKEN" --labels gpu
```

See [Remote workers](workers.md).

## Talking to an engine

The commands below are the client half: they run wherever you work, and
address an engine over its API rather than being one, except under `--local`,
which boots one inside the command instead.

### `fluksio login`

```sh
fluksio login --url https://api.fluksio.com
```

For an engine somewhere *else*. One you started yourself needs no login:
`fluksio serve` writes the token as it comes up and says where it put it.

The token goes in this project's `.fluksio/client.json`, or with `--global` in
`~/.fluksio/client.json`. Every command below reads it from there, nearest
first walking up from the working directory, or from `FLUKSIO_URL` and
`FLUKSIO_TOKEN`, or from its own `--url` and `--token`. A token an older
version wrote to `~/.config/fluksio/client.json` is still read.

### `fluksio sync`

```sh
fluksio sync [PATH_OR_MODULE ...]        # default: the current directory
```

Imports what you name, collects the flows the decorators declared, and uploads
each one with a generated import shim per node. A directory that is a package
is walked; a dotted name is imported as it stands; nothing is loaded from a
file path, because the shim has to import the same way.

A plain directory is walked all the way down, so one folder per study
(`fluksio sync dev` over `dev/s1_baseline/study.py`) needs no naming. Hidden
directories, `__pycache__`, `node_modules` and virtualenvs are left alone.

Each file is imported under the name its path spells beneath the directory
being synced, so `dev/s1/study.py` and `dev/s2/study.py` are `s1.study` and
`s2.study` and a `study.py` per study collides with nothing. No `__init__.py`
is needed, since the directories in between are namespace packages, which leaves a
bare `from study import ...` in a test beside it working. A file at the top of
what is synced keeps its plain name.

| Flag | What it does |
|---|---|
| `--dry-run` | print the flow documents and shims, upload nothing |
| `--no-publish` | leave the upload as a draft |
| `--force` | overwrite a flow, or a node body, that was edited on the canvas |

Every sync retires the engine's workers, including one that had nothing to
upload: a worker holds your package in memory, so an edit to it is invisible
until the process goes.

It also records, per node, which of your modules that node's function imports
its way to, and what they hash to. That is what the
[stage cache](../concepts/runs.md#stage-caching) keys on, so editing a
helper a node calls into is reported as that node changing (`train: updated
(flow, fit)`) and re-runs it, while editing something the node never reaches
is left alone.

A sync that changed nothing says `unchanged`, which is the answer worth
having. `published` and `draft` are said only when there was something to
publish or a draft was genuinely left behind. An engine too old to store what
a node's code reaches says so in a line naming both versions; until it is
upgraded its cache is keyed on the whole repository, as it was before. See
[Getting started: data science](../getting-started/data-science.md).

### `fluksio run`

```sh
fluksio run train --lr 0.05 --seed 7 [--wait]
```

Syncs the working directory *and everything under it*, then submits a run, so
the command after an edit is this one and nothing else, from the repository
root as readily as from the study's own folder. A study that will not import
is a warning rather than a stopped run; the upload is already a no-op for a
flow nothing changed in, so what the walk costs is importing the others.
`--sync dev/s1_baseline` (repeatable) narrows it to what you name when that
is not free, and `--no-sync` skips it entirely.

Flags that are not its own are the flow's inputs, typed by what the flow
declares them as, so a name the flow has not got is refused by name, and
`--param lr=0.002` is told that one value is `--lr 0.002` and several is a
[sweep](#fluksio-sweep). `--wait` blocks until the run
finishes and exits non-zero if it failed. `--follow` waits as well, and prints
the numbers the run reports as they arrive:

```text
  train.loss[14] = 3.40295e-06
```

Ctrl-C while either is waiting cancels the run on the engine rather than only
stopping the watching, and exits 130. `--timeout SECONDS` gives up *waiting*
after that long and leaves the run going; it is nothing to do with a node's own
timeout.

`--seed N` is the experiment's seed, and it does three things. It is recorded
on the run, so what a result came from is answerable later. It goes into the
digest that identifies a run's inputs, so two runs of one configuration with
different seeds are different runs rather than a cache hit. And if the flow
declares an input named `seed`, that is what fills it, so the number the run
is labelled with is the number your code actually drew from, instead of merely
looking like it. A flow that declares no such input still records it, and
nothing reads it. Sweep over seeds with `--param seed=1,2,3`.

Any input takes what a previous run produced, named rather than typed out: a
checkpoint, but equally a config object nobody wants to paste into a shell:

```sh
fluksio run evaluate --dataset @run:1758042000123-9f2ab41c.dataset
fluksio run evaluate --dataset sha256:6dd1f0…
fluksio run train --meta @run:1758042000123-9f2ab41c.dataset_meta
```

`@run:<id>.<output>` is whatever that run's output was, whole and with its own
type; a bare digest is the content itself, resolved into a reference. Both
spellings are reserved on every input, `str` included, so an input that has to
carry one of them literally cannot. Passing the value as JSON still works and
is what a script that already holds one does, the same thing
`flow.submit(dataset=run.result["dataset"])` does from Python.

Run a flow with no parameters at a terminal and it asks for them, one line per
declared input, with the declared value in brackets:

```text
lr (float) [0.05]: 0.01
epochs (int) [10]:
dataset (artifact): @run:1758042000123-9f2ab41c.dataset
```

Enter keeps what is in brackets, so pressing it through the lot runs the
defaults. Nothing changes for a scripted run: passing any parameter, or piping
the command, skips the questions, and `--defaults` skips them explicitly.

`--no-sync` runs what is already on the engine. Worth it in a tight loop where
you know nothing changed, since syncing retires the workers and the next call
pays its imports again. A directory that declares no flows syncs nothing and
says nothing: a flow drawn on the canvas is run the same way.

`--no-cache` executes every node, including one an earlier run already
answered. See [Stage caching](../concepts/runs.md#stage-caching).

`--local` boots the engine inside this process instead of talking to a served
one, so there is no `fluksio serve` terminal to keep open. It is the same
instance either way (the same `.fluksio`, the same database, artifacts and run
history) so a run made this way and a run made through a served
engine cache against each other. It always waits, because the engine it starts
lives exactly as long as the command. Starting one costs a few seconds of
worker pool and module reconcile, against the ~15 ms of submitting to an
engine that is already up: `--local` is for "I just want to run it", not for a
loop you are iterating in.

### `fluksio status`

```sh
fluksio status [--watch]
```

The home screen's top half in a terminal: whether the engine is healthy and
what is wrong if not, whether it is paired with a portal and reaching it, then
every flow with its state, its node count and whether it has unpublished
changes, and the last few runs and failures under them.

A `resources` line names each machine the engine can run a node on and how much
of it is in use, plus how many nodes are queued for one. It is absent on an
engine that accounts for nothing.

The portal reads one of three ways. `no portal` means this instance was
never enrolled. `portal hub.fluksio.com` means the link is up. `portal
unreachable` names the error, and is the one worth acting on: the dashboard is
served from the other end, so nobody can reach it while that is showing.

`--watch` keeps it on screen and refreshes every five seconds until Ctrl-C, the
cadence the dashboard polls at, since nothing here moves faster. It needs a
terminal; without one, run it without `--watch` and the output pipes cleanly.

`--local` reads the flows and history out of this directory with no engine
served, the same way `runs --local` does. It cannot be watched: an in-process
engine is the command itself, so nothing changes under it.

### `fluksio runs`

```sh
fluksio runs [--flow train] [--limit 20]
```

The runs an engine has recorded, newest first: id, status, flow, duration, how
long ago it was submitted, the commit of the repository it came from, and the
inputs it was given. Statuses are coloured when a terminal is reading the
output: `ok` green, `error` red, `cached` cyan.

Only the inputs that *differ from what the flow declares* are shown, and they
are clamped to what is left of the terminal's width. A run that took the
defaults lists none at all, and a flow taking a few kB of JSON does not push
everything else off the line. `Client.runs()` and
[`fluksio export runs`](#fluksio-export) are where the whole value is read.
`--local` reads the same history from an in-process engine, without one having
to be served.

### `fluksio flavors`

```sh
fluksio flavors
```

The named sizes a node can ask for, as in `@node(resources="gpu-small")`, with the
cores, memory and cards each stands for. Editing them is the Workers screen or
`POST /api/v1/flavors`; this is the read.

### `fluksio sweep`

```sh
fluksio sweep train --param lr=0.1,0.01 --param epochs=10,50 --wait
```

Every combination of the parameter lists, submitted as one group: four runs
above, sharing a `group_id` and executing in parallel. Values are typed by the
flow's inputs, the same as `run`'s are, and `--seed`, `--no-sync`,
`--no-cache` and `--local` mean what they do there. `--wait` blocks until all
of them are finished and exits non-zero if any failed.

### `fluksio export`

```sh
fluksio export metrics --flow train --list
fluksio export metrics --flow train --name train.train_loss --stride 10 -o curves.csv
fluksio export runs    --flow train --status ok > arms.csv
```

The two tables an analysis reads. `export metrics` is the long one, a row per
run, metric and step, which is what a plotting library takes without
reshaping; `--name` keeps the metrics it lists and `--stride` keeps every Nth
point of *each* curve. `export runs` is the wide one: a row per run with its
inputs as columns, its final numbers, its status, its duration and the commit
and digest of the code it ran.

Every input the selected runs recorded becomes a column, so the schema does
not move with the selection and a filter written against one export keeps
working on the next; `--params lr,seed` narrows it to the axis a comparison is
read along. `--metrics` narrows the final numbers the same way.

An input left out of a submit is recorded at the value the flow declares for
it, so every row says what it was actually run with rather than leaving the
cell blank.

A node usually returns a record rather than a scalar, so both sides take
dotted paths into one: `--metrics final_metrics.train_loss,test_metrics.known.perfect`
selects three fields rather than two blobs, and `--params model.ansatz` does
the same for an input. The defaults reach the same depth: every number a
result carries becomes a column wherever it sits, and a record's inputs are
taken leaf by leaf rather than as one blob.

Metric names are flow-qualified (a node of `train` writing `train_loss` records
`train.train_loss`) so `--list` prints the names the selected runs
carry when the spelling is not obvious.

Both take `--flow`, `--run ID` (repeat it), `--group`, `--status`, `--since`,
`--until` and `--local`, and both put the run id on every row: it is the join
back to the run page and to what the run made.

`--format` is `csv` (the default), `jsonl` or `parquet`; output goes to stdout
unless `-o FILE` names somewhere. Parquet keeps the types, needs pyarrow
(`pip install 'fluksio[parquet]'`) and needs a file to write, since it is not a
stream. In a notebook, `Client.export_metrics()` and `Client.export_runs()`
answer the same rows as a list of dicts, which `pandas.DataFrame` takes
directly.

### `fluksio artifacts`

```sh
fluksio artifacts 1758042000123-9f2ab41c
fluksio artifacts 1758042000123-9f2ab41c weights -o model.npz
```

The files a run produced: what `fluksio.save_artifact(...)` wrote, and any
artifact a node returned. Named alone it lists them: the message each left on,
its size, its media type and the filename the node gave it. Name one and it is
written here, under that filename unless `-o` says otherwise.

The message name is the one to pass, since it is what addresses the bytes;
`--local` reads them from this directory without an engine served.

## What lives in the data directory

```text
.fluksio/               (or ~/.fluksio, with `--global`)
├── client.json         the token `serve` wrote, mode 600
├── .gitignore          `*`, so a database and a credential are ignored from within
├── fluksio.db          SQLite: users, runs, metrics, observability, agents
├── flows/              a git repository, one directory per flow
│   ├── house/
│   │   ├── flow.json           the published structure
│   │   ├── nodes/*.py          the published node code
│   │   ├── flow.draft.json     unpublished edits, if any
│   │   └── nodes.draft/*.py
│   ├── _lib/           shared node sources
│   ├── _dashboards/    dashboards, drafts and all
│   └── requirements.txt        what the Modules screen installs
├── artifacts/          content-addressed bytes, two levels deep
├── user-venv/          the interpreter your node code runs on
├── secrets.enc         encrypted credentials, deliberately outside flows/
├── alerts.json         alert channels and rules
├── panels.json         wall-panel pairings
├── oauth-key.pem       signs agent tokens
├── cloud.json          the portal enrolment, if there is one
├── secret_key          signs sessions and derives the secrets key
└── env                 optional settings file
```

Two things follow from this layout and are worth internalising:

**`flows/` is a real git repository.** `git log` is the history of every change
anyone made to any flow. A run records the commit it ran at, so `git show` on
that hash is literally the code that produced the number.

**Backing up the data directory backs up the instance.** Everything else is
rebuildable. Copy it while the engine is stopped, or use SQLite's online backup
for the database if it is not.

## Settings

Settings come from the environment, or from an `env` file in the data
directory. The ones you are most likely to touch:

| Variable | Default | What it does |
|---|---|---|
| `DATA_DIR` | `./.fluksio` via the CLI; `~/.fluksio` with `--global` | everything below it derives from this |
| `DATABASE_URL` | SQLite in the data dir | any SQLAlchemy URL |
| `NODE_VENV` | `auto` | which interpreter node code runs on: `auto` adopts the venv Fluksio was installed into, `managed` builds one of its own, or name an interpreter |
| `REDIS_HOST` | unset | flow state in Redis instead of memory; survives a restart |
| `FRONTEND_HOST` | — | the address used in mails, OAuth metadata and panel links |
| `ENVIRONMENT` | `local` | `production` closes the interactive API schema |
| `MCP_ENABLED` | `false` | opens the [agent endpoint](agents.md) |
| `SECRET_KEY` | generated once | signs sessions, derives the secrets key |

The full list is in [Configuration](../reference/configuration.md).