Files
app/docs/code/cli.md
T
stroblme f2a823218a GPUs follow the engine's own CUDA_VISIBLE_DEVICES
Its entries are the cards present, and a local child's allocation is
named through them, so under =2,3 card 0 is device 2. Remote workers
keep mapping against their own mask.
2026-10-15 09:53:38 +02:00

362 lines
13 KiB
Markdown

# The `fluksio` command
Install on Linux or macOS with Python 3.12 or newer:
```sh
pip install fluksio
fluksio serve
```
For the browser dashboard, sign up at [fluksio.com](https://fluksio.com), add
an instance, then run `fluksio enroll <code>` in the same directory.
Use `fluksio --help` or `fluksio COMMAND --help` for command options.
`pip install 'fluksio[server]'` adds MQTT, InfluxDB, outbound mail and error
reporting. A machine that only executes remote nodes can install
[`fluksio-worker`](workers.md).
## Where an instance lives
By default, Fluksio finds the nearest `.fluksio/` directory by walking up from
your working directory. `serve` creates one in the current directory if needed.
Separate project directories can therefore hold separate instances.
`--global` selects `~/.fluksio`. `--data-dir PATH` or `FLUKSIO_HOME`
selects a directory explicitly. The data directory contains a `.gitignore`
that excludes its database, credentials and other contents from your project.
## `fluksio serve`
Starts the API and flow engine. No external database, broker or Docker is
needed for the default setup.
```sh
fluksio serve
fluksio serve --plain --port 9000
```
The first start prints an admin password once. Local CLI commands receive a
saved token automatically; `login` is for connecting to another engine.
The default port is 8000, advancing to the next free port if necessary.
An explicit `--port` fails when occupied. The startup banner and saved client
configuration show the selected URL.
Run one engine per instance. At a terminal, a second `serve` reattaches to
that instance's running engine; plain mode refuses to start a duplicate.
| Option | Default | Purpose |
|---|---|---|
| `--data-dir PATH` | Nearest `.fluksio`, or a new one here | Instance storage |
| `--global` | Off | Use `~/.fluksio` |
| `--host HOST` | `127.0.0.1` | Bind address |
| `--port PORT` | `8000`, or next free | Listening port |
| `--plain` | Off at a terminal | Print logs instead of opening the terminal dashboard |
| `--log-level LEVEL` | `info` | Uvicorn log level |
| `--admin-email ADDR` | `admin@example.com` | First admin account |
| `--admin-password PW` | Generated | First admin password |
| `--enroll CODE` | Unset | Enroll before starting, if not already paired |
| `--portal URL` | `https://hub.fluksio.com` | Portal used by `--enroll` |
| `--max-runs N` | `4` | Concurrent batch runs |
| `--max-cascades N` | `4` | Concurrent live cascades |
| `--max-workers N` | `4` | Local Python worker processes |
| `--gpus N` | Auto-detected | Available local GPUs |
| `--ram MB` | Auto-detected | Available local memory |
| `--worker-max-rss MB` | `0` (disabled) | Retire a worker above this resident memory after its call |
Resource defaults can also come from the corresponding `FLOW_*` settings;
see [Configuration](../reference/configuration.md). GPU detection counts the
entries of `CUDA_VISIBLE_DEVICES` when it is set, otherwise NVIDIA device
nodes. Set `--gpus` when your hardware is not detected.
Use local disk for the data directory on a cluster: SQLite's write-ahead log
does not support NFS.
### The dashboard
At a terminal, `serve` opens **Overview**, **Runs** and **Logs**:
| Key | Action |
|---|---|
| `1`, `2`, `3` | Switch tabs |
| `q` | Close the dashboard; the engine keeps running |
| `s` | Stop or start the engine |
| `r` | Restart the engine |
| `e` | Enroll with a portal |
On **Runs**, `space` selects runs, `enter` opens a chart or comparison,
`c` cancels a run, and `a` lists or downloads artifacts. Compare curves by
step, elapsed time or another metric.
Run `fluksio serve` again to reopen the dashboard. Its engine writes to
`serve.log` in the data directory; the log is truncated after 5 MB.
Without a terminal, or with `--plain`, the command prints logs directly.
## `fluksio enroll`
Connects this instance to the portal:
```sh
fluksio enroll ABCD-1234
```
Get a code from **Instances → Add instance** in the portal. Run the command
from the instance's project directory, or select it explicitly.
| Option | Purpose |
|---|---|
| `--portal URL` | Use another portal instead of `https://hub.fluksio.com` |
| `--as EMAIL` | Choose the local superuser represented by portal sessions |
| `--data-dir PATH` | Select instance storage |
| `--global` | Select `~/.fluksio` |
With multiple local superusers, `--as` is required. A new code replaces an
existing enrollment only after the portal accepts it. A running engine picks
up the change and reconnects automatically.
The instance keeps an outbound connection to the portal and reconnects after
network interruptions. See [Accounts and the portal](../interface/portal.md).
## `fluksio worker`
Starts the same agent as the standalone `fluksio-worker` command:
```sh
FLUKSIO_WORKER_TOKEN=your-token fluksio worker \
--url wss://api.example.com/api/v1/workers/attach --labels gpu
```
Create the worker token on the engine first. See [Remote workers](workers.md)
for token creation, interpreter selection and labels.
## Talking to an engine
Client commands use the nearest `.fluksio/client.json`, or the global
configuration when there is no project instance. Legacy client configuration
is a fallback.
`FLUKSIO_URL` and `FLUKSIO_TOKEN` override saved values; command options
`--url` and `--token` override both.
### `fluksio login`
```sh
fluksio login --url https://api.example.com
```
Prompts for credentials and saves an API token for a remote engine.
`--global` saves it in `~/.fluksio` instead of this project. Your locally
started engine already saves a token.
### `fluksio sync`
```sh
fluksio sync study
fluksio sync examples/myresearch
fluksio sync --dry-run study
```
Imports modules, packages or directories and uploads the declared flows.
With no target, it searches the current directory. Directory searches skip
hidden folders, virtual environments, `__pycache__` and `node_modules`.
File paths are not targets: use `study`, not `study.py`.
The generated node bodies import your functions. Keep the project available
to each machine running them; sync does not copy your package to a remote
worker. Uploading source requires a superuser token.
| Flag | Purpose |
|---|---|
| `--dry-run` | Print flow documents and node bodies without uploading |
| `--no-publish` | Save changes as drafts |
| `--force` | Overwrite canvas edits that conflict with your local definitions |
Sync records source digests for [stage caching](../concepts/runs.md#stage-caching)
and refreshes local workers, so edited imports are picked up. An unchanged
flow reports `unchanged`. See the
[data science guide](../getting-started/data-science.md) for a complete example.
### `fluksio run`
```sh
fluksio run train --lr 0.05 --epochs 20 --follow
```
Syncs local code, then submits a run. Flags such as `--lr` name the flow's
declared inputs and are converted to their declared types. Unknown inputs are
rejected. Use `--param name=a,b` with `sweep` for multiple values.
After the first sync, `run` imports the module that declared the flow. If no
declaration is recorded, it searches the working directory and skips paths
covered by `.gitignore`. Import errors in that fallback search produce
warnings. Use `--sync PATH` to choose a target explicitly.
| Flag | Purpose |
|---|---|
| `--wait` | Wait for completion; exit non-zero if the run fails |
| `--follow` | Wait and print emitted metrics |
| `--timeout SECONDS` | Stop waiting after this long; leave the run active |
| `--seed N` | Record the seed and fill a declared input named `seed` |
| `--defaults` | Use declared input defaults without prompting |
| `--sync PATH` | Choose what to sync; repeat for multiple targets |
| `--no-sync` | Run the version already on the engine |
| `--no-cache` | Execute every node |
| `--local` | Start an engine in this command and wait for the result |
With no parameters at a terminal, the command prompts for each input.
Press Enter to keep its default. Scripted commands do not prompt.
Ctrl-C while waiting cancels the run and exits with status 130.
A seed does not initialize your libraries' random number generators. Declare
a `seed` input and pass it to your code if it controls randomness.
Inputs can reference earlier results:
```sh
fluksio run evaluate --dataset @run:<run-id>.dataset
fluksio run evaluate --dataset sha256:<digest>
```
`@run:<id>.<output>` resolves a run output; `sha256:<digest>` resolves
stored bytes into an artifact reference.
`--local` uses the same data directory, history and artifacts as `serve`.
It starts and stops the engine for each command, so keep `serve` running
when iterating frequently.
### `fluksio status`
```sh
fluksio status
fluksio status --watch
```
Shows engine health, portal connectivity, flows, recent runs and available
resources. `--watch` refreshes every five seconds. `--local` reads the local
instance without a served engine.
### `fluksio runs`
```sh
fluksio runs --flow train --limit 10
```
Lists recent runs, newest first. Omit `--flow` for all flows.
`--local` reads the local instance.
### `fluksio retry`
```sh
fluksio retry <run-id>
fluksio retry --group <sweep-id>
```
Submits a new run with the previous inputs, seed and group, linked to its
parent. `--group` retries finished runs in that group that did not end
`ok`. Active runs must be cancelled before retrying.
Queued runs survive engine restarts. Interrupted running runs become
`abandoned` after their lease expires; retry them to resume with any
available cached stages.
### `fluksio flavors`
```sh
fluksio flavors
```
Lists named resource sizes, usable as `@node(resources="gpu-small")`.
Edit them in **Workers** or through the flavors API.
### `fluksio sweep`
```sh
fluksio sweep train --param lr=0.01,0.1 --param epochs=10,50 --wait
```
Submits every combination of the declared input values: four runs here,
sharing a `group_id`. Runs execute concurrently within the engine's limits.
To vary seeds, the flow must declare a `seed` input; use
`--param seed=1,2,3`.
`--seed`, `--sync`, `--no-sync`, `--no-cache`, `--timeout` and
`--local` work as for `run`. `--wait` exits non-zero if any run fails.
### `fluksio export`
```sh
fluksio export metrics --flow train --list
fluksio export metrics --flow train --name train.loss -o curves.csv
fluksio export runs --flow train --status ok -o results.csv
```
`metrics` exports a row per run, metric and step. `--list` lists metric
names; `--name A,B` selects them; `--stride N` keeps every Nth point.
`runs` exports a row per run with inputs, final numeric results, status,
duration and code identifiers. Use `--params A,B` and `--metrics A,B` to
select columns, including dotted paths such as `final_metrics.train_loss`.
Both support `--flow`, repeated `--run ID`, `--group`, `--status`,
`--since`, `--until` and `--local`. The run ID joins results to metrics.
An export stops at the newest 10 000 runs, and when a selection matches more,
the CLI and the Python client warn how many were left out.
Output defaults to CSV on stdout. `-o FILE` writes a file; `--format jsonl`
writes JSON Lines. Parquet requires `pip install 'fluksio[parquet]'` and
`--format parquet -o FILE`.
In Python, `Client.export_metrics()` and `Client.export_runs()` return the
same rows as lists of dictionaries.
### `fluksio artifacts`
```sh
fluksio artifacts <run-id>
fluksio artifacts <run-id> train.weights -o weights.json
```
Lists a run's returned artifacts, including their message names, sizes and
filenames. Pass a name from that list to download it. Output uses the stored
filename unless `-o` selects another path. `--local` reads the local instance.
## What lives in the data directory
The main files and directories are:
| Path | Contents |
|---|---|
| `client.json` | Local API URL and token |
| `fluksio.db` | SQLite database: accounts, runs, metrics and operations history |
| `flows/` | Git repository of flows, node sources, dashboards and module requirements |
| `artifacts/` | Files stored by content digest |
| `user-venv/` | Managed node environment, when used |
| `secrets.enc` | Encrypted credentials, outside the flow repository |
| `cloud.json` | Portal enrollment, when paired |
| `secret_key` | Session signing and encryption key |
| `serve.pid`, `serve.log` | Running engine information and terminal dashboard logs |
| `env` | Optional instance settings |
The flow repository records edits in Git. Runs record its commit and code
digests; code-defined nodes also depend on your project files, so keep that
project's history too.
Back up the data directory while the engine is stopped. An adopted venv and
your project files live outside it and need their own backup.
## Settings
Settings come from environment variables or an `env` file in the data
directory. Common choices:
| Variable | Purpose |
|---|---|
| `FLUKSIO_HOME` | Select the CLI's data directory |
| `NODE_VENV` | `auto` adopts an existing venv; `managed` requests a separate one; a path selects an interpreter |
| `REDIS_HOST` | Use Redis for flow state and queues |
| `FRONTEND_HOST` | Browser address used in mail, OAuth metadata and panel links |
| `ENVIRONMENT` | `production` disables interactive API documentation |
| `MCP_ENABLED` | Enable the [agent endpoint](agents.md) |
See [Configuration](../reference/configuration.md) for defaults and the full
list.