The backend image now builds the SPA and serves it on the API's own origin (fluksio/spa.py), so an instance is a single container with no proxy, DNS, port 80 or .env: `docker compose up -d` with the new compose.yml runs it on localhost:8080. It starts on fluksio serve (persisted secret key, first admin printed once) and brings its own Redis when REDIS_HOST is unset. Slimmer: python:3.13-slim, no dev group, uv once, fastapi without the cloud CLI. Optional features are per-feature extras (mqtt, influx, email, sentry, ml, server for all) picked with the EXTRAS build argument; numpy moved to `ml`. ffmpeg left the image for imageio-ffmpeg via the Modules screen. The operator stack runs the same image with app. and api. routed to it; the nginx frontend service, the LAN nginx proxy and DOMAIN in the backend are gone, and the MCP issuer is FRONTEND_HOST.
375 lines
14 KiB
Markdown
375 lines
14 KiB
Markdown
# The `fluksio` command
|
|
|
|
Install on Linux or macOS with Python 3.12 or newer:
|
|
|
|
```sh
|
|
pip install fluksio
|
|
fluksio serve
|
|
```
|
|
|
|
For the browser dashboard, sign up at [fluksio.com](https://fluksio.com), add
|
|
an instance, then run `fluksio enroll <code>` in the same directory.
|
|
|
|
Use `fluksio --help` or `fluksio COMMAND --help` for command options.
|
|
`pip install 'fluksio[server]'` adds MQTT, InfluxDB, outbound mail, error
|
|
reporting and numpy, or name only what you need, such as `fluksio[mqtt]`. A
|
|
machine that only executes remote nodes can install
|
|
[`fluksio-worker`](workers.md).
|
|
|
|
## Where an instance lives
|
|
|
|
By default, Fluksio finds the nearest `.fluksio/` directory by walking up from
|
|
your working directory. `serve` creates one in the current directory if needed.
|
|
Separate project directories can therefore hold separate instances.
|
|
|
|
`--global` selects `~/.fluksio`. `--data-dir PATH` or `FLUKSIO_HOME`
|
|
selects a directory explicitly. The data directory contains a `.gitignore`
|
|
that excludes its database, credentials and other contents from your project.
|
|
|
|
## `fluksio serve`
|
|
|
|
Starts the API and flow engine. No external database, broker or Docker is
|
|
needed for the default setup.
|
|
|
|
```sh
|
|
fluksio serve
|
|
fluksio serve --plain --port 9000
|
|
```
|
|
|
|
The first start prints an admin password once. Local CLI commands receive a
|
|
saved token automatically; `login` is for connecting to another engine.
|
|
|
|
The default port is 8000, advancing to the next free port if necessary.
|
|
An explicit `--port` fails when occupied. The startup banner and saved client
|
|
configuration show the selected URL.
|
|
|
|
Run one engine per instance. At a terminal, a second `serve` reattaches to
|
|
that instance's running engine; plain mode refuses to start a duplicate.
|
|
|
|
| Option | Default | Purpose |
|
|
|---|---|---|
|
|
| `--data-dir PATH` | Nearest `.fluksio`, or a new one here | Instance storage |
|
|
| `--global` | Off | Use `~/.fluksio` |
|
|
| `--host HOST` | `127.0.0.1` | Bind address |
|
|
| `--port PORT` | `8000`, or next free | Listening port |
|
|
| `--plain` | Off at a terminal | Print logs instead of opening the terminal dashboard |
|
|
| `--log-level LEVEL` | `info` | Uvicorn log level |
|
|
| `--admin-email ADDR` | `admin@example.com` | First admin account |
|
|
| `--admin-password PW` | Generated | First admin password |
|
|
| `--enroll CODE` | Unset | Enroll before starting, if not already paired |
|
|
| `--portal URL` | `https://hub.fluksio.com` | Portal used by `--enroll` |
|
|
| `--max-runs N` | `4` | Concurrent batch runs |
|
|
| `--max-cascades N` | `4` | Concurrent live cascades |
|
|
| `--max-workers N` | `4` | Local Python worker processes |
|
|
| `--gpus N` | Auto-detected | Available local GPUs |
|
|
| `--ram MB` | Auto-detected | Available local memory |
|
|
| `--worker-max-rss MB` | `0` (disabled) | Retire a worker above this resident memory after its call |
|
|
|
|
Resource defaults can also come from the corresponding `FLOW_*` settings;
|
|
see [Configuration](../reference/configuration.md). GPU detection counts the
|
|
entries of `CUDA_VISIBLE_DEVICES` when it is set, otherwise NVIDIA device
|
|
nodes. Set `--gpus` when your hardware is not detected.
|
|
|
|
Use local disk for the data directory on a cluster: SQLite's write-ahead log
|
|
does not support NFS.
|
|
|
|
### The dashboard
|
|
|
|
At a terminal, `serve` opens **Overview**, **Runs** and **Logs**:
|
|
|
|
| Key | Action |
|
|
|---|---|
|
|
| `1`, `2`, `3` | Switch tabs |
|
|
| `q` | Close the dashboard; the engine keeps running |
|
|
| `s` | Stop or start the engine |
|
|
| `r` | Restart the engine |
|
|
| `e` | Enroll with a portal |
|
|
|
|
On **Runs**, `space` selects runs, `enter` opens a chart or comparison,
|
|
`c` cancels a run, and `a` lists or downloads artifacts. Compare curves by
|
|
step, elapsed time or another metric.
|
|
|
|
Run `fluksio serve` again to reopen the dashboard. Its engine writes to
|
|
`serve.log` in the data directory; the log is truncated after 5 MB.
|
|
Without a terminal, or with `--plain`, the command prints logs directly.
|
|
|
|
## `fluksio enroll`
|
|
|
|
Connects this instance to the portal:
|
|
|
|
```sh
|
|
fluksio enroll ABCD-1234
|
|
```
|
|
|
|
Get a code from **Instances → Add instance** in the portal. Run the command
|
|
from the instance's project directory, or select it explicitly.
|
|
|
|
| Option | Purpose |
|
|
|---|---|
|
|
| `--portal URL` | Use another portal instead of `https://hub.fluksio.com` |
|
|
| `--as EMAIL` | Choose the local superuser represented by portal sessions |
|
|
| `--data-dir PATH` | Select instance storage |
|
|
| `--global` | Select `~/.fluksio` |
|
|
|
|
With multiple local superusers, `--as` is required. A new code replaces an
|
|
existing enrollment only after the portal accepts it. A running engine picks
|
|
up the change and reconnects automatically.
|
|
|
|
The instance keeps an outbound connection to the portal and reconnects after
|
|
network interruptions. See [Accounts and the portal](../interface/portal.md).
|
|
|
|
## `fluksio worker`
|
|
|
|
Starts the same agent as the standalone `fluksio-worker` command:
|
|
|
|
```sh
|
|
FLUKSIO_WORKER_TOKEN=your-token fluksio worker \
|
|
--url wss://fluksio.example.com/api/v1/workers/attach --labels gpu
|
|
```
|
|
|
|
Create the worker token on the engine first. See [Remote workers](workers.md)
|
|
for token creation, interpreter selection and labels.
|
|
|
|
## Talking to an engine
|
|
|
|
Client commands use the nearest `.fluksio/client.json`, or the global
|
|
configuration when there is no project instance. Legacy client configuration
|
|
is a fallback.
|
|
`FLUKSIO_URL` and `FLUKSIO_TOKEN` override saved values; command options
|
|
`--url` and `--token` override both.
|
|
|
|
### `fluksio login`
|
|
|
|
```sh
|
|
fluksio login --url https://fluksio.example.com
|
|
```
|
|
|
|
Prompts for credentials and saves an API token for a remote engine.
|
|
`--global` saves it in `~/.fluksio` instead of this project. Your locally
|
|
started engine already saves a token.
|
|
|
|
### `fluksio sync`
|
|
|
|
```sh
|
|
fluksio sync study
|
|
fluksio sync examples/myresearch
|
|
fluksio sync --dry-run study
|
|
```
|
|
|
|
Imports modules, packages or directories and uploads the declared flows.
|
|
With no target, it searches the current directory. Directory searches skip
|
|
hidden folders, virtual environments, `__pycache__` and `node_modules`.
|
|
File paths are not targets: use `study`, not `study.py`.
|
|
|
|
The generated node bodies import your functions. Keep the project available
|
|
to each machine running them; sync does not copy your package to a remote
|
|
worker. Uploading source requires a superuser token.
|
|
|
|
The flow records the git repository it was declared in, or the directory when
|
|
there is none, and each run hashes the Python files below it. The engine
|
|
refuses `/`, its home directory and relative paths there, so run sync from
|
|
inside the project's repository.
|
|
|
|
| Flag | Purpose |
|
|
|---|---|
|
|
| `--dry-run` | Print flow documents and node bodies without uploading |
|
|
| `--no-publish` | Save changes as drafts |
|
|
| `--force` | Overwrite canvas edits that conflict with your local definitions |
|
|
|
|
Sync records source digests for [stage caching](../concepts/runs.md#stage-caching)
|
|
and refreshes local workers, so edited imports are picked up. An unchanged
|
|
flow reports `unchanged`. See the
|
|
[data science guide](../getting-started/data-science.md) for a complete example.
|
|
|
|
### `fluksio run`
|
|
|
|
```sh
|
|
fluksio run train --lr 0.05 --epochs 20 --follow
|
|
```
|
|
|
|
Syncs local code, then submits a run. Flags such as `--lr` name the flow's
|
|
declared inputs and are converted to their declared types. Unknown inputs are
|
|
rejected. Use `--param name=a,b` with `sweep` for multiple values.
|
|
|
|
Everything after `--` is an input written as `name=value`. Use it for an input
|
|
that shares its name with a flag of `run`, such as `timeout`:
|
|
|
|
```sh
|
|
fluksio run train --wait --timeout 600 -- timeout=30 lr=0.05
|
|
```
|
|
|
|
After the first sync, `run` imports the module that declared the flow. If no
|
|
declaration is recorded, it searches the working directory and skips paths
|
|
covered by `.gitignore`. Import errors in that fallback search produce
|
|
warnings. Use `--sync PATH` to choose a target explicitly.
|
|
|
|
| Flag | Purpose |
|
|
|---|---|
|
|
| `--wait` | Wait for completion; exit non-zero if the run fails |
|
|
| `--follow` | Wait and print emitted metrics |
|
|
| `--timeout SECONDS` | Stop waiting after this long; leave the run active |
|
|
| `--seed N` | Record the seed and fill a declared input named `seed` |
|
|
| `--defaults` | Use declared input defaults without prompting |
|
|
| `--sync PATH` | Choose what to sync; repeat for multiple targets |
|
|
| `--no-sync` | Run the version already on the engine |
|
|
| `--no-cache` | Execute every node |
|
|
| `--local` | Start an engine in this command and wait for the result |
|
|
|
|
With no parameters at a terminal, the command prompts for each input.
|
|
Press Enter to keep its default. Scripted commands do not prompt.
|
|
Ctrl-C while waiting cancels the run and exits with status 130.
|
|
|
|
A seed does not initialize your libraries' random number generators. Declare
|
|
a `seed` input and pass it to your code if it controls randomness.
|
|
|
|
Inputs can reference earlier results:
|
|
|
|
```sh
|
|
fluksio run evaluate --dataset @run:<run-id>.dataset
|
|
fluksio run evaluate --dataset sha256:<digest>
|
|
```
|
|
|
|
`@run:<id>.<output>` resolves a run output; `sha256:<digest>` resolves
|
|
stored bytes into an artifact reference.
|
|
|
|
`--local` uses the same data directory, history and artifacts as `serve`.
|
|
It starts and stops the engine for each command, so keep `serve` running
|
|
when iterating frequently.
|
|
|
|
### `fluksio status`
|
|
|
|
```sh
|
|
fluksio status
|
|
fluksio status --watch
|
|
```
|
|
|
|
Shows engine health, portal connectivity, flows, recent runs and available
|
|
resources. `--watch` refreshes every five seconds. `--local` reads the local
|
|
instance without a served engine.
|
|
|
|
### `fluksio runs`
|
|
|
|
```sh
|
|
fluksio runs --flow train --limit 10
|
|
```
|
|
|
|
Lists recent runs, newest first. Omit `--flow` for all flows.
|
|
`--local` reads the local instance.
|
|
|
|
### `fluksio retry`
|
|
|
|
```sh
|
|
fluksio retry <run-id>
|
|
fluksio retry --group <sweep-id>
|
|
```
|
|
|
|
Submits a new run with the previous inputs, seed and group, linked to its
|
|
parent. `--group` retries finished runs in that group that did not end
|
|
`ok`. Active runs must be cancelled before retrying.
|
|
|
|
Queued runs survive engine restarts. Interrupted running runs become
|
|
`abandoned` after their lease expires; retry them to resume with any
|
|
available cached stages.
|
|
|
|
### `fluksio flavors`
|
|
|
|
```sh
|
|
fluksio flavors
|
|
```
|
|
|
|
Lists named resource sizes, usable as `@node(resources="gpu-small")`.
|
|
Edit them in **Workers** or through the flavors API.
|
|
|
|
### `fluksio sweep`
|
|
|
|
```sh
|
|
fluksio sweep train --param lr=0.01,0.1 --param epochs=10,50 --wait
|
|
```
|
|
|
|
Submits every combination of the declared input values: four runs here,
|
|
sharing a `group_id`. Runs execute concurrently within the engine's limits.
|
|
To vary seeds, the flow must declare a `seed` input; use
|
|
`--param seed=1,2,3`.
|
|
|
|
`--seed`, `--sync`, `--no-sync`, `--no-cache`, `--timeout` and
|
|
`--local` work as for `run`. `--wait` exits non-zero if any run fails.
|
|
|
|
### `fluksio export`
|
|
|
|
```sh
|
|
fluksio export metrics --flow train --list
|
|
fluksio export metrics --flow train --name train.loss -o curves.csv
|
|
fluksio export runs --flow train --status ok -o results.csv
|
|
```
|
|
|
|
`metrics` exports a row per run, metric and step. `--list` lists metric
|
|
names; `--name A,B` selects them; `--stride N` keeps every Nth point.
|
|
|
|
`runs` exports a row per run with inputs, final numeric results, status,
|
|
duration and code identifiers. Use `--params A,B` and `--metrics A,B` to
|
|
select columns, including dotted paths such as `final_metrics.train_loss`.
|
|
|
|
Both support `--flow`, repeated `--run ID`, `--group`, `--status`,
|
|
`--since`, `--until` and `--local`. The run ID joins results to metrics.
|
|
An export stops at the newest 10 000 runs, and when a selection matches more,
|
|
the CLI and the Python client warn how many were left out.
|
|
|
|
Output defaults to CSV on stdout. `-o FILE` writes a file; `--format jsonl`
|
|
writes JSON Lines. Parquet requires `pip install 'fluksio[parquet]'` and
|
|
`--format parquet -o FILE`.
|
|
|
|
In Python, `Client.export_metrics()` and `Client.export_runs()` return the
|
|
same rows as lists of dictionaries.
|
|
|
|
### `fluksio artifacts`
|
|
|
|
```sh
|
|
fluksio artifacts <run-id>
|
|
fluksio artifacts <run-id> train.weights -o weights.json
|
|
```
|
|
|
|
Lists a run's returned artifacts, including their message names, sizes and
|
|
filenames. Pass a name from that list to download it. Output uses the stored
|
|
filename unless `-o` selects another path. `--local` reads the local instance.
|
|
|
|
## What lives in the data directory
|
|
|
|
The main files and directories are:
|
|
|
|
| Path | Contents |
|
|
|---|---|
|
|
| `client.json` | Local API URL and token |
|
|
| `fluksio.db` | SQLite database: accounts, runs, metrics and operations history |
|
|
| `flows/` | Git repository of flows, node sources, dashboards and module requirements |
|
|
| `artifacts/` | Files stored by content digest |
|
|
| `user-venv/` | Managed node environment, when used |
|
|
| `secrets.enc` | Encrypted credentials, outside the flow repository |
|
|
| `cloud.json` | Portal enrollment, when paired |
|
|
| `secret_key` | Session signing and encryption key |
|
|
| `serve.pid`, `serve.log` | Running engine information and terminal dashboard logs |
|
|
| `env` | Optional instance settings |
|
|
|
|
The flow repository records edits in Git. Runs record its commit and code
|
|
digests; code-defined nodes also depend on your project files, so keep that
|
|
project's history too.
|
|
|
|
Back up the data directory while the engine is stopped. An adopted venv and
|
|
your project files live outside it and need their own backup.
|
|
|
|
## Settings
|
|
|
|
Settings come from environment variables or an `env` file in the data
|
|
directory. Common choices:
|
|
|
|
| Variable | Purpose |
|
|
|---|---|
|
|
| `FLUKSIO_HOME` | Select the CLI's data directory |
|
|
| `NODE_VENV` | `auto` adopts an existing venv; `managed` requests a separate one; a path selects an interpreter |
|
|
| `REDIS_HOST` | Use Redis for flow state and queues |
|
|
| `FRONTEND_HOST` | Browser address used in mail, OAuth metadata and panel links |
|
|
| `ENVIRONMENT` | `production` disables interactive API documentation |
|
|
| `MCP_ENABLED` | Enable the [agent endpoint](agents.md) |
|
|
|
|
See [Configuration](../reference/configuration.md) for defaults and the full
|
|
list.
|