stroblmeandClaude Fable 5
fea57064f9
The workflow this serves: make a venv, install what you work with, then `pip install fluksio` into the same one. Building a second environment beside it was exactly wrong — the packages the nodes need are already here, and the Modules screen was asking for them a second time. `NODE_VENV=auto` (the default) adopts that venv. It declines in the three cases where adopting would be wrong: `managed` says otherwise, a managed venv already exists and may hold packages somebody installed on purpose, or the engine is not running from a venv at all. The images set `managed`, since the venv in them holds the app and nothing of anybody else's. An adopted venv is never written to. `uv pip sync` makes a venv hold exactly the manifest, so pointed at somebody's own environment it uninstalls their work and the engine with it — `sync()` refuses outright and `reconcile()` returns before it can be called at startup, which is where that would have happened first. The Modules screen lists what is installed and drops its editor; `pip` is how that environment changes. `fluksio serve` now names the interpreter node code runs on, which is the thing a data scientist most needs to know at that moment. `fluksio-worker` already defaulted `--python` to its own interpreter, so a GPU box works the same way — that was only ever undocumented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
216 lines
8.0 KiB
Markdown
216 lines
8.0 KiB
Markdown
# The `fluksio` command
|
|
|
|
```sh
|
|
pip install fluksio
|
|
```
|
|
|
|
Installs the engine and the `fluksio` command. Python 3.10 or newer, Linux or
|
|
macOS.
|
|
|
|
There is a second, smaller distribution — `fluksio-worker` — for a machine that
|
|
should only *run nodes* for an engine elsewhere. It has none of the engine in
|
|
it. See [Remote workers](workers.md).
|
|
|
|
The command is two things at once: `serve`, `enroll` and `worker` *are* an
|
|
installation, while `login`, `sync`, `run` and `runs` talk to one that may be
|
|
anywhere.
|
|
|
|
## `fluksio serve`
|
|
|
|
Runs the engine.
|
|
|
|
```sh
|
|
fluksio serve
|
|
```
|
|
|
|
On the first start it creates an admin account and prints its password **once**.
|
|
Nothing else has to be running: no database server, no message broker, no
|
|
Docker.
|
|
|
|
| Option | Default | What it does |
|
|
|---|---|---|
|
|
| `--data-dir PATH` | `~/.fluksio` (or `$FLUKSIO_HOME`) | where this installation keeps everything |
|
|
| `--host HOST` | `127.0.0.1` | what to bind |
|
|
| `--port PORT` | `8000` | what to listen on |
|
|
| `--log-level LEVEL` | `info` | uvicorn's log level |
|
|
| `--admin-email ADDR` | `admin@example.com` | the account created on first run |
|
|
| `--admin-password PW` | generated | set it instead of having one generated |
|
|
| `--enroll CODE` | — | pair with a portal as part of coming up |
|
|
| `--portal URL` | — | the portal `--enroll` redeems at |
|
|
|
|
`--enroll` with `--portal` is the one-command setup: it pairs before the engine
|
|
starts, so the connection is dialled as part of coming up rather than needing a
|
|
restart. It is skipped if the installation is already enrolled.
|
|
|
|
!!! warning "One process"
|
|
|
|
`fluksio serve` holds the flow engine. A second one is a *second engine* —
|
|
duplicated subscriptions, duplicated cron ticks, two webhooks answering the
|
|
same path. Run one, and distribute work with
|
|
[workers](workers.md) instead.
|
|
|
|
!!! note "`$HOME` on a cluster"
|
|
|
|
A login node's home directory is often NFS, where SQLite's write-ahead log
|
|
does not work — the database would be locked or corrupt. `fluksio serve`
|
|
warns when it notices; point `--data-dir` at local disk.
|
|
|
|
### What it prints
|
|
|
|
```text
|
|
Created the admin account admin@example.com
|
|
password: k3Qm-8vTpLdX
|
|
Shown once. Change it from the dashboard.
|
|
Fluksio 0.1.0 — data in /home/you/.fluksio
|
|
API http://127.0.0.1:8000/api/v1
|
|
No portal. Pair this installation with:
|
|
fluksio enroll <code> --portal https://hub.example.com
|
|
```
|
|
|
|
An enrolled installation says which portal it is on instead, and notes that the
|
|
dashboard is served from there rather than here.
|
|
|
|
## `fluksio enroll`
|
|
|
|
Pairs an existing installation with a portal.
|
|
|
|
```sh
|
|
fluksio enroll ABCD-1234 --portal https://hub.fluksio.com
|
|
```
|
|
|
|
| Option | What it does |
|
|
|---|---|
|
|
| `--portal URL` | **required** — the portal the code was minted on |
|
|
| `--as EMAIL` | the local account a portal session arrives as |
|
|
| `--data-dir PATH` | which installation, if not the default |
|
|
|
|
Get the code from the portal under **Installations → Add installation**. It is
|
|
single-use and expires in fifteen minutes. `--as` matters when the installation
|
|
has several superusers — without it, enrolment refuses rather than guessing.
|
|
|
|
Afterwards, `fluksio serve` dials the portal as it comes up. See
|
|
[Accounts and the portal](../interface/portal.md).
|
|
|
|
## `fluksio worker`
|
|
|
|
Runs nodes for an engine elsewhere. Everything after `worker` belongs to the
|
|
agent's own parser — it is the same program `fluksio-worker` installs, so the
|
|
two are interchangeable:
|
|
|
|
```sh
|
|
fluksio worker --url wss://api.example.com/api/v1/workers/attach \
|
|
--token "$FLUKSIO_WORKER_TOKEN" --labels gpu
|
|
```
|
|
|
|
See [Remote workers](workers.md).
|
|
|
|
## Talking to an engine
|
|
|
|
The four commands below are the client half: they run wherever you work, and
|
|
address an engine over its API rather than being one.
|
|
|
|
### `fluksio login`
|
|
|
|
```sh
|
|
fluksio login --url http://127.0.0.1:8000
|
|
```
|
|
|
|
Asks for an email and password, and keeps the token it gets in
|
|
`~/.config/fluksio/client.json` (`$XDG_CONFIG_HOME` is honoured). Everything
|
|
below reads it from there, or from `FLUKSIO_URL` and `FLUKSIO_TOKEN`, or from
|
|
its own `--url` and `--token`.
|
|
|
|
### `fluksio sync`
|
|
|
|
```sh
|
|
fluksio sync [PATH_OR_MODULE ...] # default: the current directory
|
|
```
|
|
|
|
Imports what you name, collects the flows the decorators declared, and uploads
|
|
each one with a generated import shim per node. A directory that is a package
|
|
is walked; a dotted name is imported as it stands; nothing is loaded from a
|
|
file path, because the shim has to import the same way.
|
|
|
|
| Flag | What it does |
|
|
|---|---|
|
|
| `--dry-run` | print the flow documents and shims, upload nothing |
|
|
| `--no-publish` | leave the upload as a draft |
|
|
| `--force` | overwrite a flow, or a node body, that was edited on the canvas |
|
|
|
|
Every sync retires the engine's workers, including one that had nothing to
|
|
upload — a worker holds your package in memory, so an edit to it is invisible
|
|
until the process goes. See
|
|
[Getting started: data science](../getting-started/data-science.md).
|
|
|
|
### `fluksio run`
|
|
|
|
```sh
|
|
fluksio run train --lr 0.05 --seed 7 [--wait]
|
|
```
|
|
|
|
Submits a run. Flags that are not its own are the flow's inputs, typed by what
|
|
the flow declares them as. `--wait` blocks until the run finishes and exits
|
|
non-zero if it failed.
|
|
|
|
### `fluksio runs`
|
|
|
|
```sh
|
|
fluksio runs [--flow train] [--limit 20]
|
|
```
|
|
|
|
The runs an engine has recorded, newest first: id, status, flow, duration, the
|
|
commit of the repository it came from, and its parameters.
|
|
|
|
## What lives in the data directory
|
|
|
|
```text
|
|
~/.fluksio/
|
|
├── fluksio.db SQLite: users, runs, metrics, observability, agents
|
|
├── flows/ a git repository — one directory per flow
|
|
│ ├── house/
|
|
│ │ ├── flow.json the published structure
|
|
│ │ ├── nodes/*.py the published node code
|
|
│ │ ├── flow.draft.json unpublished edits, if any
|
|
│ │ └── nodes.draft/*.py
|
|
│ ├── _lib/ shared node sources
|
|
│ ├── _dashboards/ dashboards, drafts and all
|
|
│ └── requirements.txt what the Modules screen installs
|
|
├── artifacts/ content-addressed bytes, two levels deep
|
|
├── user-venv/ the interpreter your node code runs on
|
|
├── secrets.enc encrypted credentials, deliberately outside flows/
|
|
├── alerts.json alert channels and rules
|
|
├── panels.json wall-panel pairings
|
|
├── oauth-key.pem signs agent tokens
|
|
├── cloud.json the portal enrolment, if there is one
|
|
├── secret_key signs sessions and derives the secrets key
|
|
└── env optional settings file
|
|
```
|
|
|
|
Two things follow from this layout and are worth internalising:
|
|
|
|
**`flows/` is a real git repository.** `git log` is the history of every change
|
|
anyone made to any flow. A run records the commit it ran at, so `git show` on
|
|
that hash is literally the code that produced the number.
|
|
|
|
**Backing up the data directory backs up the installation.** Everything else is
|
|
rebuildable. Copy it while the engine is stopped, or use SQLite's online backup
|
|
for the database if it is not.
|
|
|
|
## Settings
|
|
|
|
Settings come from the environment, or from an `env` file in the data
|
|
directory. The ones you are most likely to touch:
|
|
|
|
| Variable | Default | What it does |
|
|
|---|---|---|
|
|
| `DATA_DIR` | `~/.fluksio` via the CLI | everything below it derives from this |
|
|
| `DATABASE_URL` | SQLite in the data dir | any SQLAlchemy URL |
|
|
| `NODE_VENV` | `auto` | which interpreter node code runs on: `auto` adopts the venv Fluksio was installed into, `managed` builds one of its own, or name an interpreter |
|
|
| `REDIS_HOST` | unset | flow state in Redis instead of memory; survives a restart |
|
|
| `FRONTEND_HOST` | — | the address used in mails, OAuth metadata and panel links |
|
|
| `ENVIRONMENT` | `local` | `production` closes the interactive API schema |
|
|
| `MCP_ENABLED` | `false` | opens the [agent endpoint](agents.md) |
|
|
| `SECRET_KEY` | generated once | signs sessions, derives the secrets key |
|
|
|
|
The full list is in [Configuration](../reference/configuration.md).
|