stroblmeandClaude Opus 5
058f16ec1d
Each was a loose end recorded under `### SDK` in the notepad. `serve` takes its own pidfile down on SIGTERM. uvicorn restores the handler it found and re-raises the signal it stopped on, so the default handler ended the process without unwinding and the `finally` never ran — which is what a stop sends, and what left `serve.pid` behind. `serve.log` is cut back past 5 MB by the engine rather than by the screen that started it, so an adopted engine is bounded too. Gated on its own stdout being an appended regular file, which is what makes the cut safe: the kernel then puts the next write at the new end. Cards are counted from `/dev/nvidia[0-9]*`, so `FLOW_GPUS`/`--gpus` of 0 means "work it out" the way `FLOW_CPUS` always has. The engine counts, not the accountant — a remote worker builds one of those from its own inventory, and detecting there would hand it the engine host's cards. The worker counts last: what a batch job says it was granted still wins. `GET /runs/metrics/names` is the distinct over a selection that `--list` and the terminal's metric picker were approximating by reading the newest run that had measured anything, which missed a name only an older run ever wrote. `MetricSink` announces each batch it has written (`run_metric`, carrying the names). Not a per-point event: one covers up to 500 points or two seconds of them, and the rows stay the record. The terminal comparison fills in as the first readings land instead of staying blank until reopened, and the browser refetches the run and any comparison rather than the list behind them. `retry --group` pages the list route by `before` instead of stopping at 500. The terminal dashboard takes the terminal's colours (`ansi-dark`), and the web UI can re-pair from Settings without disconnecting first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PRQ9bmTvCbqCwXo9mxZzzV
218 lines
11 KiB
Markdown
218 lines
11 KiB
Markdown
# Configuration
|
|
|
|
Every setting comes from the environment, or from an env file. Which file
|
|
depends on how the instance was started:
|
|
|
|
| Started with | Reads |
|
|
|---|---|
|
|
| `fluksio serve` | `env` inside the data directory (`$FLUKSIO_ENV_FILE`) |
|
|
| the Docker stack | `.env` beside `docker/` |
|
|
|
|
Anything already exported wins over the file.
|
|
|
|
## Storage
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `DATA_DIR` | `flow-data` (`./.fluksio` via the CLI; `~/.fluksio` with `--global`) | everything below derives from this |
|
|
| `DATABASE_URL` | SQLite in `DATA_DIR` | any SQLAlchemy URL |
|
|
| `FLOWS_DIR` | `$DATA_DIR/flows` | the git repository holding flows |
|
|
| `SECRETS_FILE` | `$DATA_DIR/secrets.enc` | encrypted credentials, kept outside the repo |
|
|
| `ALERTS_FILE` | `$DATA_DIR/alerts.json` | channels and rules |
|
|
| `PROVISIONERS_FILE` | `$DATA_DIR/provisioners.json` | clusters a machine can be started from; absent means none |
|
|
| `PANELS_FILE` | `$DATA_DIR/panels.json` | wall-panel pairings |
|
|
| `OAUTH_PRIVATE_KEY_FILE` | `$DATA_DIR/oauth-key.pem` | signs agent and worker tokens |
|
|
| `CLOUD_CONFIG_FILE` | `$DATA_DIR/cloud.json` | the portal enrolment, if any |
|
|
| `NODE_VENV` | `auto` | which interpreter node code runs on — see below |
|
|
|
|
Set `DATA_DIR` and the rest follow. Set one explicitly and it wins, which is
|
|
what the container images do to pin everything onto `/data`.
|
|
|
|
`NODE_VENV` is the exception, being about an environment rather than a path:
|
|
|
|
| Value | What node code runs on |
|
|
|---|---|
|
|
| `auto` (default) | the venv Fluksio was installed into, when it was installed into one and there is no venv of its own already built. `pip install fluksio` beside your own packages is this case, and the packages are then already there — the Modules screen turns read-only, because that environment is not Fluksio's to install into. |
|
|
| `managed` | a venv the engine builds under `DATA_DIR` and owns, which the Modules screen installs into with `uv pip sync`. The container images set this: the venv in them holds the app and nothing of anybody else's. |
|
|
| a path | that interpreter, or that venv, whatever it is. |
|
|
|
|
An instance that already has a managed venv keeps it on upgrade under
|
|
`auto`, because it may hold packages somebody installed on purpose.
|
|
|
|
!!! warning "The four files that must be on persistent storage"
|
|
|
|
`secrets.enc`, `alerts.json`, `panels.json` and `oauth-key.pem` are written
|
|
at runtime. In a container, anything not on a volume lands in the writable
|
|
layer and is lost on the next rebuild, un-pairing every screen and
|
|
revoking every agent.
|
|
|
|
## State
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `REDIS_HOST` | unset | without it, flow state lives in memory and does not survive a restart |
|
|
| `REDIS_PORT` | `6379` | |
|
|
|
|
Flow state is the last value of every message, node memory, and the run queue.
|
|
Redis here is persistence, not a cache. Run it with append-only persistence on.
|
|
|
|
## Identity and access
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `SECRET_KEY` | generated | signs sessions and derives the secrets-store key |
|
|
| `ACCESS_TOKEN_EXPIRE_MINUTES` | `11520` (8 days) | |
|
|
| `FIRST_SUPERUSER` | — | absent means the CLI creates one on first run |
|
|
| `FIRST_SUPERUSER_PASSWORD` | — | absent means one is generated and printed once |
|
|
| `DOMAIN` | `localhost` | what the API and OAuth issuer are built from |
|
|
| `FRONTEND_HOST` | `http://localhost:5173` | used in mails, OAuth metadata and panel pairing links |
|
|
| `BACKEND_CORS_ORIGINS` | `[]` | comma-separated; `FRONTEND_HOST` is always allowed |
|
|
|
|
## Where the interface looks for the API
|
|
|
|
The dashboard is a static bundle, so this one is a **build** argument of the
|
|
`frontend` image rather than a setting the running stack reads.
|
|
|
|
| Argument | Used by | Effect |
|
|
|---|---|---|
|
|
| `VITE_API_URL` | `frontend` at build time | the address the interface calls |
|
|
|
|
The compose stack takes it from `.env`, falling back to `https://api.${DOMAIN}`.
|
|
`scripts/setup.sh` writes it there with the scheme `ENVIRONMENT` implies, so a
|
|
local build calls `http://` and does not fail a certificate check nothing is
|
|
there to satisfy.
|
|
|
|
Empty is the useful value: the interface then addresses the API relative to
|
|
whichever origin served the page, so one image answers on a hostname, on a
|
|
`http://<host-ip>:<port>`, and through an ssh tunnel alike, and no origin has
|
|
to be added to `BACKEND_CORS_ORIGINS`, because there is only one.
|
|
`docker/compose.lan.yml` builds it that way and puts an `/api` proxy in front
|
|
of the backend to complete it; see
|
|
[getting started](../getting-started/facility-automation.md#on-your-own-network-by-address).
|
|
|
|
Set it to an absolute URL only when the API genuinely lives somewhere else, and
|
|
remember it is fixed at build time: changing it means rebuilding that image.
|
|
|
|
!!! danger "Rotating `SECRET_KEY`"
|
|
|
|
The secrets store is encrypted with a key derived from it. Change it and
|
|
the store stops decrypting, and every session is signed out. Re-enter your
|
|
secrets, or plan the rotation properly.
|
|
|
|
## Environment
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `ENVIRONMENT` | `local` | `local`, `staging` or `production` |
|
|
| `PRIVATE_API_ENABLED` | `false` | unauthenticated test-only endpoints; needs `ENVIRONMENT=local` too |
|
|
| `TZ` | `UTC` | the timezone every schedule is written in |
|
|
|
|
`TZ` is the container's own, not a setting the code reads: an `inject` or a
|
|
`delay` with a cron expression fires on local time. Left at `UTC`, "off at
|
|
02:00" means two in the morning UTC, which in most of the world is neither two
|
|
o'clock nor the same hour in summer as in winter. Set it to where the
|
|
instance is.
|
|
|
|
`production` closes `/docs`, `/redoc` and the OpenAPI document, because the
|
|
schema enumerates every endpoint the instance serves, including the paths
|
|
webhook nodes mounted at runtime. It also turns a `changethis` secret from a
|
|
warning into a refusal to start.
|
|
|
|
## The engine
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `FLOW_MAX_WORKERS` | `4` | node-code subprocesses run in parallel |
|
|
| `FLOW_MAX_CASCADES` | `4` | cascades in flight at once; throughput is this over the mean cascade time, so raise it where nodes wait on a network rather than a CPU |
|
|
| `FLOW_MAX_RUNS` | `4` | batch runs driven at once. A different limit from the one above: a run drives a whole graph, and its nodes are bounded by `FLOW_MAX_WORKERS`. This is what a sweep queues behind |
|
|
| `FLOW_NODE_TIMEOUT` | `0` | seconds a node may be silent, unless it sets its own; 0 is no limit |
|
|
| `FLOW_CPUS` | `0` | cores nodes that declare `resources` may be given; 0 works it out as every core but two, which are what keeps the engine answering while the machine is busy |
|
|
| `FLOW_GPUS` | `0` | GPUs on this machine, each held by one node at a time. 0 counts NVIDIA's device nodes; say a number for anything they miss |
|
|
| `OBS_RETENTION_DAYS` | `30` | how long metrics, events and run records are kept |
|
|
| `ARTIFACT_GC_INTERVAL_S` | `3600` | how often artifact bytes nothing refers to are swept away; 0 never sweeps |
|
|
| `ARTIFACT_GC_GRACE_S` | `3600` | how long a freshly written artifact is spared, whatever refers to it |
|
|
| `MAX_ARTIFACT_BYTES` | `2147483648` | the largest body `PUT /artifacts` will take; 0 removes the limit |
|
|
| `ARTIFACT_VOLATILE_DIR` | worked out | where frames a flow only shows live are held; empty picks a directory under `/dev/shm` named for the data directory, and falls back to the temporary directory |
|
|
| `ARTIFACT_VOLATILE_BYTES` | `50331648` | how much that ring holds before the oldest frames fall out; 0 turns it off and volatile saves land in the store |
|
|
|
|
The three concurrency limits are also flags on `fluksio serve`
|
|
(`--max-workers`, `--max-cascades`, `--max-runs`), as is the card count,
|
|
`--gpus`. The flags
|
|
outrank the file, and the engine says which numbers it started with in its
|
|
first lines. Each pool size must be at least 1 and the card count at least 0:
|
|
a number below that is refused as a flag error naming it, rather than read as
|
|
the default. Leave one empty (or unset) to get the default.
|
|
|
|
An artifact is referred to by a run that recorded it or by a message currently
|
|
holding it; anything else is what a camera published four hours ago, and the
|
|
sweep is what keeps a flow streaming media from filling the disk. It stands
|
|
aside entirely while a run is in flight, since a node may store a checkpoint
|
|
long before it returns the reference to it.
|
|
|
|
Frames saved with `volatile=True` skip all of that. They go to a ring in memory
|
|
instead of the volume, the oldest falling out once the newest need the room,
|
|
and the engine pushes them down the websocket to whichever screens are drawing
|
|
them — which is what a camera at ten frames a second needs and the store cannot
|
|
give it. A frame a run *records* is copied into the store on the way, so
|
|
returned media is kept and emitted media is not. Under Docker the ring lives in
|
|
the container's `/dev/shm`, whose default is 64 MB: raise `shm_size` alongside
|
|
`ARTIFACT_VOLATILE_BYTES`.
|
|
|
|
A node that declares nothing is not accounted against `FLOW_CPUS`; it runs on
|
|
the shared pool and is given `FLOW_CPUS / FLOW_MAX_WORKERS` as a thread cap, so
|
|
several at once cannot each size themselves to the whole machine. Setting
|
|
`OMP_NUM_THREADS` (or any of its siblings) on the engine yourself overrides
|
|
that default.
|
|
|
|
These two are this machine's figures. An attached worker reports its own when
|
|
it dials in, and a node goes to whichever machine can grant what it asked for,
|
|
so a GPU on a worker needs no `FLOW_GPUS` here. What every machine has free, and
|
|
which nodes are queued, is `GET /api/v1/workers/resources` and the Workers
|
|
screen. Named sizes live in the database and are `GET /api/v1/flavors`;
|
|
`PROVISIONERS_FILE` is where machines can be started from. See
|
|
[declaring resources](../getting-started/data-science.md#declaring-what-a-node-needs).
|
|
|
|
## Agents
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `MCP_ENABLED` | `false` | opens the `/mcp` endpoint **and** OAuth client registration |
|
|
| `MCP_TOKEN_EXPIRE_MINUTES` | `60` | an agent's token is a bearer secret held by a program |
|
|
| `MCP_REFRESH_EXPIRE_DAYS` | `30` | |
|
|
| `OAUTH_CODE_EXPIRE_SECONDS` | `60` | |
|
|
|
|
See [Agents over MCP](../code/agents.md).
|
|
|
|
## Mail
|
|
|
|
Needed for password-reset mails. Without `SMTP_HOST` and `EMAILS_FROM_EMAIL`,
|
|
mail is simply off.
|
|
|
|
| Variable | Default |
|
|
|---|---|
|
|
| `SMTP_HOST` | — |
|
|
| `SMTP_PORT` | `587` |
|
|
| `SMTP_USER` / `SMTP_PASSWORD` | — |
|
|
| `SMTP_TLS` / `SMTP_SSL` | `true` / `false` |
|
|
| `EMAILS_FROM_EMAIL` | — |
|
|
| `EMAILS_FROM_NAME` | `Fluksio` |
|
|
| `EMAIL_RESET_TOKEN_EXPIRE_HOURS` | `48` |
|
|
|
|
## Monitoring
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `SENTRY_DSN` | — | error reporting, if you want it |
|
|
|
|
## Health check
|
|
|
|
`GET /api/v1/utils/health/` is a *deep* check: it fails when the event loop is
|
|
wedged or the state backend is gone, not just when the process is up. That is
|
|
what the container healthcheck probes, and what an autoheal sidecar restarts
|
|
on.
|
|
|
|
## See also
|
|
|
|
- [The `fluksio` command](../code/cli.md) — what the data directory holds
|
|
- [Getting started: facility automation](../getting-started/facility-automation.md)
|