Melvin Strobl
72c1d73e96
C4: a device's `viewers` (superusers by default, or members) covers the image and video messages its ports publish. Anyone else gets none of their values on the socket, in the catalogue or a flow's state, 403 for their history, for the frames they hold (the last 8 included) and for TURN credentials, no bytes when asking for them, and `bye: forbidden` for a live view. A device credential stays bounded by its dashboards. The device page sets it in its Camera section, and a tile the viewer may not see says "No access to this camera" and asks the camera for nothing. C3: - A volatile upload is held to MAX_FRAME_BYTES (4 MiB, 413 above it, read without extra copies), and so is a frame on a device's socket. - A frame the engine holds in memory is written to the ring directory before a local worker is handed it, so load_artifact finds it. - A socket the bus dropped events for catches up with a snapshot, demand included, and a device's credential is checked again per frame, cached by the flow store revision and its nonce, so a lost event never keeps an unpaired device's frames coming. - A report is at most 64 KB once its named config_schemas are put in place. - A device away for 60 s has its true bool messages set to false. - Refused frames are logged by fixed keys, a frame is named after its message, `fluksio serve` waits 5 s for open connections, a tile with a stream URL creates no demand, the 8-name cap keeps the newest names. - Every container's log rotates (json-file, 3 x 10 MB).
232 lines
12 KiB
Markdown
232 lines
12 KiB
Markdown
# Configuration
|
|
|
|
Most installations can keep the defaults. Settings come from environment
|
|
variables and an env file:
|
|
|
|
| How you start Fluksio | Env file |
|
|
|---|---|
|
|
| `fluksio serve` | `<data-dir>/env`, unless `FLUKSIO_ENV_FILE` names another file |
|
|
| Docker with the Makefile | `.env` in this repository's root |
|
|
|
|
Exported variables override the file. `fluksio serve` flags override both.
|
|
For CLI installations, select storage with `--data-dir`, `FLUKSIO_HOME` or
|
|
`--global`; see [The fluksio command](../code/cli.md).
|
|
|
|
## Storage
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `DATA_DIR` | `flow-data`; CLI uses `./.fluksio` or `~/.fluksio` with `--global` | base directory for instance data |
|
|
| `DATABASE_URL` | SQLite at `$DATA_DIR/fluksio.db` | SQLite URL; other database engines are unsupported |
|
|
| `FLOWS_DIR` | `$DATA_DIR/flows` | flow git repository |
|
|
| `SECRETS_FILE` | `$DATA_DIR/secrets.enc` | encrypted credentials |
|
|
| `ALERTS_FILE` | `$DATA_DIR/alerts.json` | alert channels and rules |
|
|
| `WEBPUSH_FILE` | `$DATA_DIR/webpush.json` | push keys and browser subscriptions |
|
|
| `PROVISIONERS_FILE` | `$DATA_DIR/provisioners.json` | optional worker provisioning configuration |
|
|
| `DEVICES_FILE` | `$DATA_DIR/devices.json` | device credential nonces, outside the flow repository |
|
|
| `PANELS_FILE` | `$DATA_DIR/panels.json` | panels of an instance from before devices, moved into devices on first start and renamed `.migrated` |
|
|
| `OAUTH_PRIVATE_KEY_FILE` | `$DATA_DIR/oauth-key.pem` | agent and worker signing key |
|
|
| `CLOUD_CONFIG_FILE` | `$DATA_DIR/cloud.json` | optional portal enrollment |
|
|
| `AI_CONFIG_FILE` | `$DATA_DIR/ai.json` | optional assistant provider configuration |
|
|
| `CHATS_DIR` | `$DATA_DIR/chats` | assistant transcripts |
|
|
| `NODE_VENV` | `auto` | interpreter for node code |
|
|
|
|
Changing `DATA_DIR` moves the derived paths unless you set them individually.
|
|
Docker stores instance data under `/data` on the `app-flow-data` volume.
|
|
|
|
| `NODE_VENV` value | Behavior |
|
|
|---|---|
|
|
| `auto` | keeps an existing managed venv; otherwise uses the installation's venv when available, or creates a managed one |
|
|
| `managed` | creates a venv under `DATA_DIR`; the Modules screen manages its packages |
|
|
| interpreter or venv path | uses the specified Python environment |
|
|
|
|
Docker uses `managed`. When Fluksio adopts your own venv, the Modules screen
|
|
is read-only; install packages into that environment yourself.
|
|
|
|
Back up the whole data directory and the persistent `SECRET_KEY`. The single
|
|
container keeps both, and its Redis, on the `/data` volume. The operator stack
|
|
under `docker/` keeps `.env` and `app-flow-data`, plus `app-redis-data` for
|
|
live state. Compose prefixes volume names with the project name.
|
|
Keep runtime files on persistent storage so rebuilds preserve credentials,
|
|
pairings and subscriptions.
|
|
|
|
## Live state
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `REDIS_HOST` | unset | unset uses memory, or the image's own Redis in Docker; a Redis host enables persistent flow state |
|
|
| `REDIS_PORT` | `6379` | Redis port |
|
|
|
|
Live state includes message values, node memory and queued work. In-memory
|
|
state is lost on restart. The Docker image starts a Redis of its own with
|
|
append-only persistence in `/data/redis` whenever `REDIS_HOST` is unset. Set
|
|
`REDIS_HOST` to use an external one instead, as the operator stack does with
|
|
its `app-redis-data` volume. The default sync policy may lose about a second of
|
|
recent writes after a power cut.
|
|
|
|
## Accounts and access
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `SECRET_KEY` | generated | signs sessions and encrypts the secrets store; the CLI persists it and `make init` writes it to `.env` |
|
|
| `ACCESS_TOKEN_EXPIRE_MINUTES` | `11520` (8 days) | session-token lifetime |
|
|
| `FIRST_SUPERUSER` | unset | initial administrator email; the CLI creates an account if needed |
|
|
| `FIRST_SUPERUSER_PASSWORD` | unset | initial password; the CLI generates and prints one if needed |
|
|
| `FRONTEND_HOST` | `http://localhost:5173`, `http://localhost:8000` in Docker | the address people and devices reach the instance at; panel links, mails and the OAuth issuer use it |
|
|
| `BACKEND_CORS_ORIGINS` | `[]` | allowed origins, as comma-separated URLs or a JSON list; `FRONTEND_HOST` is also allowed |
|
|
| `FORWARDED_ALLOW_IPS` | `127.0.0.1` | trusted proxies; Docker adds `172.16.0.0/12` |
|
|
| `SIGNUP_ENABLED` | `false` | allows account creation from the login page; otherwise use **Admin → Users** |
|
|
|
|
!!! warning "Keep your secret key"
|
|
|
|
Changing `SECRET_KEY` signs everyone out and makes existing encrypted
|
|
secrets unreadable. Back it up with your data.
|
|
|
|
## Interface address
|
|
|
|
The image serves the interface and the API on one origin, so the interface
|
|
calls the API as a path on whatever address served it. One build works behind
|
|
any port, hostname or reverse proxy. `FRONTEND_DIST` points at the built
|
|
interface and is set by the image. A checkout leaves it empty and runs the Vite
|
|
dev server, which needs its origin in `BACKEND_CORS_ORIGINS`.
|
|
|
|
## Image options
|
|
|
|
The `EXTRAS` build argument picks the optional features built into the image,
|
|
comma separated. Empty, the default, is the minimal engine.
|
|
|
|
| Extra | Adds |
|
|
|---|---|
|
|
| `mqtt` | the MQTT node |
|
|
| `influx` | the InfluxDB node |
|
|
| `email` | outbound mail, such as password resets |
|
|
| `sentry` | error reporting to `SENTRY_DSN` |
|
|
| `ml` | numpy, for the `mlp` node |
|
|
| `server` | all of the above |
|
|
|
|
A node whose extra is missing says so on the canvas. The same names work as
|
|
pip extras, such as `pip install 'fluksio[mqtt]'`. Connector packages built
|
|
with `make connectors` are installed into the image too.
|
|
|
|
## Environment
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `ENVIRONMENT` | `local` | `local`, `staging` or `production` |
|
|
| `PRIVATE_API_ENABLED` | `false` | test endpoints; also requires `ENVIRONMENT=local` |
|
|
| `API_DOCS_ENABLED` | `false` | opens `/docs`, `/redoc` and `/api/v1/openapi.json` to network callers outside production |
|
|
| `TZ` | host timezone; Docker defaults to `UTC` | timezone used by cron schedules |
|
|
|
|
API documentation is available to loopback callers outside production.
|
|
`API_DOCS_ENABLED=true` opens it to other callers, as the Docker development
|
|
stack does. Production disables it entirely. Staging and production also
|
|
refuse `changethis` values for the session key or administrator password.
|
|
|
|
Set `TZ` in `.env` for schedules that should follow local time, such as
|
|
`Europe/Berlin`. `make up`, `make update` and `make dev` warn when `.env` lacks
|
|
it, or any other key `.env.example` lists that Compose would otherwise default.
|
|
|
|
## Engine and resources
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `FLOW_MAX_WORKERS` | `4` | local node-code subprocesses |
|
|
| `FLOW_MAX_CASCADES` | `4` | concurrent live message cascades |
|
|
| `FLOW_MAX_RUNS` | `4` | concurrent batch runs |
|
|
| `FLOW_NODE_TIMEOUT` | `0` | maximum node silence in seconds, unless the node overrides it; `0` disables |
|
|
| `FLOW_CPUS` | `0` | CPU budget; `0` detects cores and reserves two for the engine |
|
|
| `FLOW_GPUS` | `0` | local GPU count; `0` counts the engine's `CUDA_VISIBLE_DEVICES`, or NVIDIA device nodes without it |
|
|
| `FLOW_RAM` | `0` | memory budget in MB; `0` detects host/container memory and reserves 1 GiB |
|
|
| `FLOW_WORKER_MAX_RSS_MB` | `0` | retire a local worker above this memory use after a call; `0` disables |
|
|
| `FLOW_NODE_STOP_TIMEOUT` | `5` | seconds a node gets to stop during a rebuild before the rebuild moves on |
|
|
| `FLOW_PIPELINE_REBUILD_WAIT` | `15` | seconds a request waits for a running whole-pipeline rebuild before it gets a 503 |
|
|
| `FLOW_REBUILD_WAIT` | `5` | the same for a single flow's rebuild, such as a deploy |
|
|
| `FLOW_RESTART_BACKOFF` | `[1,5,30,60]` | seconds between restarts of a crashed node loop; the last step repeats |
|
|
| `FLOW_CANCEL_GRACE` | `5` | seconds a cancelled node loop gets to stop before it is abandoned |
|
|
| `OBS_RETENTION_DAYS` | `30` | retention for metrics, events and run records |
|
|
|
|
The three concurrency limits must be at least `1`. They also have CLI flags:
|
|
`--max-workers`, `--max-cascades` and `--max-runs`. When `FLOW_MAX_WORKERS` or
|
|
`FLOW_MAX_CASCADES` is more than four times the machine's core count, the
|
|
engine logs a warning at startup and uses the value anyway. Resource flags are
|
|
`--gpus`, `--ram` and `--worker-max-rss`.
|
|
|
|
Nodes with declared resources are scheduled against the available budget.
|
|
Undeclared nodes use the shared pool and receive thread caps. Attached workers
|
|
report their own inventory; the engine's resource settings describe only its
|
|
own machine. See [Remote workers](../code/workers.md) and the **Workers** screen.
|
|
|
|
When the engine runs with `CUDA_VISIBLE_DEVICES`, a node's card is named the
|
|
way that list names it, so under `2,3` the first card is device `2`. UUID and
|
|
MIG entries are passed on as written.
|
|
|
|
## Artifacts
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `MAX_ARTIFACT_BYTES` | `2147483648` (2 GiB) | upload limit; `0` removes it |
|
|
| `ARTIFACT_GC_INTERVAL_S` | `3600` | cleanup interval; `0` disables |
|
|
| `ARTIFACT_GC_GRACE_S` | `3600` | minimum age before cleanup |
|
|
| `ARTIFACT_VOLATILE_DIR` | auto-detected | where local workers write live frames, under `/dev/shm` or the temporary directory |
|
|
| `ARTIFACT_VOLATILE_BYTES` | `50331648` (48 MiB) | live-media ring size, in the engine's memory and again in that directory; `0` stores volatile saves durably |
|
|
| `MAX_FRAME_BYTES` | `4194304` (4 MiB) | the largest live frame: a volatile upload past it gets 413, a device's frame on its socket is dropped; keep it under 16 MiB, the websocket's limit |
|
|
|
|
Cleanup removes durable artifacts no longer referenced by recorded runs or
|
|
current messages, and pauses while a run is active. Volatile artifacts live in
|
|
a bounded ring in the engine's memory, and the oldest disappear as it fills.
|
|
Recorded run outputs are copied into durable storage. A worker in the engine's
|
|
container writes its frames to `ARTIFACT_VOLATILE_DIR` instead, which the
|
|
engine trims to the same size every few seconds. A frame the engine holds is
|
|
written there too before a node in this container is handed it, so
|
|
`load_artifact` finds it. If workers publish or read frames under Docker,
|
|
raise `shm_size` with the ring size. The ring never holds more than its size
|
|
plus one frame.
|
|
|
|
## Agents
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `MCP_ENABLED` | `false` | enables `/mcp` and OAuth client registration; `.env.example` sets it to `true` |
|
|
| `MCP_TOKEN_EXPIRE_MINUTES` | `60` | access-token lifetime |
|
|
| `MCP_REFRESH_EXPIRE_DAYS` | `30` | refresh-token lifetime |
|
|
| `OAUTH_CODE_EXPIRE_SECONDS` | `60` | authorization-code lifetime |
|
|
|
|
See [Agents over MCP](../code/agents.md).
|
|
|
|
## Live view
|
|
|
|
A camera's [live view](../interface/dashboards.md#live-view) connects the
|
|
browser and the device directly. These name the servers that help when the two
|
|
are on different networks. Unset, only direct connections are tried, and no
|
|
third party learns who watches what.
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `STUN_URLS` | unset | STUN servers, comma separated, such as `stun:turn.example.com:3478` |
|
|
| `TURN_URLS` | unset | TURN relays, comma separated, such as `turn:turn.example.com:3478,turns:turn.example.com:5349?transport=tcp` |
|
|
| `TURN_SECRET` | unset | the secret shared with the TURN server (coturn's `static-auth-secret`); TURN is used only with it |
|
|
|
|
Each viewer and device gets TURN credentials of its own per live view, good for
|
|
an hour, which coturn checks against the shared secret. A view relayed through
|
|
TURN therefore ends after an hour. coturn also answers STUN, so one server
|
|
covers both.
|
|
|
|
## Mail and monitoring
|
|
|
|
Password-reset mail requires `SMTP_HOST` and `EMAILS_FROM_EMAIL`.
|
|
|
|
| Variable | Default |
|
|
|---|---|
|
|
| `SMTP_HOST` | unset |
|
|
| `SMTP_PORT` | `587` |
|
|
| `SMTP_USER` / `SMTP_PASSWORD` | unset |
|
|
| `SMTP_TLS` / `SMTP_SSL` | `true` / `false` |
|
|
| `EMAILS_FROM_EMAIL` | unset |
|
|
| `EMAILS_FROM_NAME` | `PROJECT_NAME`, normally `Fluksio` |
|
|
| `EMAIL_RESET_TOKEN_EXPIRE_HOURS` | `48` |
|
|
| `SENTRY_DSN` | unset; enables error reporting when configured |
|
|
|
|
`GET /api/v1/utils/health/` checks the event loop and state backend. Docker
|
|
uses it as a health check; `make up` also starts the autoheal service to restart
|
|
an unhealthy backend.
|