stroblme
c24c4b40ca
The alert manager kept its cooldowns and its ceiling per hour in memory, so a node that stayed down was announced again by every restart. Each delivered alert is now a row in the new alert_sent table (additive migration), loaded for the last seven days when the manager starts and pruned as new ones come in. A ledger that fails never stops an alert. The manager's clock is wall time for the same reason. A node_health "down" alert also waits 60 s (HEALTH_GRACE_S) in a pending task per node; a later event that is not "down" cancels it, as does stopping the manager, so a broker restarting with the stack alerts nobody. The cooldown used to be stamped before the ceiling per hour was checked, so an alert the ceiling held back still spent its cooldown. It is stamped only once the alert is sent now. Claude-Session: https://claude.ai/code/session_01SSi5AvZ7pFmySK8zy3NPMh
4.6 KiB
Secrets, modules and alerts
Store credentials, manage node dependencies and choose where failures go. These screens help keep an instance humming.
Secrets
Open Secrets as a superuser, then add a name and value. The list shows names only; saved values are not displayed again.
Reference a secret from a node setting:
{"$secret": "influx-token"}
Fluksio resolves it when building the node. Credential fields in the editor offer a secret picker. Missing names produce an error on the affected node.
Secrets are encrypted with a key derived from SECRET_KEY and stored outside
the flow repository. Using references keeps credentials out of saved flows.
!!! warning "Keep your encryption key"
Changing `SECRET_KEY` makes existing secrets unreadable. Keep the old key
for re-encryption, or re-enter the secrets. Source installations include
`backend/scripts/rotate_secret_key.py` for rotating the secrets store.
Modules
Modules shows the Python environment used by node code, its installed packages and Python version. There are two setups:
- Managed environment: Docker uses a separate node environment. A
superuser edits Requirements and presses Apply. The list is resolved
with
uv pip compileand synced withuv pip sync, so each package brings its dependencies and deleting a requirement removes it. The manifest is versioned with the flows, and successful installs refresh the workers. - Adopted environment: a fresh pip installation inside your own virtual environment can use that environment for nodes too. Modules is read-only. Install dependencies there with pip or uv, then sync or restart Fluksio.
Set NODE_VENV=managed to request a separate environment. Installation
failures show the package manager's output on the page.
For an existing scientific environment on another machine, attach a worker
with fluksio-worker --python /path/to/venv/bin/python. See
Remote workers.
Alerts
Create a channel, choose events in a rule, and use Test to check delivery. Enable alerts for the rules to send. Channels and rules belong to superusers, whose Alerts page holds them. Any other account sees only the button that subscribes its own browser to a webpush channel.
Channels
| Kind | Destination |
|---|---|
| ntfy | server and topic; optional token |
| SMTP | recipient address, using the instance's mail settings |
| Webhook | URL that receives a POST |
| Dashboard | declared message read by a Notification widget |
| webpush | browsers subscribed from the Alerts page |
Channel settings may use {"$secret": "name"} references. Browser push also
needs notification permission and a secure browser context.
Rules
A rule selects events and destination channels. Leaving all events unchecked matches every supported alert event, including future additions.
| Event | Trigger |
|---|---|
node_error |
a node fails |
node_health |
a connection goes down |
flow_quarantined |
repeated crashes stop a flow's background work |
task_crashed |
a background task crashes |
engine_degraded |
engine health degrades |
cascade_dropped |
the engine drops work |
queue_unavailable |
the work queue becomes unreachable |
run_finished |
a batch run ends in error |
Repeated faults are deduplicated with a 15-minute default cooldown. The
engine remembers what it sent in its database, so a restart does not start
every cooldown over, and the ten-per-hour cap holds across one too. Cooldowns
longer than seven days are not kept. A node_health alert waits until the
connection has been down for a minute, so a broker that restarts with the
stack or a device that reconnects alerts nobody. Flapping connections are
suppressed. A quiet channel may be cooling down; Test helps distinguish
that from a broken configuration.
Admin: users and clients
Superusers use Admin to add, edit and remove accounts, and manage OAuth clients for agents.
Accounts admitted from a portal are marked Portal. They sign in through the portal, have no local password and are not superusers. To block local access, edit the account and clear Is active?. See Accounts and the portal.
Settings
Change your name, email, password and appearance under Settings.
Remote access manages enrollment and admitted users; see Accounts and the portal.
A superuser can configure Assistant: choose a provider, store its key, fetch models and select one. The encrypted key is shared for assistant access on this instance. The chat appears after configuration; see The assistant.