The alert manager kept its cooldowns and its ceiling per hour in memory, so a node that stayed down was announced again by every restart. Each delivered alert is now a row in the new alert_sent table (additive migration), loaded for the last seven days when the manager starts and pruned as new ones come in. A ledger that fails never stops an alert. The manager's clock is wall time for the same reason. A node_health "down" alert also waits 60 s (HEALTH_GRACE_S) in a pending task per node; a later event that is not "down" cancels it, as does stopping the manager, so a broker restarting with the stack alerts nobody. The cooldown used to be stamped before the ceiling per hour was checked, so an alert the ceiling held back still spent its cooldown. It is stamped only once the alert is sent now.
Fluksio
Turn Python functions into connected nodes, run experiments, and keep their metrics and artifacts together. Start an engine with Python 3.12 or newer on Linux or macOS:
pip install fluksio
fluksio serve
The first start prints an admin password and opens a terminal dashboard. For a browser dashboard, connect the instance to the optional portal.
For data science
Save this small experiment as study.py. No dataset or GPU needed:
import math
from fluksio import Flow, Port, node
@node(
requires=[Port("lr", "float")],
provides=[Port("loss", "float", stream=True), Port("final_loss", "float")],
)
def fit(lr, epochs=25):
for epoch in range(epochs):
loss = math.exp(-lr * (epoch + 1))
yield {"loss": loss}
return {"final_loss": loss}
train = Flow(
"train",
nodes=[fit],
inputs=[Port("lr", "float", initial=0.01)],
outputs=["final_loss"],
)
Matching message names connect the nodes and determine their order. After syncing the project, start a run:
fluksio sync study
fluksio run train --lr 0.05 --wait
Each yield records a point on the loss curve. Replace the toy calculation with your own training code when you're ready. For artifacts and comparisons, try the data science guide.
More to explore
- Flows: typed messages between nodes, wired by name, edited on a canvas or declared in code. Every change is a commit in a git repository you own.
- Runs: an experiment and a CI-style job are the same entity. Parameters, seed, result, per-node timings, artifacts and the commit it ran at.
- Dashboards: charts and controls bound to the same messages the flows carry, with no separate metrics pipeline.
- Remote workers:
pip install fluksio-workeron the GPU box; it dials out over one websocket, so nothing there has to be reachable.
For sensors, devices and live automation, try the facility automation guide.
License
Copyright (C) 2026 Melvin Strobl - GNU Affero General Public License v3.0 or later. Running a modified version over a network obliges you to offer its users the corresponding source (AGPL §13).