Files
app/backend/fluksio/alembic/versions/b9e4d2a7c1f6_alert_sent.py
T
stroblme bfaaa7cd03 Alerts: cooldowns survive a restart, and a connection must stay down a minute before it alerts
The alert manager kept its cooldowns and its ceiling per hour in memory, so a
node that stayed down was announced again by every restart. Each delivered
alert is now a row in the new alert_sent table (additive migration), loaded
for the last seven days when the manager starts and pruned as new ones come
in. A ledger that fails never stops an alert. The manager's clock is wall
time for the same reason.

A node_health "down" alert also waits 60 s (HEALTH_GRACE_S) in a pending task
per node; a later event that is not "down" cancels it, as does stopping the
manager, so a broker restarting with the stack alerts nobody.

The cooldown used to be stamped before the ceiling per hour was checked, so
an alert the ceiling held back still spent its cooldown. It is stamped only
once the alert is sent now.
2026-10-28 12:46:42 +01:00

36 lines
882 B
Python

"""alert_sent
What the alert manager has sent, so a cooldown holds across a restart.
Revision ID: b9e4d2a7c1f6
Revises: a3c7e5d91f42
Create Date: 2026-10-28
"""
import sqlalchemy as sa
import sqlmodel.sql.sqltypes
from alembic import op
# revision identifiers, used by Alembic.
revision = "b9e4d2a7c1f6"
down_revision = "a3c7e5d91f42"
branch_labels = None
depends_on = None
def upgrade():
op.create_table(
"alert_sent",
sa.Column("id", sa.Integer(), nullable=False),
sa.Column("key", sqlmodel.sql.sqltypes.AutoString(length=255), nullable=False),
sa.Column("ts", sa.Float(), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(op.f("ix_alert_sent_ts"), "alert_sent", ["ts"], unique=False)
def downgrade():
op.drop_index(op.f("ix_alert_sent_ts"), table_name="alert_sent")
op.drop_table("alert_sent")