Files
app/backend/fluksio/api/routes/panels.py
T
stroblmeandClaude Opus 5 57eace2226 Bound what the API accepts, and close the holes the audit found
**SQLite is the database, and now says so.** `metric_minute` and every run
table are written with `sqlalchemy.dialects.sqlite.insert(...)
.on_conflict_do_update` and with `max(a, b)`, neither of which another
dialect has — so pointing `DATABASE_URL` at Postgres migrated cleanly,
served, logged in, and then lost every observability flush into the
collector's hold buffer and failed every run. It refuses at startup
instead. (The Postgres in the compose stack is Umami's; the engine's own
database has been a file beside the flows since 2026-08-21.)

**Every integer query parameter is bounded.** The caps were written as
`min(limit, 500)`, which a negative walks straight through — `?limit=-1`
compiles to `LIMIT -1` and SQLite returns the whole table. Ten signatures,
now `Query(ge=…, le=…)`. `hours=0` still means an hour, which
`_window_hours` was already deliberate about.

**Exports are capped at 10 000 runs** and say so with `X-Truncated`. The
filters bounded a sensible request and nothing bounded an unfiltered one,
which read every row into memory before a byte was streamed. `_series`
resolves cached curves in two queries rather than a `Run` lookup and a
`RunMetric` query per restored node — a comparison of twenty runs was
calling that twenty times over.

**`PUT /artifacts` has a size limit** (`MAX_ARTIFACT_BYTES`, 2 GiB, 0 to
disable), checked against `Content-Length` and again against the stream for
a chunked body, and its writes moved off the event loop.

**`/observability/timeseries` takes `since`/`until`**, the same window
`/runs` and `/events` take, capped at 2000 points — `hours=720&bucket_s=60`
was 43 200 of them in one array. It is also what a dragged chart needs to
re-fetch at its own resolution rather than magnifying buckets it has.

**Composite indexes** for the three list screens: `run(flow, created_at)`
and `(status, created_at)`, `flow_run(flow, started_at)`,
`engine_event(type, ts)`. Every index was single-column, so SQLite picked
one and sorted the rest by hand. Verified against a copy of a live database
(250k `flow_run` rows): the planner takes all four.

**Redis clients have socket timeouts.** A Redis that stops answering
without closing the connection hung the caller until the kernel gave up —
including `/utils/health/`, whose job is to notice.

**The panels file is written under one lock.** `save_panels` and
`unpair_panel` are both read-modify-write, and a save that read before an
unpair wrote put the old nonce back — silently un-revoking a screen that
had just been unpaired. The nonce carry-forward was written to make that
impossible; the gap between its read and its write is where it happened.

**Startup releases what it acquired.** Everything past `event_bus.bind`
registers how to close itself and the `finally` walks that list backwards;
a failure part-way through used to reach none of the shutdown steps and
leave the worker pool's subprocesses and every background task behind —
under `--reload`, once per bad edit. `modules.reconcile` moved into the
background: `uv` gets five minutes twice over, the healthcheck allows
eighty seconds, and the autoheal restarted the container before it could
finish installing.

`delete_run` takes SQLite's write lock up front (`core.db.writing`) rather
than upgrading a deferred transaction and losing to whichever flush
committed in between. `modules.sync` is serialised — two applies mutated
one venv at once. The proxied-call and stream dicts are bounded, and a
reused id cancels its predecessor instead of dropping the reference.

Security, found in passing and small enough to fix here:

- **`/secrets/` required only a signed-in user.** The names alone say what
  this installation talks to, and `PUT /{name}` takes any name, so any
  account could overwrite the credential a flow authenticates with.
  Superuser now — which `/search` already assumed and said so.
- **`POST /login/access-token` had no rate limit.** Argon2 is deliberately
  expensive and the route is unauthenticated and runs in the shared
  threadpool. Ten *failed* attempts per address per five minutes; a
  successful sign-in spends nothing.
- **a password reset link worked repeatedly for 48 hours.** The token now
  carries a digest of the password hash it was minted against, so it stops
  verifying once it has set one. No table of spent tokens needed.
- **enrolment accepted `http://`**, sending the claim code and then this
  installation's credential in clear. https, or a local address.
- the rate limiter read `request.client.host`, which behind Traefik is the
  proxy — so every per-address limit was one global bucket and one caller
  could lock out everyone. It reads the forwarded address, and its
  bucket table is capped rather than growing one key per address forever.
- SMTP has a timeout and sends after the response, so an unreachable mail
  host cannot pin a threadpool worker, and a reply's timing no longer says
  whether the address exists.

Test suite: engine-written rows are cleared between modules. A `FlowRun`
left `running` by one module turned up in another's query. Per-test
rollback is not available here — the module-scoped `client` runs the real
lifespan and its collector and run service write through sessions of their
own — so this bounds it where the writes come from. Three consecutive
green runs, orders randomised.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M6hPWS6YEbT1P8LxhhFb2T
2026-08-29 20:40:05 +02:00

490 lines
18 KiB
Python

"""Panels: which dashboards a device shows, and how it gets a credential.
A wall tablet has no keyboard, so it cannot log in. It asks for a code instead,
shows it on the wall, and somebody with an account types that code into the
panels dialog to say which panel the device is. The device polls, collects the
credential the approval minted, and never asks again.
A screen hanging somewhere this installation is not reachable from does the
same thing through the portal, which forwards those two calls down the tunnel
without a session — a device with no credential is the whole point of them —
and mints the credential itself when the approval comes. Which side minted it
changes nothing about what it may do: the scope check is here either way.
The credential is scoped: ``fluksio.api.deps`` lets it reach the dashboards that
panel was assigned, the messages their widgets bind to, and nothing else.
Removing the panel revokes it; so does bumping the panel's nonce, which is how
one screen is re-paired without disturbing what it was showing.
"""
from __future__ import annotations
import secrets
import time
from datetime import timedelta
from typing import Any, cast
import httpx
import redis
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.concurrency import run_in_threadpool
from pydantic import BaseModel, Field
from fluksio.api.deps import CurrentUser, get_current_active_superuser, get_current_user
from fluksio.cloud import config as cloud_config
from fluksio.core import security
from fluksio.core.config import settings
from fluksio.flow.events import event_bus
from fluksio.flow.panels import (
PanelDef,
PanelsConfig,
edit_lock,
find,
read_config,
write_config,
)
from fluksio.flow.state import CONNECT_TIMEOUT_S, SOCKET_TIMEOUT_S
from fluksio.models import Message
#: Gated per route rather than on the router: the two pairing endpoints are the
#: only unauthenticated ones in the app, because a device with no credential is
#: the whole point of them.
router = APIRouter(prefix="/panels", tags=["panels"])
#: Long, because a panel is a screen somebody hung once and left running, with
#: no keyboard to sign in again with. Same reasoning as a remote worker's.
TOKEN_DAYS = 365
#: How long a device's code is worth typing. Long enough to walk to a computer.
PAIR_TTL = 600
#: No I/O/0/1: the code is read off a wall and typed on a keyboard.
CODE_ALPHABET = "ABCDEFGHJKLMNPQRSTUVWXYZ23456789"
CODE_LENGTH = 6
#: Refuse to grow past this. Codes are free to ask for and cost memory.
MAX_PENDING = 50
class _Pending(BaseModel):
"""A device waiting to be told what it is."""
#: Held by the device, never shown.
secret: str
#: Wall-clock rather than monotonic: an entry outlives the process that
#: minted it, and monotonic clocks are not comparable across processes.
expires: float
token: str = ""
panel: str = ""
#: What the request looked like, shown to whoever approves the code so
#: they can tell the screen in the hall from one they were not expecting.
#: Self-reported and worth what that is worth.
device: str = ""
#: Whether it came down the tunnel. A device that reached the portal
#: cannot reach this installation, so its credential has to be minted
#: where it can collect it.
remote: bool = False
#: The entry itself, one key per code, expiring on its own.
_PENDING_KEY = "panels:pair"
#: An index of the live codes, so :data:`MAX_PENDING` means the same thing to
#: every worker. Scored by expiry, which is how it is pruned.
_PENDING_INDEX = "panels:pair:__codes__"
class _PendingStore:
"""The codes waiting for somebody to say what they are.
A device's poll lands on whichever API worker the proxy picked, and a
screen pairing through the portal lands on whichever one holds the tunnel
— so a code minted by one worker has to be findable from the next. Redis
is where this installation already keeps what must outlive a process.
Without one there is one process by definition (the pip install, and the
tests), and a dictionary is the same thing for it.
"""
def __init__(self) -> None:
self._local: dict[str, _Pending] = {}
self._redis: redis.Redis | None = (
redis.Redis(
host=settings.REDIS_HOST,
port=settings.REDIS_PORT,
decode_responses=True,
# A silent Redis must fail this request, not hold it open.
socket_timeout=SOCKET_TIMEOUT_S,
socket_connect_timeout=CONNECT_TIMEOUT_S,
health_check_interval=30,
)
if settings.REDIS_HOST
else None
)
def _key(self, code: str) -> str:
return f"{_PENDING_KEY}:{code}"
def count(self) -> int:
"""How many are waiting, the expired ones dropped first."""
now = time.time()
if self._redis is None:
for code in [c for c, e in self._local.items() if e.expires < now]:
del self._local[code]
return len(self._local)
self._redis.zremrangebyscore(_PENDING_INDEX, "-inf", now)
return int(cast(int, self._redis.zcard(_PENDING_INDEX)))
def add(self, code: str, entry: _Pending) -> bool:
"""Claim this code, or say that somebody already holds it."""
if self._redis is None:
if code in self._local:
return False
self._local[code] = entry
return True
if not self._redis.set(
self._key(code), entry.model_dump_json(), ex=PAIR_TTL, nx=True
):
return False
self._redis.zadd(_PENDING_INDEX, {code: entry.expires})
return True
def get(self, code: str) -> _Pending | None:
if self._redis is None:
entry = self._local.get(code)
else:
raw = cast("str | None", self._redis.get(self._key(code)))
entry = _Pending.model_validate_json(raw) if raw else None
return entry if entry is not None and entry.expires >= time.time() else None
def save(self, code: str, entry: _Pending) -> None:
"""Write an approval back, without moving what it expires at."""
if self._redis is None:
self._local[code] = entry
return
self._redis.set(
self._key(code),
entry.model_dump_json(),
ex=max(1, int(entry.expires - time.time())),
)
def drop(self, code: str) -> None:
if self._redis is None:
self._local.pop(code, None)
return
self._redis.delete(self._key(code))
self._redis.zrem(_PENDING_INDEX, code)
_pending = _PendingStore()
class PairStarted(BaseModel):
"""What the device puts on the wall, and what it polls with."""
code: str
#: Held by the device, never shown. Without it the code alone would let
#: anyone who reads it off the wall collect the credential first.
secret: str
expires_in: int = PAIR_TTL
class PairStatus(BaseModel):
"""Nothing yet, or the credential and the panel it is for."""
access_token: str | None = None
panel: str | None = None
class PairRequest(BaseModel):
code: str
class PendingDevice(BaseModel):
"""Who is asking, as far as the request itself says."""
device: str
remote: bool = False
def _describe(request: Request) -> str:
"""A line naming the device behind a pairing request.
# ponytail: the raw user agent, trimmed. Parse it into "iPad · Safari" if
# it reads badly on the approval screen.
"""
agent = request.headers.get("user-agent", "").strip()[:120]
forwarded = request.headers.get("x-forwarded-for", "").split(",")[0].strip()
# Proxied requests are replayed into this process over an ASGI transport,
# which reports every caller as localhost; the forwarded address is the
# only true one there, and behind the local reverse proxy it is too.
address = forwarded or (request.client.host if request.client else "")
return " · ".join(part for part in (agent or "Unknown device", address) if part)
def _mint_at_hub(panel_id: str) -> str:
"""Ask the portal for this panel's credential.
A device that arrived through the portal cannot reach this installation, so
a token this installation signed would be one it could never present: the
portal verifies what crosses its tunnel, and it verifies against its own
key. It mints, we say which panel — and the scope check here decides the
rest, on this call and on every later one.
"""
config = cloud_config.load()
if config is None:
raise HTTPException(
status_code=409,
detail="That device came through a portal this installation is no "
"longer enrolled with",
)
try:
response = httpx.post(
f"{config.portal_url.rstrip('/')}/api/v1/panel-tokens/",
headers={"Authorization": f"Bearer {config.token}"},
json={"panel": panel_id},
timeout=15.0,
)
except httpx.HTTPError as exc:
raise HTTPException(
status_code=502, detail=f"Could not reach the portal: {exc}"
) from exc
if response.status_code != 200:
raise HTTPException(
status_code=502,
detail=f"The portal refused to mint a credential ({response.status_code})",
)
token: str = response.json()["access_token"]
return token
class PanelsPublic(BaseModel):
"""The panels, and the address a device should be pointed at.
The address is the server's own, because the browser's origin is not a
reliable answer to it: an admin working through the portal is on the
portal's origin, and this one is for a screen on this network.
An installation enrolled with a portal has a second address, built by the
dialog from what ``/cloud/status`` reports rather than from here — a panel
is not the thing that knows whether remote access is on.
"""
panels: list[PanelDef] = Field(default_factory=list)
#: Whatever this installation was told it is reachable at. The same setting
#: the password-reset links are built from, so an installation that has it
#: wrong has it wrong in both places.
frontend_host: str = ""
def _public(config: PanelsConfig) -> PanelsPublic:
return PanelsPublic(
panels=config.panels, frontend_host=settings.FRONTEND_HOST.rstrip("/")
)
@router.get("/", response_model=PanelsPublic, dependencies=[Depends(get_current_user)])
async def read_panels() -> Any:
"""Every panel, what each one shows, and where to point a device."""
return _public(await run_in_threadpool(read_config))
@router.put(
"/",
response_model=PanelsPublic,
dependencies=[Depends(get_current_active_superuser)],
)
async def save_panels(body: PanelsConfig) -> Any:
"""Replace the panels. Takes effect on the devices' next read.
A panel that disappears here takes its credential with it, so this is also
how a device is unpaired along with its assignment. Re-pairing one screen
and keeping the assignment is ``/{panel_id}/unpair`` below.
"""
seen = set()
for panel in body.panels:
if panel.id in seen:
raise HTTPException(
status_code=422, detail=f"Two panels named {panel.id!r}"
)
seen.add(panel.id)
def _save() -> None:
# Read and write under one lock. The nonce belongs to this
# installation, not to whoever is writing the panels back: a client
# holding an older copy must not be able to undo a revocation by
# saving an arrangement — and reading it in a separate step from
# writing it is exactly how an unpair in between was undone.
with edit_lock:
stored = {p.id: p.nonce for p in read_config().panels}
for panel in body.panels:
panel.nonce = stored.get(panel.id, 0)
write_config(body)
await run_in_threadpool(_save)
# Which dashboards hang on which panel just changed. An empty name says
# that much and no more: every screen listening rescopes and refetches
# what it shows, rather than waiting for whenever it next reads.
event_bus.publish({"type": "dashboard_changed", "dashboard": "", "ts": time.time()})
return _public(body)
@router.post("/pair", response_model=PairStarted)
def start_pairing(request: Request) -> Any:
"""A device asks to be adopted. Unauthenticated, by necessity.
All this hands out is a code that means nothing until somebody with an
account approves it, so the worst an unwelcome caller achieves is an entry
that expires ten minutes later. Reachable from the internet when this
installation is enrolled with a portal, which is what the cap and the
portal's own per-address limits are between.
"""
if _pending.count() >= MAX_PENDING:
raise HTTPException(
status_code=429, detail="Too many devices are waiting to be paired"
)
remote = secrets.compare_digest(
request.headers.get(cloud_config.VIA_HEADER, ""), cloud_config.VIA_PORTAL
)
entry = _Pending(
secret=secrets.token_urlsafe(16),
expires=time.time() + PAIR_TTL,
device=_describe(request),
remote=remote,
)
# Claiming the code is what settles a collision, rather than looking first:
# between the look and the write sits another worker doing the same thing.
for _ in range(10):
code = "".join(secrets.choice(CODE_ALPHABET) for _ in range(CODE_LENGTH))
if _pending.add(code, entry):
break
else:
raise HTTPException(status_code=503, detail="Could not mint a pairing code")
return PairStarted(code=code, secret=entry.secret)
@router.get("/pair/{code}", response_model=PairStatus)
def poll_pairing(code: str, secret: str = "") -> Any:
"""Has anyone claimed this device yet?
Answers the same 404 for a code that never existed, one that expired and
one polled with the wrong secret: a caller reading a code off a wall learns
nothing by guessing.
"""
entry = _pending.get(code)
if entry is None or not secrets.compare_digest(entry.secret, secret):
raise HTTPException(
status_code=404, detail="No pairing is waiting on that code"
)
if not entry.token:
return PairStatus()
# Handed over once. A credential left lying in the store is a second copy
# of it, and the device has the only one it needs.
_pending.drop(code)
return PairStatus(access_token=entry.token, panel=entry.panel)
@router.get(
"/pair/{code}/device",
response_model=PendingDevice,
dependencies=[Depends(get_current_active_superuser)],
)
def pending_device(code: str) -> Any:
"""What is waiting on this code, before anyone says what it is.
Approving a code adopts whatever is holding it, so it is worth seeing that
it looks like the screen you just hung.
"""
entry = _pending.get(code.strip().upper())
if entry is None:
raise HTTPException(status_code=404, detail="No device is waiting on that code")
return PendingDevice(device=entry.device, remote=entry.remote)
@router.post(
"/{panel_id}/pair",
response_model=Message,
dependencies=[Depends(get_current_active_superuser)],
)
def approve_pairing(panel_id: str, body: PairRequest, current_user: CurrentUser) -> Any:
"""Say which panel the device showing this code is.
A credential minted here names the approver, so what the panel does stays
attributable to a person rather than to nobody. One minted by the portal —
for a device that reached this installation only through it — names the
account this installation was enrolled with instead, since that is the one
every portal-borne request already acts as.
"""
panel = find(panel_id)
if panel is None:
raise HTTPException(status_code=404, detail=f"No panel named {panel_id!r}")
code = body.code.strip().upper()
entry = _pending.get(code)
if entry is None:
raise HTTPException(
status_code=404,
detail="No device is waiting on that code — check it again",
)
if entry.remote:
entry.token = _mint_at_hub(panel_id)
else:
entry.token = security.create_panel_token(
panel_id, current_user.id, timedelta(days=TOKEN_DAYS), panel.nonce
)
entry.panel = panel_id
_pending.save(code, entry)
return Message(message=f"Paired {entry.device} with {panel_id}")
@router.post(
"/{panel_id}/unpair",
response_model=Message,
dependencies=[Depends(get_current_active_superuser)],
)
async def unpair_panel(panel_id: str) -> Any:
"""Stop honouring this panel's credential, and keep the panel.
Bumping the nonce refuses the screen hanging there on its next request, so
it goes back to showing a pairing code — while the panel, the dashboards it
was assigned and their arrangement stay exactly as they were. Deleting the
panel is still what throws all three away together.
A credential the portal minted for a remote screen carries no nonce, so
this does not reach it; that one is revoked at the hub.
"""
def _unpair() -> bool:
with edit_lock:
config = read_config()
panel = next((p for p in config.panels if p.id == panel_id), None)
if panel is None:
return False
panel.nonce += 1
write_config(config)
return True
if not await run_in_threadpool(_unpair):
raise HTTPException(status_code=404, detail=f"No panel named {panel_id!r}")
# The screen is still holding a socket. One event and it refetches, which
# is where it meets the 401 that sends it back to the pairing code.
event_bus.publish({"type": "dashboard_changed", "dashboard": "", "ts": time.time()})
return Message(message=f"Unpaired {panel_id}")
@router.get(
"/{panel_id}", response_model=PanelDef, dependencies=[Depends(get_current_user)]
)
async def read_panel(panel_id: str) -> Any:
"""One panel — read by a person, or by the device holding its credential.
Declared after the pairing routes so a device polling ``/panels/pair/...``
is never mistaken for someone reading a panel named ``pair``.
"""
panel = await run_in_threadpool(find, panel_id)
if panel is None:
raise HTTPException(status_code=404, detail=f"No panel named {panel_id!r}")
return panel