Docs / docs (push) Successful in 30s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m7s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m54s
pre-commit / pre-commit (push) Failing after 4m24s
Test Backend / test-backend (push) Successful in 3m8s
Compose Smoke Test / test-compose (push) Successful in 40s
Playwright Tests / merge-reports (push) Successful in 1m33s
A port may now declare `image`, `audio` or `video`. Each is the artifact
reference the engine already had, narrowed by the `media_type` on it, so a
speech recogniser declares what it eats rather than taking any bytes at all and
finding out. Bytes still never travel as a message and nothing on the wire
stops being JSON: a camera publishes one reference per frame, a microphone one
per chunk, and a reference may carry a `meta` dict nothing here interprets.
Streaming media is therefore an ordinary streaming port — with one change to
what that means. An emission used to journal an item with no payload, so
downstream read whatever was current when the item was claimed; a consumer
slower than its producer saw only the newest chunk and the ones between were
lost. That is right for a training curve and wrong for a second of speech, so
an emission now journals a `kind="emission"` item carrying its values, and the
executor hands them to the nodes reading that message instead of writing them
to state again. The value in state stays the latest, which is what everything
else reads, and the wave is filtered by what actually changed rather than
walking everything reachable. No queue serialization change — the existing
`outputs` field carries it.
Continuous media makes the store's missing GC a real problem, so this closes
it: `sweep_artifacts` runs hourly, keeps every digest a `run_artifact` row
records or a live message holds, spares anything written in the last hour, and
stands aside entirely while a run is in flight, since a node may store a
checkpoint long before it returns the reference to it. That also collects the
orphans a deleted flow has always left behind. `ARTIFACT_GC_INTERVAL_S=0` turns
it off.
Around the edges: `GET /artifacts/{digest}` serves the media type the caller
passes and answers ranged requests, so a browser plays a clip rather than
downloading it; `PUT` spools to disk instead of holding the whole body in
memory, as does `save_artifact` given a path; a Media widget draws whatever its
message points at, and a wall panel may fetch the bytes its own tiles are
showing and nothing else; and a connector gets `save_artifact`, for a device
whose readings are bytes.
What this cannot do is live video: a frame every second or two is a glance, and
the honest answer above that is the camera's own stream, which the widget takes
as a URL and the browser plays from source.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
343 lines
14 KiB
Python
343 lines
14 KiB
Python
import uuid
|
|
from collections.abc import Generator
|
|
from typing import Annotated, Any
|
|
from urllib.parse import unquote
|
|
|
|
import jwt
|
|
from fastapi import Depends, HTTPException, Request, status
|
|
from fastapi.security import OAuth2PasswordBearer
|
|
from jwt.exceptions import InvalidTokenError
|
|
from pydantic import ValidationError
|
|
from sqlmodel import Session, select
|
|
|
|
from fluksio.cloud import config as cloud_config
|
|
from fluksio.core import security
|
|
from fluksio.core.config import settings
|
|
from fluksio.core.db import engine
|
|
from fluksio.flow import panels
|
|
from fluksio.flow.artifacts import is_reference
|
|
from fluksio.flow.controller import FlowController
|
|
from fluksio.flow.dashboards import DashboardStore
|
|
from fluksio.flow.workers import PythonWorkerPool
|
|
from fluksio.models import TokenPayload, User
|
|
|
|
reusable_oauth2 = OAuth2PasswordBearer(
|
|
tokenUrl=f"{settings.API_V1_STR}/login/access-token"
|
|
)
|
|
|
|
|
|
def get_db() -> Generator[Session, None, None]:
|
|
with Session(engine) as session:
|
|
yield session
|
|
|
|
|
|
SessionDep = Annotated[Session, Depends(get_db)]
|
|
TokenDep = Annotated[str, Depends(reusable_oauth2)]
|
|
|
|
|
|
#: What a `draft` query parameter has to say to mean "no". Pydantic's own false
|
|
#: literals; anything else is refused rather than guessed at.
|
|
_NOT_DRAFT = {"false", "0", "off", "f", "n", "no", ""}
|
|
|
|
#: The one route under a message name a panel reads.
|
|
_HISTORY = "/history"
|
|
|
|
|
|
def _panel_for(payload: dict[str, Any]) -> panels.PanelDef:
|
|
"""The panel a credential names, if that credential still stands.
|
|
|
|
Two ways it stops standing. The panel was deleted, which revokes every
|
|
credential ever minted for it — read from disk on each call, so it takes
|
|
effect at once. Or the panel's nonce moved on, which revokes exactly one
|
|
screen's and leaves the panel, its dashboards and their arrangement alone.
|
|
|
|
The nonce is only held against a credential this installation signed. One
|
|
the portal minted for a remote screen carries none — the portal names the
|
|
panel and nothing else — and is revoked at the hub instead.
|
|
"""
|
|
panel = panels.find(str(payload.get("panel", "")))
|
|
if panel is None:
|
|
raise InvalidTokenError("This panel no longer exists")
|
|
if (
|
|
payload.get("aud") == security.PANEL_AUDIENCE
|
|
and int(payload.get("pnc") or 0) != panel.nonce
|
|
):
|
|
raise InvalidTokenError("This panel was paired with another device")
|
|
return panel
|
|
|
|
|
|
def _panel_messages(panel_id: str, request: Request) -> set[str]:
|
|
"""Every message this panel's widgets read or write.
|
|
|
|
The same walk that bounds its socket, so the two surfaces a screen has
|
|
agree on what it is entitled to. No store means nothing resolves, which is
|
|
the answer to give when the answer cannot be worked out.
|
|
"""
|
|
store: DashboardStore | None = getattr(request.app.state, "dashboard_store", None)
|
|
if store is None:
|
|
return set()
|
|
return panels.messages_for(panel_id, store)
|
|
|
|
|
|
def _panel_digests(panel_id: str, request: Request) -> set[str]:
|
|
"""The artifacts this panel's messages are pointing at right now.
|
|
|
|
A media tile fetches the bytes its message names, so the messages already
|
|
bounding the panel bound this too — one step further along, through
|
|
whatever those messages currently hold.
|
|
"""
|
|
controller: FlowController | None = getattr(
|
|
request.app.state, "flow_controller", None
|
|
)
|
|
if controller is None:
|
|
return set()
|
|
names = _panel_messages(panel_id, request)
|
|
if not names:
|
|
return set()
|
|
found: set[str] = set()
|
|
for value in controller.state.get_present(sorted(names)).values():
|
|
if is_reference(value):
|
|
found.add(str(value["digest"]))
|
|
return found
|
|
|
|
|
|
def _panel_may(payload: dict[str, Any], request: Request) -> None:
|
|
"""Refuse anything a wall panel has no business asking for.
|
|
|
|
A panel credential names the account that approved the pairing, so without
|
|
this it would be that person's session hanging on a wall. What a panel
|
|
genuinely needs is small and worth writing out: the dashboards it was
|
|
assigned, its own definition, the messages its own widgets bind to, and the
|
|
bytes those messages currently point at, for a tile drawing a camera frame.
|
|
|
|
Publishing is in the list because a panel cannot be strictly read-only — a
|
|
control on a panel is the point of putting one there, and a querying chart
|
|
asks for its window by publishing a request, which is why that request
|
|
counts as one of its widget's messages. Bounded to those, though: a screen
|
|
on a wall has no business reaching a message no tile on it draws, and the
|
|
catalogue behind ``GET /messages/`` is the whole namespace at once.
|
|
|
|
# ponytail: the panel file and the published dashboards are re-read per
|
|
# request. Cache them behind the store's version if this shows up in a
|
|
# profile.
|
|
"""
|
|
panel = _panel_for(payload)
|
|
|
|
api = settings.API_V1_STR
|
|
path = request.url.path
|
|
method = request.method
|
|
|
|
allowed = False
|
|
if method == "GET" and path.startswith(f"{api}/dashboards/"):
|
|
name = path[len(f"{api}/dashboards/") :]
|
|
# One segment only: a dashboard it shows, published — never a draft,
|
|
# and never a sibling route like `/publish`. The flag is read the way
|
|
# the endpoint reads it rather than by presence, because the generated
|
|
# client spells the default out as `?draft=false` — and it is read
|
|
# fail-closed, so a spelling neither side recognises is a draft.
|
|
draft = request.query_params.get("draft")
|
|
published = draft is None or draft.lower() in _NOT_DRAFT
|
|
allowed = "/" not in name and name in panel.dashboards and published
|
|
elif method == "GET" and path == f"{api}/panels/{panel.id}":
|
|
allowed = True
|
|
elif path.startswith(f"{api}/messages/"):
|
|
# The two a widget speaks: a chart reads a series, a control puts a
|
|
# value in. Percent-decoded, because a message name is a path segment
|
|
# here and the client encodes it as one.
|
|
rest = unquote(path[len(f"{api}/messages/") :])
|
|
if method == "GET" and rest.endswith(_HISTORY):
|
|
name = rest[: -len(_HISTORY)]
|
|
elif method == "POST":
|
|
name = rest
|
|
else:
|
|
name = ""
|
|
allowed = bool(name) and name in _panel_messages(panel.id, request)
|
|
elif method == "GET" and path.startswith(f"{api}/artifacts/"):
|
|
# The bytes behind a media message a tile on this panel is drawing.
|
|
# Scoped to what those messages hold *now*, which is exactly what a
|
|
# live widget asks for — a screen has no business reading an artifact
|
|
# off an old run because it happens to know the digest.
|
|
digest = unquote(path[len(f"{api}/artifacts/") :])
|
|
allowed = "/" not in digest and digest in _panel_digests(panel.id, request)
|
|
|
|
if not allowed:
|
|
raise HTTPException(
|
|
status_code=status.HTTP_403_FORBIDDEN,
|
|
detail="A panel credential cannot do that",
|
|
)
|
|
|
|
|
|
def decode_token(token: str, request: Request | None = None) -> dict[str, Any]:
|
|
"""Read a bearer token, whichever channel issued it.
|
|
|
|
A browser session is signed with the app's own secret; an agent's token is
|
|
signed with the OAuth keypair, so that set can be revoked on its own and
|
|
the public half published. Both name a user, and both grant that user's
|
|
rights — the difference is only in who is holding it, which the MCP
|
|
endpoint checks separately.
|
|
|
|
A paired wall panel's token is signed with the app's secret too, and is
|
|
told apart from a session by its audience: the session decode above
|
|
refuses it outright, so the only door it fits is the one ``_panel_may``
|
|
guards.
|
|
|
|
The last branch is the seam a hosted deployment widens: a portal this
|
|
installation was enrolled with signs tokens with a key pinned at
|
|
enrolment, and they name the portal account holding them, which resolves
|
|
to whichever local account was mapped to it — the superuser who enrolled,
|
|
or a remote user one of them admitted since. With no enrolment the branch
|
|
raises immediately, so an offline installation pays nothing for the
|
|
possibility. A screen that paired through that portal
|
|
arrives there too, naming a panel — which is why the gate below is applied
|
|
to whichever branch produced the claims rather than to one of them: where a
|
|
panel credential was minted is not what decides what it may read.
|
|
"""
|
|
try:
|
|
session: dict[str, Any] = jwt.decode(
|
|
token, settings.SECRET_KEY, algorithms=[security.ALGORITHM]
|
|
)
|
|
return session
|
|
except InvalidTokenError:
|
|
pass
|
|
try:
|
|
panel_token = security.decode_panel_token(token)
|
|
except InvalidTokenError:
|
|
pass
|
|
else:
|
|
if not panel_token.get("panel"):
|
|
raise InvalidTokenError("a panel token must name its panel")
|
|
return _gate_panel(panel_token, request)
|
|
try:
|
|
return security.decode_oauth_token(token)
|
|
except InvalidTokenError:
|
|
return _gate_panel(cloud_config.decode_portal_token(token), request)
|
|
|
|
|
|
def _gate_panel(payload: dict[str, Any], request: Request | None) -> dict[str, Any]:
|
|
"""Scope a payload that names a panel; pass anything else through.
|
|
|
|
Without a request — from the websocket, which has no route to scope — the
|
|
check is only that the credential still stands, which is what makes
|
|
deleting a panel, or bumping its nonce, revoke one.
|
|
"""
|
|
if not payload.get("panel"):
|
|
return payload
|
|
if request is not None:
|
|
_panel_may(payload, request)
|
|
else:
|
|
_panel_for(payload)
|
|
return payload
|
|
|
|
|
|
def _user_for(session: Session, token_data: TokenPayload) -> User | None:
|
|
"""The local account a payload names, by id or by portal identity.
|
|
|
|
A token the portal minted names a person on the portal, not a user here, so
|
|
the mapping a superuser made when they admitted them is what turns one into
|
|
the other. No mapping, no user — the caller answers that the same way it
|
|
answers a token naming a deleted account, which is what makes deleting the
|
|
local user the whole of the revocation.
|
|
"""
|
|
if token_data.portal_sub:
|
|
return session.exec(
|
|
select(User).where(User.portal_sub == token_data.portal_sub)
|
|
).first()
|
|
if not token_data.sub:
|
|
return None
|
|
try:
|
|
# The subject is a string on the wire. Postgres cast it on the way in;
|
|
# nothing else does, so parse it here rather than hand a driver a
|
|
# string where it wants a UUID.
|
|
user_id = uuid.UUID(token_data.sub)
|
|
except ValueError:
|
|
return None
|
|
return session.get(User, user_id)
|
|
|
|
|
|
def user_from_token(
|
|
session: Session, token: str, request: Request | None = None
|
|
) -> User | None:
|
|
"""Resolve a bearer token to its user, or None if it does not hold up.
|
|
|
|
Shared with the websocket, which cannot use the HTTP security scheme, and
|
|
with the artifact endpoint, which accepts a worker's credential as well as
|
|
a person's. Pass the request wherever there is one: a panel's credential is
|
|
scoped by route, and without it the scope check cannot run.
|
|
"""
|
|
try:
|
|
token_data = TokenPayload(**decode_token(token, request))
|
|
except (InvalidTokenError, ValidationError):
|
|
return None
|
|
user = _user_for(session, token_data)
|
|
if user is None or not user.is_active:
|
|
return None
|
|
return user
|
|
|
|
|
|
def get_current_user(request: Request, session: SessionDep, token: TokenDep) -> User:
|
|
"""Resolve the bearer token to its user.
|
|
|
|
Every failure here is an authentication failure, so all of them answer 401:
|
|
a token naming a user who no longer exists is a session to log in again,
|
|
not a missing resource to report. A panel credential reaching past what it
|
|
is allowed is the one exception — that is a 403 raised inside
|
|
``decode_token``, and it travels through here untouched.
|
|
"""
|
|
try:
|
|
token_data = TokenPayload(**decode_token(token, request))
|
|
except (InvalidTokenError, ValidationError):
|
|
raise HTTPException(
|
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
|
detail="Could not validate credentials",
|
|
)
|
|
user = _user_for(session, token_data)
|
|
if user is None or not user.is_active:
|
|
raise HTTPException(
|
|
status_code=status.HTTP_401_UNAUTHORIZED,
|
|
detail="This session is no longer valid — please log in again",
|
|
)
|
|
return user
|
|
|
|
|
|
CurrentUser = Annotated[User, Depends(get_current_user)]
|
|
|
|
|
|
def get_flow_controller(request: Request) -> FlowController:
|
|
controller: FlowController | None = getattr(
|
|
request.app.state, "flow_controller", None
|
|
)
|
|
if controller is None:
|
|
raise HTTPException(status_code=503, detail="The flow engine is not running")
|
|
return controller
|
|
|
|
|
|
FlowControllerDep = Annotated[FlowController, Depends(get_flow_controller)]
|
|
|
|
|
|
def get_dashboard_store(request: Request) -> DashboardStore:
|
|
store: DashboardStore | None = getattr(request.app.state, "dashboard_store", None)
|
|
if store is None:
|
|
raise HTTPException(status_code=503, detail="The flow engine is not running")
|
|
return store
|
|
|
|
|
|
DashboardStoreDep = Annotated[DashboardStore, Depends(get_dashboard_store)]
|
|
|
|
|
|
def get_worker_pool(request: Request) -> PythonWorkerPool:
|
|
pool: PythonWorkerPool | None = getattr(request.app.state, "worker_pool", None)
|
|
if pool is None:
|
|
raise HTTPException(status_code=503, detail="The flow engine is not running")
|
|
return pool
|
|
|
|
|
|
WorkerPoolDep = Annotated[PythonWorkerPool, Depends(get_worker_pool)]
|
|
|
|
|
|
def get_current_active_superuser(current_user: CurrentUser) -> User:
|
|
if not current_user.is_superuser:
|
|
raise HTTPException(
|
|
status_code=403, detail="The user doesn't have enough privileges"
|
|
)
|
|
return current_user
|