One image for the whole app: engine, API and dashboard on one port

The backend image now builds the SPA and serves it on the API's own origin
(fluksio/spa.py), so an instance is a single container with no proxy, DNS,
port 80 or .env: `docker compose up -d` with the new compose.yml runs it on
localhost:8080. It starts on fluksio serve (persisted secret key, first
admin printed once) and brings its own Redis when REDIS_HOST is unset.

Slimmer: python:3.13-slim, no dev group, uv once, fastapi without the cloud
CLI. Optional features are per-feature extras (mqtt, influx, email, sentry,
ml, server for all) picked with the EXTRAS build argument; numpy moved to
`ml`. ffmpeg left the image for imageio-ffmpeg via the Modules screen.

The operator stack runs the same image with app. and api. routed to it; the
nginx frontend service, the LAN nginx proxy and DOMAIN in the backend are
gone, and the MCP issuer is FRONTEND_HOST.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-16 11:40:02 +02:00
co-authored by Claude Opus 5.5
parent 29aca91978
commit 48aac3855b
39 changed files with 573 additions and 728 deletions
+73 -41
View File
@@ -1,47 +1,44 @@
FROM python:3.13
# The whole instance in one image: engine, API and dashboard on port 8000.
#
# docker build -f backend/Dockerfile -t fluksio .
# docker run -d -p 8080:8000 -v fluksio-data:/data fluksio
#
# The build context is the app repository root. EXTRAS picks the optional
# features (mqtt, influx, email, sentry, ml, or server for all of them); the
# default is the minimal engine.
ENV PYTHONUNBUFFERED=1
# ── Dashboard ────────────────────────────────────────────────────
FROM oven/bun:1 AS frontend
# Install uv
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#installing-uv
COPY --from=ghcr.io/astral-sh/uv:0.9.26 /uv /uvx /bin/
WORKDIR /app
COPY package.json bun.lock /app/
COPY frontend/package.json /app/frontend/
WORKDIR /app/frontend
RUN bun install
# Compile bytecode
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#compiling-bytecode
ENV UV_COMPILE_BYTECODE=1
COPY ./frontend /app/frontend
# Empty: the dashboard calls the API as a path on whichever origin served it,
# so one build works behind any port, hostname or proxy.
ENV VITE_API_URL=
RUN bun run build
# uv Cache
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#caching
ENV UV_LINK_MODE=copy
# ── Python environment ───────────────────────────────────────────
FROM python:3.13-slim AS backend
# Ref: https://docs.astral.sh/uv/guides/integration/docker/
COPY --from=ghcr.io/astral-sh/uv:0.9.26 /uv /bin/
ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy
WORKDIR /app/
# Place executables in the environment at the front of the path
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#using-the-environment
ENV PATH="/app/.venv/bin:$PATH"
ARG EXTRAS=""
# /app/.venv holds the app and nothing of anybody else's, so there is nothing
# here to adopt: node code gets a venv of its own on the data volume, which is
# what the Modules screen installs into. A `pip install fluksio` into an
# environment somebody already works in is the case that adopts instead.
ENV NODE_VENV=managed
# ffmpeg for node code that transcodes, such as the clip recipe in
# docs/code/nodes.md. Off by default: Debian's package and what it pulls in add
# about 400 MB. Set WITH_FFMPEG=true in .env and rebuild to include it.
ARG WITH_FFMPEG=false
RUN if [ "$WITH_FFMPEG" = true ]; then \
apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg \
&& rm -rf /var/lib/apt/lists/*; \
fi
# Install dependencies
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#intermediate-layers
# Dependencies first, so a source change does not reinstall them.
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --frozen --no-install-workspace --package fluksio --extra server
uv sync --frozen --no-dev --no-install-workspace --package fluksio \
$(for e in $(echo "$EXTRAS" | tr , ' '); do printf -- '--extra %s ' "$e"; done)
COPY ./backend/scripts /app/backend/scripts
@@ -57,12 +54,11 @@ COPY ./backend/fluksio /app/backend/fluksio
# node code with.
COPY ./worker /app/worker
# Sync the project
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#intermediate-layers
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=uv.lock,target=uv.lock \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --frozen --package fluksio --extra server
uv sync --frozen --no-dev --package fluksio \
$(for e in $(echo "$EXTRAS" | tr , ' '); do printf -- '--extra %s ' "$e"; done)
# Connectors are ordinary installed packages found through the
# `fluksio.node_types` entry point. `make connectors` builds them into here;
@@ -73,9 +69,45 @@ RUN --mount=type=cache,target=/root/.cache/uv \
&& uv pip install --python /app/.venv/bin/python /tmp/connector-wheels/*.whl \
|| echo "no connectors bundled"'
WORKDIR /app/backend/
# ── Runtime ──────────────────────────────────────────────────────
FROM python:3.13-slim
# Single worker on purpose: the process hosts the flow engine, and a second
# worker would be a second engine — duplicated subscriptions, cron ticks and
# webhooks. Scaling out is the M5 worker split, not more uvicorn processes.
CMD ["fastapi", "run", "fluksio/main.py"]
# git versions the flows. redis-server keeps message state when no external
# Redis is named. tini reaps the python node workers that outlive their pool,
# which would otherwise pile up as zombies under PID 1. procps is uptime, ps
# and free for Command nodes, the demo's among them.
RUN apt-get update \
&& apt-get install -y --no-install-recommends git procps redis-server tini \
&& rm -rf /var/lib/apt/lists/*
# /app/.venv holds the app and nothing of anybody else's, so there is nothing
# here to adopt: node code gets a venv of its own on the data volume, which is
# what the Modules screen installs into.
ENV PYTHONUNBUFFERED=1 \
PATH="/app/.venv/bin:$PATH" \
NODE_VENV=managed \
FRONTEND_DIST=/app/frontend/dist \
FRONTEND_HOST=http://localhost:8000
# The venv installs the app editable, so the source stays at the path it was
# synced from. uv comes with it: the Modules screen installs with it.
COPY --from=backend /app /app
COPY --from=frontend /app/frontend/dist /app/frontend/dist
COPY ./backend/docker-entrypoint.sh /usr/local/bin/
WORKDIR /app/backend/
VOLUME /data
EXPOSE 8000
# Deep health: fails when the event loop is wedged or Redis is gone, not just
# when the process is dead.
HEALTHCHECK --interval=10s --timeout=5s --start-period=30s --retries=5 \
CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/api/v1/utils/health/', timeout=4)"
ENTRYPOINT ["tini", "--", "docker-entrypoint.sh"]
# Single process on purpose: it hosts the flow engine, and a second worker
# would be a second engine — duplicated subscriptions, cron ticks and webhooks.
# `serve` makes the secret key and the first admin on first start, and prints
# the admin's password once.
CMD ["fluksio", "serve", "--plain", "--host", "0.0.0.0", "--port", "8000", "--data-dir", "/data"]
+23
View File
@@ -0,0 +1,23 @@
#!/bin/sh
# Starts the bundled Redis unless an external one is named, then the command.
#
# Redis holds the last value of every message, node state and the work queue,
# so without it a restart forgets all three. In the background on purpose: its
# appendonly file is written on every change, so the SIGKILL it gets when the
# container stops loses nothing that was acknowledged.
set -e
if [ -z "$REDIS_HOST" ]; then
mkdir -p /data/redis
redis-server --bind 127.0.0.1 --port 6379 --dir /data/redis \
--appendonly yes --loglevel warning &
i=0
until redis-cli -h 127.0.0.1 ping >/dev/null 2>&1; do
i=$((i + 1))
[ "$i" -lt 100 ] || { echo "bundled redis did not start" >&2; exit 1; }
sleep 0.1
done
export REDIS_HOST=127.0.0.1
fi
exec "$@"
+10 -6
View File
@@ -54,7 +54,12 @@ class Settings(BaseSettings):
SECRET_KEY: str = secrets.token_urlsafe(32)
# 60 minutes * 24 hours * 8 days = 8 days
ACCESS_TOKEN_EXPIRE_MINUTES: int = 60 * 24 * 8
#: The address people and devices reach this instance at: panel links,
#: mails and the MCP issuer are built from it.
FRONTEND_HOST: str = "http://localhost:5173"
#: The built dashboard, served on the API's own origin when set. The image
#: sets it; a checkout runs Vite instead and leaves it empty.
FRONTEND_DIST: Path | None = None
ENVIRONMENT: Literal["local", "staging", "production"] = "local"
#: The proxies whose X-Forwarded-For is believed: uvicorn's own setting,
#: read here too so a call replayed off the portal tunnel resolves its
@@ -111,7 +116,6 @@ class Settings(BaseSettings):
# unless asked for: an account can change what flows do, so who gets one
# is a superuser's call (Admin → Users).
SIGNUP_ENABLED: bool = False
DOMAIN: str = "localhost"
OAUTH_PRIVATE_KEY_FILE: Path = Path("flow-data/oauth-key.pem")
# Written only when someone enrols this instance with a portal.
# Its absence is what keeps remote access off.
@@ -237,14 +241,14 @@ class Settings(BaseSettings):
@computed_field # type: ignore[prop-decorator]
@property
def oauth_issuer(self) -> str:
"""Who issues MCP tokens — this app, on its API host.
"""Who issues MCP tokens — this app, at the address it is reached on.
Kept separate from the app's own URL because a hosted deployment can
later point agents at a different issuer without the resource server
The API is a path on the dashboard's origin, so that one address is
both. Kept a property of its own because a hosted deployment can later
point agents at a different issuer without the resource server
changing: it validates whatever issuer it is configured to trust.
"""
scheme = "http" if self.ENVIRONMENT == "local" else "https"
return f"{scheme}://api.{self.DOMAIN}"
return self.FRONTEND_HOST.rstrip("/")
@computed_field # type: ignore[prop-decorator]
@property
+3 -2
View File
@@ -15,7 +15,7 @@ logger = logging.getLogger(__name__)
def _influxdb() -> Any:
"""The client library, which is a `fluksio[server]` extra.
"""The client library, which is the `influx` extra.
Imported per use rather than at module level, because the node type is
registered at boot and an instance with no InfluxDB behind it should
@@ -25,7 +25,8 @@ def _influxdb() -> Any:
import influxdb_client
except ImportError:
raise RuntimeError(
"the influxdb node needs the server extra: pip install 'fluksio[server]'"
"the influxdb node needs the influx extra: pip install 'fluksio[influx]', "
"or build the image with EXTRAS=influx"
) from None
return influxdb_client
+18 -1
View File
@@ -7,7 +7,6 @@ import threading
from collections.abc import Iterable
from typing import Any
import numpy as np
from pydantic import BaseModel, ConfigDict
from fluksio.flow.messages import MessageSpec
@@ -16,6 +15,22 @@ from fluksio.flow.nodes.base import Node
logger = logging.getLogger(__name__)
def _numpy() -> Any:
"""numpy, which is the `ml` extra.
Imported when a node is built rather than at boot, so an instance without
it still offers every other type, and a node placed without it says so.
"""
try:
import numpy
except ImportError:
raise RuntimeError(
"the mlp node needs the ml extra: pip install 'fluksio[ml]', "
"or build the image with EXTRAS=ml"
) from None
return numpy
class MLPNode(Node):
"""
Multi-Layer Perceptron node for neural network processing in pipelines.
@@ -72,6 +87,7 @@ class MLPNode(Node):
)
cfg = self.Params.model_validate(self.params)
np = self._np = _numpy()
rng = np.random.default_rng(seed=cfg.seed)
num_inputs = max(1, len(self.input_ports))
num_outputs = max(1, len(self.output_ports))
@@ -91,6 +107,7 @@ class MLPNode(Node):
self.id,
)
np = self._np
if kwargs:
input_array = np.array([float(v) for v in kwargs.values()])
else:
+3 -2
View File
@@ -21,7 +21,7 @@ logger = logging.getLogger(__name__)
def _aiomqtt() -> Any:
"""The client library, which is a `fluksio[server]` extra.
"""The client library, which is the `mqtt` extra.
Imported per use rather than at module level, because the node type is
registered at boot and an instance that talks to no broker should not
@@ -31,7 +31,8 @@ def _aiomqtt() -> Any:
import aiomqtt
except ImportError:
raise RuntimeError(
"the mqtt node needs the server extra: pip install 'fluksio[server]'"
"the mqtt node needs the mqtt extra: pip install 'fluksio[mqtt]', "
"or build the image with EXTRAS=mqtt"
) from None
return aiomqtt
+6 -1
View File
@@ -90,7 +90,7 @@ def custom_generate_unique_id(route: APIRoute) -> str:
if settings.SENTRY_DSN and settings.ENVIRONMENT != "local":
# Imported here rather than at the top: it is a `fluksio[server]` extra, so
# Imported here rather than at the top: it is the `sentry` extra, so
# a pip install without one has no sentry to import — and no DSN either.
import sentry_sdk
@@ -506,6 +506,11 @@ if settings.all_cors_origins:
app.add_middleware(DocsForThisMachine)
if settings.FRONTEND_DIST:
from fluksio.spa import Dashboard
app.add_middleware(Dashboard, dist=settings.FRONTEND_DIST)
# The caller's address, for every per-address limit and the docs check above:
# the rightmost X-Forwarded-For hop that is not a trusted proxy. uvicorn does
# this for what it serves, but a call the portal tunnel replays arrives over an
+88
View File
@@ -0,0 +1,88 @@
"""The dashboard, served by the API on its own origin.
One process and one port: the built SPA comes off disk, and every path the API
does not own answers with ``index.html``, so a client-side route survives a
reload. Same origin also means the dashboard calls the API as a path, needs no
CORS allowance and no second hostname, which is what lets the whole instance
sit behind a single published port.
"""
from __future__ import annotations
import base64
import hashlib
import mimetypes
import re
from pathlib import Path
from starlette.middleware.gzip import GZipMiddleware
from starlette.responses import FileResponse
from starlette.types import ASGIApp, Receive, Scope, Send
#: What the API answers itself. Everything else is the dashboard's.
API_PREFIXES = ("/api/", "/hooks/", "/mcp", "/.well-known/", "/docs", "/redoc")
# Older mime tables predate the manifest, and it would go out as a download.
mimetypes.add_type("application/manifest+json", ".webmanifest")
def content_security_policy(index_html: str) -> str:
"""The policy for the dashboard, admitting its inline scripts by hash.
The hash is taken from the file being served, so editing the theme script
in ``index.html`` cannot leave a stale one behind.
style-src 'unsafe-inline': Monaco, Radix's scroll lock and the toasts insert
<style> elements at runtime, which no hash can name. img/frame-src take any
http(s): a camera stream or website widget points wherever its dashboard
says.
"""
hashes = "".join(
f" 'sha256-{base64.b64encode(hashlib.sha256(script.encode()).digest()).decode()}'"
for script in re.findall(r"<script>([\s\S]*?)</script>", index_html)
)
return (
f"default-src 'self'; script-src 'self'{hashes}; "
"style-src 'self' 'unsafe-inline'; img-src 'self' data: blob: http: https:; "
"media-src 'self' blob:; font-src 'self'; connect-src 'self'; "
"worker-src 'self'; frame-src http: https:; frame-ancestors 'self'; "
"object-src 'none'; base-uri 'self'; form-action 'self'"
)
class Dashboard:
"""Serve the built SPA for every GET the API does not own."""
def __init__(self, app: ASGIApp, dist: Path) -> None:
self.app = app
self.dist = dist.resolve()
self.index = self.dist / "index.html"
self.headers = {
"Content-Security-Policy": content_security_policy(self.index.read_text()),
"X-Content-Type-Options": "nosniff",
"Referrer-Policy": "strict-origin-when-cross-origin",
}
async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
path: str = scope.get("path", "")
if (
scope["type"] != "http"
or scope["method"] not in ("GET", "HEAD")
or path.startswith(API_PREFIXES)
):
await self.app(scope, receive, send)
return
file = (self.dist / path.lstrip("/")).resolve()
if file.is_file() and file.is_relative_to(self.dist) and file != self.index:
headers = dict(self.headers)
# Vite puts a content hash in every asset's name, so one never
# changes under its URL.
if path.startswith("/assets/"):
headers["Cache-Control"] = "public, max-age=31536000, immutable"
else:
# The one file that must not be cached: it names the current
# assets, so a stale copy pins the browser to the previous build.
file = self.index
headers = {**self.headers, "Cache-Control": "no-cache"}
response = FileResponse(file, headers=headers)
await GZipMiddleware(response, minimum_size=1024)(scope, receive, send)
+2 -1
View File
@@ -44,7 +44,8 @@ def send_email(
import emails # type: ignore
except ImportError:
raise RuntimeError(
"sending mail needs the server extra: pip install 'fluksio[server]'"
"sending mail needs the email extra: pip install 'fluksio[email]', "
"or build the image with EXTRAS=email"
) from None
message = emails.Message(
subject=subject,
+15 -15
View File
@@ -26,8 +26,10 @@ classifiers = [
# email-validator, jinja2 and httpx — also arrive inside `fastapi[standard]`,
# so those lines look redundant and install nothing extra; they are here so a
# direct import does not rest on somebody else's extra keeping its contents.
# The variant without fastapi-cloud-cli, which brings sentry-sdk and a deploy
# command nothing here runs.
dependencies = [
"fastapi[standard]<1.0.0,>=0.114.2",
"fastapi[standard-no-fastapi-cloud-cli]<1.0.0,>=0.114.2",
"pydantic>2.0",
"pydantic-settings<3.0.0,>=2.2.1",
"sqlmodel<1.0.0,>=0.0.21",
@@ -56,9 +58,6 @@ dependencies = [
# the journal, the worker pipe. Engine-side only: `fluksio-worker` is
# copied onto other people's machines and stays dependency-free.
"orjson>=3.10",
# The `mlp` node, which is registered at boot. Kept out of the extras
# because a data-science environment has it already.
"numpy>=2.2.6",
# The `wren` node runs its model on one BLAS thread, which answers in about
# half the time. Scoped to the call: OPENBLAS_NUM_THREADS would bind every
# other numpy call in the engine too, and only counts before numpy loads.
@@ -94,17 +93,18 @@ dependencies = [
# extra rather than a dependency: csv and jsonl need nothing, and pyarrow is
# tens of megabytes for whoever wants dtypes kept.
parquet = ["pyarrow>=17"]
# What a deployment has and a laptop does not: the device connectors, outbound
# mail and error reporting. The engine, the CLI and every python node work
# without them — `pip install fluksio` in a data-science environment is the
# case this exists for, and lxml and the aiohttp stack are most of its wait.
# Each import is guarded and names this extra. The image installs it.
server = [
"emails<1.0,>=0.6",
"sentry-sdk[fastapi]>=2.20.0",
"aiomqtt>=2.0.0",
"influxdb-client[async]>=1.40.0",
]
# One per feature, so an instance carries only what it talks to. The engine,
# the CLI and every python node work without any of them. Each import is
# guarded and names its extra; the image takes them as EXTRAS=mqtt,influx.
mqtt = ["aiomqtt>=2.0.0"]
influx = ["influxdb-client[async]>=1.40.0"]
# Outbound mail. lxml and its stylesheet inliner are most of this.
email = ["emails<1.0,>=0.6"]
sentry = ["sentry-sdk[fastapi]>=2.20.0"]
# The `mlp` node. A data-science environment has it already.
ml = ["numpy>=2.2.6"]
# All of the above: what a full deployment runs.
server = ["fluksio[mqtt,influx,email,sentry,ml]"]
[project.urls]
Homepage = "https://fluksio.com"
+2 -3
View File
@@ -40,7 +40,6 @@ os.environ["NODE_VENV"] = "managed"
# (with the dev stack) where they are meant to work.
os.environ["PRIVATE_API_ENABLED"] = "true"
# The suite is the development configuration by definition: `/private` is
# gated on it (app/api/routes/private.py) and the MCP host allow-list is
# built from DOMAIN (app/mcp/http.py), while the tests speak to api.localhost.
# gated on it (app/api/routes/private.py) and the MCP host allow-list admits
# localhost under it (app/mcp/http.py).
os.environ["ENVIRONMENT"] = "local"
os.environ["DOMAIN"] = "localhost"
+1 -1
View File
@@ -59,7 +59,7 @@ def over_mcp(monkeypatch: pytest.MonkeyPatch):
async with mcp_server.mcp.session_manager.run():
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=mcp_app),
base_url="http://api.localhost",
base_url=settings.oauth_issuer,
) as client:
return await block(client)
+39
View File
@@ -0,0 +1,39 @@
import base64
import hashlib
from pathlib import Path
from starlette.applications import Starlette
from starlette.responses import PlainTextResponse
from starlette.routing import Route
from starlette.testclient import TestClient
from fluksio.spa import Dashboard
THEME = "document.documentElement.classList.add('dark')"
def _client(dist: Path) -> TestClient:
(dist / "assets").mkdir(parents=True)
(dist / "index.html").write_text(f"<html><script>{THEME}</script></html>")
(dist / "assets" / "app-1a2b.js").write_text("console.log(1)")
api = Starlette(routes=[Route("/api/v1/ping", lambda _: PlainTextResponse("api"))])
return TestClient(Dashboard(api, dist))
def test_the_dashboard_and_the_api_share_one_origin(tmp_path: Path) -> None:
client = _client(tmp_path)
# A client-side route survives a reload, with the inline script admitted.
page = client.get("/flows/house")
digest = base64.b64encode(hashlib.sha256(THEME.encode()).digest()).decode()
assert THEME in page.text
assert page.headers["cache-control"] == "no-cache"
assert f"'sha256-{digest}'" in page.headers["content-security-policy"]
asset = client.get("/assets/app-1a2b.js")
assert asset.text == "console.log(1)"
assert "immutable" in asset.headers["cache-control"]
assert client.get("/api/v1/ping").text == "api"
# Nothing outside the build directory is served, only the shell.
assert THEME in client.get("/assets/../../etc/passwd").text