Commit Graph
13 Commits
Author SHA1 Message Date
stroblme dde3d0af0d Portal app-shell as a busybox target, git without perl, quieter first start
The portal's copy of the dashboard is the `appshell` target of
backend/Dockerfile (18 MB instead of the 252 MB nginx image); the nginx
frontend Dockerfile, its confs and csp.check.ts are gone, since the app and
the hub both hash the CSP from the index.html they serve.

The runtime image copies the 4 MB git binary alone: the store's init, add,
commit, gc and rev-parse are builtins, and a build-time round trip fails the
build if that ever changes. About 100 MB of perl and helpers less.

The bundled Redis's overcommit warning and FastMCP's lifespan warning no
longer greet every `docker logs`.
2026-10-16 12:20:13 +02:00
stroblme dea25ec08d One image for the whole app: engine, API and dashboard on one port
The backend image now builds the SPA and serves it on the API's own origin
(fluksio/spa.py), so an instance is a single container with no proxy, DNS,
port 80 or .env: `docker compose up -d` with the new compose.yml runs it on
localhost:8080. It starts on fluksio serve (persisted secret key, first
admin printed once) and brings its own Redis when REDIS_HOST is unset.

Slimmer: python:3.13-slim, no dev group, uv once, fastapi without the cloud
CLI. Optional features are per-feature extras (mqtt, influx, email, sentry,
ml, server for all) picked with the EXTRAS build argument; numpy moved to
`ml`. ffmpeg left the image for imageio-ffmpeg via the Modules screen.

The operator stack runs the same image with app. and api. routed to it; the
nginx frontend service, the LAN nginx proxy and DOMAIN in the backend are
gone, and the MCP issuer is FRONTEND_HOST.
2026-10-16 11:40:02 +02:00
stroblme 96618e40a2 Opt-in WITH_FFMPEG build arg installs Debian's ffmpeg in the backend image
Off by default, since it adds about 400 MB. Documented with the clip recipe
and in the configuration reference.
2026-10-15 15:55:02 +02:00
Melvin Strobl 0264310d4f Defer ffmpeg in the image; record the decision 2026-10-14 07:33:59 +02:00
Melvin Strobl 1964c5b2e8 Clip transcoding: ffmpeg in the backend image, panel recipe in the node docs 2026-10-14 07:30:36 +02:00
stroblme ab4ffefa0f Put the deployment-only dependencies behind a server extra
A data-science environment installing fluksio waited for lxml, aiohttp and
the rest of a connector stack it has nothing to talk to. Outbound mail,
error reporting and the MQTT and InfluxDB clients moved to
`fluksio[server]`, which the image installs; each import is guarded and
names the extra. `tenacity` had no import site at all and is gone.

23 fewer packages and the compiled ones among them — a bare `pip install
fluksio` still serves, runs every python node, and registers the mqtt and
influxdb node types, which only need the library when one is actually
built. sentry-sdk arrives anyway underneath `fastapi[standard]`; what
changed there is that nothing of ours requires it.

The dev environment keeps every extra: the suite exercises the connectors
and strict mypy checks their call sites.
2026-08-29 14:06:50 +02:00
stroblme af797ac765 Ship the files pyproject.toml points at
readme and license-files name README.md and LICENSE, but the image only ever
received pyproject.toml, so hatchling failed metadata validation the moment
the layer cache stopped hiding it:

  OSError: Readme file does not exist: README.md
2026-08-24 14:47:44 +02:00
stroblme 86ceccdf7f Run node code on the venv Fluksio was installed into
The workflow this serves: make a venv, install what you work with, then `pip
install fluksio` into the same one. Building a second environment beside it
was exactly wrong — the packages the nodes need are already here, and the
Modules screen was asking for them a second time.

`NODE_VENV=auto` (the default) adopts that venv. It declines in the three
cases where adopting would be wrong: `managed` says otherwise, a managed venv
already exists and may hold packages somebody installed on purpose, or the
engine is not running from a venv at all. The images set `managed`, since the
venv in them holds the app and nothing of anybody else's.

An adopted venv is never written to. `uv pip sync` makes a venv hold exactly
the manifest, so pointed at somebody's own environment it uninstalls their
work and the engine with it — `sync()` refuses outright and `reconcile()`
returns before it can be called at startup, which is where that would have
happened first. The Modules screen lists what is installed and drops its
editor; `pip` is how that environment changes.

`fluksio serve` now names the interpreter node code runs on, which is the
thing a data scientist most needs to know at that moment. `fluksio-worker`
already defaulted `--python` to its own interpreter, so a GPU box works the
same way — that was only ever undocumented.
2026-08-24 10:35:13 +02:00
stroblme e3171f87e2 Split the worker into a distribution of its own
A cluster or GPU host installs `pip install fluksio-worker` and gets the
agent and the runner, not psycopg, numpy and the MCP SDK. The engine
depends on it as a workspace member, so the file it launches node code
with is the same file a remote worker runs — which is what keeps a node
unable to tell the difference. Copying the two files by hand still works.
2026-08-21 21:54:10 +02:00
stroblme 99534a82ee Rename the import package app to fluksio
A wheel whose top-level module is `app` collides with anything else in a
user's venv, so the package that is about to be published takes the name
it is published under. Only the Python package moves; the repo, the
Docker WORKDIR and the compose project keep theirs.
2026-08-21 21:48:05 +02:00
stroblme 49eae9f6df Record the connectors and what is still read-only 2026-08-16 10:00:10 +02:00
root 2a224ac94e Supervise the engine's host: deep health, loop watchdog, one worker
The API image ran four uvicorn workers, and each one built a full flow
controller — four sets of MQTT subscriptions, cron ticks and webhooks.
Runs one worker now; scaling out is the worker split, not more processes.

Adds a loop-lag watchdog and a deep /utils/health/ that fails when the
event loop is wedged or Redis is unreachable, the two failure modes a
process-alive check never sees. Autoheal restarts on that signal, behind
a compose profile because it mounts the Docker socket.

The private user-seeding routes now need an explicit opt-in rather than
just ENVIRONMENT=local, so a deployment that kept the default never
exposes them.
2026-08-16 07:14:28 +02:00
Melvin Strobl 8be7f9770e Vendor backend and frontend into the monorepo
The submodule collapse was only half applied: .gitmodules was deleted but
backend/ and frontend/ were still recorded as gitlinks, so none of their
files were tracked. Replace the gitlinks with the real trees.

Also untrack .env (it carried placeholder secrets) in favour of a tracked
.env.example, drop the committed __pycache__, and narrow the blanket *.png
ignore that would have swallowed design assets.
2026-08-09 15:14:54 +02:00