stroblmeandClaude Opus 5
a8065ad91c
The queue shared the `pipeline:` prefix with flow state, so `RedisState.clear()` could DEL the queue stream and `keys()` enumerated queue entries — only callers filtering `__`-prefixed names kept it safe. It moves to `queue:` without a migration: whatever is in flight at the upgrade is dropped once, documented in DEPLOY.md rather than papered over. Alongside it: `pool_pre_ping`, so a connection idle across a Postgres restart costs a round trip instead of a failed request; the test suite pins ENVIRONMENT=local and DOMAIN=localhost itself rather than inheriting a deployment's .env; and `depth` leaves the queue stats, where it reported the capped journal length as if it were a backlog. ALERTS_FILE and PANELS_FILE now point at /data. They defaulted to a path on no volume, so alert routing and every wall-panel pairing were living in the container's writable layer and vanishing on each rebuild. Carrying the existing files across is a manual step; DEPLOY.md has it. development.md was still the upstream template — compose.override.yml, localhost.tiangolo.com, `docker compose watch` as the dev flow — and said nothing about the Playwright suite. Rewritten against what the Makefiles actually do. deployment.md was template text too, duplicating the root DEPLOY.md, and is gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uq8mtNb97A7praJLyeEYgs
172 lines
9.2 KiB
Markdown
172 lines
9.2 KiB
Markdown
# Development
|
|
|
|
How to run, test and lint the `app` stack. The workspace root owns the rest:
|
|
[`README.md`](../README.md) for the quick start, [`DEPLOY.md`](../DEPLOY.md) for
|
|
production, [`DESIGN-GUIDELINES.md`](../DESIGN-GUIDELINES.md) before any UI work.
|
|
[`ROADMAP.md`](ROADMAP.md) is what is planned, [`NOTEPAD.md`](NOTEPAD.md) what is deferred.
|
|
|
|
## Running it
|
|
|
|
**Integrated — the normal way.** From the workspace root, once: `make init` (submodules,
|
|
secrets, the per-stack `.env` files, the shared external `proxy` docker network). Then
|
|
`make dev` brings both stacks up detached behind a single Traefik — which is what lets the
|
|
SPA, the API and the marketing site answer on one port. `make status` lists the containers,
|
|
`make down` stops everything.
|
|
|
|
The hostname comes from `DOMAIN` in `app/.env`, which `scripts/setup.sh` copies out of the
|
|
root `.env` — change it there and re-run `make init`. `make dev DOMAIN=…` at the root does
|
|
*not* reach this stack: `app/Makefile` assigns `DOMAIN` from `app/.env`, and a makefile
|
|
assignment overrides an inherited environment variable.
|
|
|
|
**App only.** `cd app && make dev` starts this stack on its own Traefik with all host ports
|
|
published, in the foreground. `make dev-utils` starts just db, adminer, proxy, mailcatcher
|
|
and prestart — the useful half when the backend runs on the host.
|
|
|
|
**No Docker.** `make install` once, then `make dev-backend` (FastAPI on :8000) and
|
|
`make dev-frontend` (Vite on :5173) in two terminals. A local stack's CORS list already
|
|
contains `http://localhost:5173`, so a host Vite server can talk to a containerised API.
|
|
|
|
Nothing invokes `docker compose` bare, and neither should you: both stacks keep their
|
|
compose files in a `docker/` directory, so without `-p fluksio-app` they collide in a
|
|
project named after that directory, and `--env-file` is required because compose
|
|
interpolates `.env` before it reads `compose.yml`. The `COMPOSE` variable in `Makefile` has
|
|
both. The files layer as `compose.yml` (production) → `compose.dev.yml` (local Traefik,
|
|
published ports, hot reload, test helpers) → `compose.local.yml` (integrated). CI layers
|
|
`compose.ci.yml` instead of the last one; `compose.traefik.yml` is the production edge
|
|
proxy, deployed on its own.
|
|
|
|
## URLs
|
|
|
|
Integrated stack (root `make dev`). Traefik's port 80 is the **only** published port:
|
|
`compose.local.yml` drops the rest, since behind the proxy they add nothing and one of them
|
|
already being taken would stop the whole stack from starting.
|
|
|
|
| URL | Service |
|
|
|---|---|
|
|
| http://app.localhost | dashboard SPA |
|
|
| http://api.localhost | backend API |
|
|
| http://api.localhost/docs | OpenAPI / Swagger UI |
|
|
| http://localhost | marketing site (the `index` stack) |
|
|
|
|
Adminer and mailcatcher have no route here, so use the containers directly:
|
|
`docker exec -it fluksio-db psql -U postgres -d app` for the database,
|
|
`docker logs -f fluksio-app-mailcatcher-1` for captured mail.
|
|
|
|
Standalone (`cd app && make dev`) publishes the full set:
|
|
|
|
| URL | Service |
|
|
|---|---|
|
|
| http://app.localhost, http://api.localhost | this stack's own Traefik on :80 |
|
|
| http://localhost:5173 | SPA (the nginx image, not the Vite dev server) |
|
|
| http://localhost:8000 | backend API |
|
|
| http://localhost:8090 | Traefik dashboard |
|
|
| http://127.0.0.1:8080 | adminer |
|
|
| http://localhost:1080 | mailcatcher web UI (SMTP on 1025) |
|
|
| localhost:5432 | Postgres |
|
|
|
|
## Hot reload, and what it misses
|
|
|
|
`compose.dev.yml` bind-mounts `backend/app` into the api container and runs uvicorn with
|
|
`--reload`, so a backend edit is live. The `develop.watch` block next to it syncs the same
|
|
directory, but only under `docker compose watch`, which no target runs — the detached
|
|
`up -d` flow never triggers it. The mount is what does the work, so an api container created
|
|
before the mount existed keeps serving the source baked into its image and has to be
|
|
recreated once.
|
|
|
|
The frontend is built into an nginx image, so a UI change needs an explicit rebuild, from
|
|
`app/`:
|
|
|
|
```sh
|
|
docker compose -p fluksio-app --env-file "$PWD/.env" \
|
|
-f docker/compose.yml -f docker/compose.dev.yml -f docker/compose.local.yml \
|
|
up --build -d frontend
|
|
```
|
|
|
|
## Services the dev stack adds
|
|
|
|
`compose.dev.yml` brings up a broker and a time-series database of the stack's own, so the
|
|
mqtt and influx node types are testable without external hardware. Neither publishes a host
|
|
port, on purpose — nothing outside the stack needs them.
|
|
|
|
- **mosquitto** — anonymous MQTT. Point a node at `broker_host: mosquitto`, port 1883.
|
|
- **influxdb 2.7** — `http://influxdb:8086`, org and bucket `fluksio`, token
|
|
`fluksio-dev-token`. No volume, so `down -v` starts it over.
|
|
- **mailcatcher** — SMTP on 1025, web UI on 1080; the backend is pointed at it, so no
|
|
development mail leaves the machine.
|
|
|
|
`adminer` and `redis` come from `compose.yml` and exist in production too; dev only differs
|
|
in that adminer gets a host port. Without `REDIS_HOST` the flow engine keeps state in
|
|
memory instead of Redis.
|
|
|
|
## Configuration and secrets
|
|
|
|
The root `.env` is the source of truth. `scripts/setup.sh` creates `app/.env` from
|
|
`app/.env.example` and keeps the shared keys — domain, secret key, database password,
|
|
superuser — in step with it. Only the `.env.example` files are tracked; `make secrets`
|
|
reprints the generated values. A release that adds a key needs `scripts/setup.sh --secrets`
|
|
before `make update`, so the key exists before the stack is rebuilt.
|
|
|
|
The backend settings load `../.env` relative to `backend/` through pydantic-settings, which
|
|
is why an exported environment variable always outranks the file — how CI points the suite
|
|
at a Docker-assigned Postgres port.
|
|
|
|
## Tests
|
|
|
|
`make test` is `test-backend` plus `test-frontend`.
|
|
|
|
**Backend.** `cd backend && uv run bash scripts/tests-start.sh` waits for Postgres, then
|
|
runs pytest under coverage; the HTML report lands in `backend/htmlcov`. Postgres is the only
|
|
service needed — mail is patched out in `tests/`, and the flow engine falls back to
|
|
`MemoryState` while `REDIS_HOST` is empty. `tests/__init__.py` pins the suite's own
|
|
environment before anything imports the settings: database `app_test` (created and dropped
|
|
per session, so a run never touches development data), plus `ENVIRONMENT=local` and
|
|
`DOMAIN=localhost`, so a checkout configured for a deployment cannot drag that deployment's
|
|
configuration into the run. `make test-backend` resolves `POSTGRES_SERVER` from the running
|
|
`fluksio-db` container's address, because the integrated stack publishes no 5432.
|
|
|
|
**Frontend (e2e).** `make test-frontend` runs the suite inside the pinned
|
|
`mcr.microsoft.com/playwright:v<version>-noble` image (version read from
|
|
`frontend/package.json`) on the `proxy` network, against a stack that must already be up. It
|
|
maps `app.$(DOMAIN)` and `api.$(DOMAIN)` onto the Traefik container's address twice, with
|
|
`--add-host` and with `HOST_RESOLVER_RULES`, because Chromium pins `*.localhost` to loopback
|
|
whatever `/etc/hosts` says. Narrow a run with
|
|
`make test-frontend PLAYWRIGHT_ARGS="--grep flows"`.
|
|
|
|
The specs are `frontend/tests/*.spec.ts`; `playwright.config.ts` defines `setup` →
|
|
`chromium` and `mobile`. `auth.setup.ts` logs in once into `playwright/.auth/user.json`,
|
|
which both browser projects reuse; `mobile` runs `mobile.spec.ts` only, on a Pixel 5 at
|
|
393px.
|
|
|
|
Both origins come from `PLAYWRIGHT_BASE_URL` and `PLAYWRIGHT_API_URL`, deliberately never
|
|
from `VITE_API_URL` — that one belongs to the app build and in a deployment checkout it
|
|
names the deployment, which would mean driving a browser at the local stack while sending
|
|
teardown `DELETE`s to the live instance. `tests/guard.ts` runs first and refuses the whole
|
|
run when either origin resolves outside loopback or the private ranges, because the suite
|
|
creates and deletes flows, dashboards and users. `PLAYWRIGHT_ALLOW_PUBLIC=1` overrides it.
|
|
|
|
**Visual check.** From the root, `make verify` logs in and screenshots both themes into
|
|
`app/frontend/screenshots/{light,dark}/`; `make verify-docker` does the same from the
|
|
Playwright container when the host lacks the browser libraries. Both default to `localhost`
|
|
rather than reading `.env`, so on a checkout configured for a deployment name the domain the
|
|
stack actually serves: `make verify DOMAIN=example.com`.
|
|
|
|
## Lint, hooks and the generated client
|
|
|
|
`make lint` runs ruff, mypy and biome. `make hooks` at the root installs the pre-commit
|
|
hooks in both stacks; `.pre-commit-config.yaml` holds them. One is worth knowing about:
|
|
`generate-frontend-sdk` regenerates the SPA's API client from the backend's OpenAPI schema
|
|
whenever anything under `backend/` changes, so a route change reaches the frontend without
|
|
being asked. `make generate-client` forces it.
|
|
|
|
Run the root's `make design-check` before pushing anything that touches the design tokens,
|
|
`lib/motion.ts`, `components.json` or the font asset — the design system is a duplication
|
|
contract between the two frontends, and that target is the only thing checking it.
|
|
|
|
## CI
|
|
|
|
`.gitea/workflows/` runs four jobs, each writing `.env` from `.env.example` first:
|
|
`test-backend.yml` (pytest against a compose Postgres), `playwright.yml` (the e2e suite in
|
|
two shards, one compose project each, the browser loading the SPA from the nginx `frontend`
|
|
service), `pre-commit.yml` (the hooks above, plus strict mypy, which they do not cover) and
|
|
`test-compose.yml` (a smoke test that the production images actually come up).
|