Keep the engine's state in SQLite, not Postgres

One process owns this database — the image has run a single uvicorn
worker for that reason since the four-engines bug — so a file beside the
flows is the honest shape for it, and it is what lets `fluksio serve`
need no infrastructure at all. Live values, node execution and the work
queue never came here anyway; what does is a rollup a minute at a time,
a row per cascade and the run history, and WAL keeps the readers going
while that one writer works.

DATA_DIR is now the one setting that moves everything an installation
keeps; the rest derive from it and the images still spell theirs out.
The schema is prepared in-process at startup, so the prestart service is
gone, and the ten Postgres-only revisions collapse into one portable
baseline.

Three things only worked because psycopg was casting for us: a token's
subject arriving as a string where the column is a UUID, `greatest`, and
`date_bin`. The timestamps needed a column type of their own — SQLite
stores no offset, and a naive datetime read back either raises against an
aware `now` or serialises as local time.

Postgres stays in the stack only for Umami, behind the analytics profile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-21 22:19:45 +02:00
co-authored by Claude Opus 5
parent 2c369ac75f
commit 961a8f881d
51 changed files with 841 additions and 1089 deletions
-13
View File
@@ -1,13 +0,0 @@
#! /usr/bin/env bash
set -e
set -x
# Let the DB start
python -m fluksio.backend_pre_start
# Run migrations
alembic upgrade head
# Create initial data in DB
python -m fluksio.initial_data
+16 -19
View File
@@ -53,10 +53,10 @@ COMPOSE_PROJECT = "fluksio-app"
REDIS = "fluksio-redis"
API = "fluksio-api"
BROKER = "fluksio-app-mosquitto-1"
DB = "fluksio-db"
#: The only containers this harness may ever name. The first three are the
#: chaos targets; the database is read and cleaned up, never stopped.
ALLOWED = frozenset({REDIS, API, BROKER, DB})
ALLOWED = frozenset({REDIS, API, BROKER})
class HostSafety(RuntimeError):
@@ -805,25 +805,22 @@ def clear_redis() -> None:
def clear_observability() -> None:
"""The collector wrote rollups, runs and events for the soak flows."""
names = "', '".join(FLOWS)
statement = "; ".join(
f"DELETE FROM {table} WHERE flow IN ('{names}')"
for table in ("metric_minute", "flow_run", "engine_event")
names = ", ".join(repr(flow) for flow in FLOWS)
script = "\n".join(
[
"import sqlite3",
"db = sqlite3.connect('/data/fluksio.db')",
*(
f"db.execute('DELETE FROM {table} WHERE flow IN ({names})')"
for table in ("metric_minute", "flow_run", "engine_event")
),
"db.commit()",
]
)
try:
docker(
DB,
"exec",
DB,
"psql",
"-q",
"-U",
settings.POSTGRES_USER,
"-d",
settings.POSTGRES_DB,
"-c",
statement,
)
# Through the api container: the database is a file on its volume, not
# a server of its own.
docker(API, "exec", API, "python", "-c", script)
log.info("deleted observability rows for the soak flows")
except Exception as exc:
log.error("observability rows may be left behind: %s", exc)
-7
View File
@@ -1,7 +0,0 @@
#! /usr/bin/env bash
set -e
set -x
python -m fluksio.tests_pre_start
bash scripts/test.sh "$@"