Keep the engine's state in SQLite, not Postgres

One process owns this database — the image has run a single uvicorn
worker for that reason since the four-engines bug — so a file beside the
flows is the honest shape for it, and it is what lets `fluksio serve`
need no infrastructure at all. Live values, node execution and the work
queue never came here anyway; what does is a rollup a minute at a time,
a row per cascade and the run history, and WAL keeps the readers going
while that one writer works.

DATA_DIR is now the one setting that moves everything an installation
keeps; the rest derive from it and the images still spell theirs out.
The schema is prepared in-process at startup, so the prestart service is
gone, and the ten Postgres-only revisions collapse into one portable
baseline.

Three things only worked because psycopg was casting for us: a token's
subject arriving as a string where the column is a UUID, `greatest`, and
`date_bin`. The timestamps needed a column type of their own — SQLite
stores no offset, and a naive datetime read back either raises against an
aware `now` or serialises as local time.

Postgres stays in the stack only for Umami, behind the analytics profile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-21 22:19:45 +02:00
co-authored by Claude Opus 5
parent f8abd91fc0
commit 73eeec29b1
51 changed files with 841 additions and 1089 deletions
+4 -4
View File
@@ -19,7 +19,7 @@ from datetime import datetime, timedelta, timezone
from typing import Any
from sqlalchemy import delete, func, update
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.dialects.sqlite import insert
from sqlmodel import Session, col
from fluksio.core.config import settings
@@ -354,9 +354,9 @@ class MetricsCollector:
for name in SUMMED
}
| {
name: func.greatest(
col(getattr(MetricBucket, name)), new[name]
)
# `max` of two values, not the aggregate: SQLite's
# scalar form, which is what `greatest` is elsewhere.
name: func.max(col(getattr(MetricBucket, name)), new[name])
for name in MAXIMA
},
)
+2 -4
View File
@@ -39,7 +39,7 @@ from datetime import datetime, timedelta, timezone
from typing import Any
from sqlalchemy import update
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.dialects.sqlite import insert as upsert
from sqlmodel import Session, col, select
from fluksio.core.db import engine as db_engine
@@ -258,9 +258,7 @@ class MetricSink:
def _write(self, rows: list[RunMetric]) -> None:
try:
with Session(db_engine) as session:
statement = pg_insert(RunMetric).values(
[row.model_dump() for row in rows]
)
statement = upsert(RunMetric).values([row.model_dump() for row in rows])
session.exec(
statement.on_conflict_do_update(
index_elements=["run_id", "name", "step"],
+1 -1
View File
@@ -42,7 +42,7 @@ COMPILE_TIMEOUT = 60.0
#: Environment the worker is not given. ``SECRET_KEY`` decrypts every stored
#: secret, not only the ones bound to the node asking.
ENV_DENY_PREFIXES = ("POSTGRES_", "FIRST_SUPERUSER", "SENTRY_DSN")
ENV_DENY_PREFIXES = ("DATABASE_URL", "FIRST_SUPERUSER", "SENTRY_DSN")
ENV_DENY_WORDS = ("PASSWORD", "SECRET", "TOKEN")