Keep the engine's state in SQLite, not Postgres
One process owns this database — the image has run a single uvicorn worker for that reason since the four-engines bug — so a file beside the flows is the honest shape for it, and it is what lets `fluksio serve` need no infrastructure at all. Live values, node execution and the work queue never came here anyway; what does is a rollup a minute at a time, a row per cascade and the run history, and WAL keeps the readers going while that one writer works. DATA_DIR is now the one setting that moves everything an installation keeps; the rest derive from it and the images still spell theirs out. The schema is prepared in-process at startup, so the prestart service is gone, and the ten Postgres-only revisions collapse into one portable baseline. Three things only worked because psycopg was casting for us: a token's subject arriving as a string where the column is a UUID, `greatest`, and `date_bin`. The timestamps needed a column type of their own — SQLite stores no offset, and a naive datetime read back either raises against an aware `now` or serialises as local time. Postgres stays in the stack only for Umami, behind the analytics profile. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -19,7 +19,7 @@ from datetime import datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import delete, func, update
|
||||
from sqlalchemy.dialects.postgresql import insert
|
||||
from sqlalchemy.dialects.sqlite import insert
|
||||
from sqlmodel import Session, col
|
||||
|
||||
from fluksio.core.config import settings
|
||||
@@ -354,9 +354,9 @@ class MetricsCollector:
|
||||
for name in SUMMED
|
||||
}
|
||||
| {
|
||||
name: func.greatest(
|
||||
col(getattr(MetricBucket, name)), new[name]
|
||||
)
|
||||
# `max` of two values, not the aggregate: SQLite's
|
||||
# scalar form, which is what `greatest` is elsewhere.
|
||||
name: func.max(col(getattr(MetricBucket, name)), new[name])
|
||||
for name in MAXIMA
|
||||
},
|
||||
)
|
||||
|
||||
@@ -39,7 +39,7 @@ from datetime import datetime, timedelta, timezone
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import update
|
||||
from sqlalchemy.dialects.postgresql import insert as pg_insert
|
||||
from sqlalchemy.dialects.sqlite import insert as upsert
|
||||
from sqlmodel import Session, col, select
|
||||
|
||||
from fluksio.core.db import engine as db_engine
|
||||
@@ -258,9 +258,7 @@ class MetricSink:
|
||||
def _write(self, rows: list[RunMetric]) -> None:
|
||||
try:
|
||||
with Session(db_engine) as session:
|
||||
statement = pg_insert(RunMetric).values(
|
||||
[row.model_dump() for row in rows]
|
||||
)
|
||||
statement = upsert(RunMetric).values([row.model_dump() for row in rows])
|
||||
session.exec(
|
||||
statement.on_conflict_do_update(
|
||||
index_elements=["run_id", "name", "step"],
|
||||
|
||||
@@ -42,7 +42,7 @@ COMPILE_TIMEOUT = 60.0
|
||||
|
||||
#: Environment the worker is not given. ``SECRET_KEY`` decrypts every stored
|
||||
#: secret, not only the ones bound to the node asking.
|
||||
ENV_DENY_PREFIXES = ("POSTGRES_", "FIRST_SUPERUSER", "SENTRY_DSN")
|
||||
ENV_DENY_PREFIXES = ("DATABASE_URL", "FIRST_SUPERUSER", "SENTRY_DSN")
|
||||
ENV_DENY_WORDS = ("PASSWORD", "SECRET", "TOKEN")
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user