c09095d369363765818c34d51d1fbe57bb6ca4a4
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  stroblmeandClaude Opus 5
|
57eace2226 |
Bound what the API accepts, and close the holes the audit found
**SQLite is the database, and now says so.** `metric_minute` and every run
table are written with `sqlalchemy.dialects.sqlite.insert(...)
.on_conflict_do_update` and with `max(a, b)`, neither of which another
dialect has — so pointing `DATABASE_URL` at Postgres migrated cleanly,
served, logged in, and then lost every observability flush into the
collector's hold buffer and failed every run. It refuses at startup
instead. (The Postgres in the compose stack is Umami's; the engine's own
database has been a file beside the flows since 2026-08-21.)
**Every integer query parameter is bounded.** The caps were written as
`min(limit, 500)`, which a negative walks straight through — `?limit=-1`
compiles to `LIMIT -1` and SQLite returns the whole table. Ten signatures,
now `Query(ge=…, le=…)`. `hours=0` still means an hour, which
`_window_hours` was already deliberate about.
**Exports are capped at 10 000 runs** and say so with `X-Truncated`. The
filters bounded a sensible request and nothing bounded an unfiltered one,
which read every row into memory before a byte was streamed. `_series`
resolves cached curves in two queries rather than a `Run` lookup and a
`RunMetric` query per restored node — a comparison of twenty runs was
calling that twenty times over.
**`PUT /artifacts` has a size limit** (`MAX_ARTIFACT_BYTES`, 2 GiB, 0 to
disable), checked against `Content-Length` and again against the stream for
a chunked body, and its writes moved off the event loop.
**`/observability/timeseries` takes `since`/`until`**, the same window
`/runs` and `/events` take, capped at 2000 points — `hours=720&bucket_s=60`
was 43 200 of them in one array. It is also what a dragged chart needs to
re-fetch at its own resolution rather than magnifying buckets it has.
**Composite indexes** for the three list screens: `run(flow, created_at)`
and `(status, created_at)`, `flow_run(flow, started_at)`,
`engine_event(type, ts)`. Every index was single-column, so SQLite picked
one and sorted the rest by hand. Verified against a copy of a live database
(250k `flow_run` rows): the planner takes all four.
**Redis clients have socket timeouts.** A Redis that stops answering
without closing the connection hung the caller until the kernel gave up —
including `/utils/health/`, whose job is to notice.
**The panels file is written under one lock.** `save_panels` and
`unpair_panel` are both read-modify-write, and a save that read before an
unpair wrote put the old nonce back — silently un-revoking a screen that
had just been unpaired. The nonce carry-forward was written to make that
impossible; the gap between its read and its write is where it happened.
**Startup releases what it acquired.** Everything past `event_bus.bind`
registers how to close itself and the `finally` walks that list backwards;
a failure part-way through used to reach none of the shutdown steps and
leave the worker pool's subprocesses and every background task behind —
under `--reload`, once per bad edit. `modules.reconcile` moved into the
background: `uv` gets five minutes twice over, the healthcheck allows
eighty seconds, and the autoheal restarted the container before it could
finish installing.
`delete_run` takes SQLite's write lock up front (`core.db.writing`) rather
than upgrading a deferred transaction and losing to whichever flush
committed in between. `modules.sync` is serialised — two applies mutated
one venv at once. The proxied-call and stream dicts are bounded, and a
reused id cancels its predecessor instead of dropping the reference.
Security, found in passing and small enough to fix here:
- **`/secrets/` required only a signed-in user.** The names alone say what
this installation talks to, and `PUT /{name}` takes any name, so any
account could overwrite the credential a flow authenticates with.
Superuser now — which `/search` already assumed and said so.
- **`POST /login/access-token` had no rate limit.** Argon2 is deliberately
expensive and the route is unauthenticated and runs in the shared
threadpool. Ten *failed* attempts per address per five minutes; a
successful sign-in spends nothing.
- **a password reset link worked repeatedly for 48 hours.** The token now
carries a digest of the password hash it was minted against, so it stops
verifying once it has set one. No table of spent tokens needed.
- **enrolment accepted `http://`**, sending the claim code and then this
installation's credential in clear. https, or a local address.
- the rate limiter read `request.client.host`, which behind Traefik is the
proxy — so every per-address limit was one global bucket and one caller
could lock out everyone. It reads the forwarded address, and its
bucket table is capped rather than growing one key per address forever.
- SMTP has a timeout and sends after the response, so an unreachable mail
host cannot pin a threadpool worker, and a reply's timing no longer says
whether the address exists.
Test suite: engine-written rows are cleared between modules. A `FlowRun`
left `running` by one module turned up in another's query. Per-test
rollback is not available here — the module-scoped `client` runs the real
lifespan and its collector and run service write through sessions of their
own — so this bounds it where the writes come from. Three consecutive
green runs, orders randomised.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M6hPWS6YEbT1P8LxhhFb2T
|
||
|
stroblmeandClaude Opus 5
|
60757fa7fa |
Style the engine's own logs, notice enrolment while serving, say more in status
Docs / docs (push) Successful in 21s
Playwright Tests / test-playwright (1, 2) (push) Failing after 4m24s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m37s
pre-commit / pre-commit (push) Failing after 3m14s
Test Backend / test-backend (push) Successful in 2m15s
Compose Smoke Test / test-compose (push) Successful in 34s
Playwright Tests / merge-reports (push) Failing after 1m3s
Four things from a testing pass. `fluksio serve` printed its own lines through the root logger, which has no handler and falls back to `INFO:fluksio.cloud.connector:...` — beside uvicorn's aligned output it reads like something went wrong. The engine's loggers and alembic's now use uvicorn's own handler. Named rather than configuring the root: httpx logs every portal call at INFO and none of that is printed today. `fluksio enroll` writes its config from another process, so an engine already serving never learned it had been paired. It now looks for one every few seconds and dials when it appears. `load()` rather than `exists()`, or a file that does not parse would be restarted forever. `fluksio status` says where the installation stands with its portal — never paired, linked, or paired and unreachable, which is the one worth acting on. `--seed` and `--timeout` had no help text at all. Both say what they are for now, and the docs say what a seed is actually for: recorded on the run, part of its input digest, and passed to an input named `seed` when the flow declares one, so the number a run is labelled with is the one the code drew from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69 |
||
|
stroblmeandClaude Fable 5
|
7e4f03369b |
Three fewer things to remember
**The portal link puts itself back up.** It already retried a connection that raised, but a session that ended *cleanly* — a portal restarting, a proxy closing an idle socket — returned normally and went straight back round the loop with no wait at all, so an engine could spin against a portal that was merely saying goodbye politely. Every ending now reconnects on a delay, and the delay turns on whether the attempt got as far as attaching: one that stood up and dropped is a network event and retries at once, one that never stood up waits longer each time. Jittered, so a portal coming back is not met by every installation it serves in the same instant. Ping timeouts are named rather than defaulted, since they are what bounds how long a suspended laptop's dead socket looks alive, and the keepalive task is awaited so the reason a link went reaches the log instead of the garbage collector. **`fluksio enroll <code>`** is the whole command now; hub.fluksio.com is the default and `--portal` names another. The one command run before anything works should not need two flags. **`fluksio run` syncs first.** The reason a run exists is usually the edit before it, so remembering to sync was remembering to do something the computer could do — including the worker refresh, which is what makes an edit to your own package take effect at all. `--no-sync` opts out for a tight loop. That last one needed discovery fixed: it only ever looked at top-level `*.py`, so a repository whose code is in a package — the ordinary shape — found nothing from its own root. It now descends into the packages it holds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Opus 5
|
d4a9406c51 |
Fix the CI gates: Python 3.13, concurrency groups, hook violations
Docs / docs (push) Successful in 49s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m11s
Playwright Tests / test-playwright (2, 2) (push) Failing after 23s
pre-commit / pre-commit (push) Successful in 3m2s
Test Backend / test-backend (push) Successful in 2m22s
Compose Smoke Test / test-compose (push) Failing after 22s
Playwright Tests / merge-reports (push) Canceled after 1s
The gates have never gone green on the new runners. Three separate reasons: - backend/Dockerfile shipped Python 3.10 while the code imports typing.Self and datetime.UTC, so the container exited on import and the suite could not even load its conftest. The image moves to 3.13 and the packages declare >=3.12, which is the floor the tests actually pass on; ruff's target follows and rewrites timezone.utc and asyncio.TimeoutError accordingly. Relocking drops the 3.10 branch, which bumps FastAPI and so regenerates the SDK. - frontend/README.md had no trailing newline and two dashboard widgets used arbitrary text-[…] sizes. Both are em-relative on purpose, so they move to the inline style the neighbouring ramp already uses. - Every commit left its own run queued: without a concurrency group a runner that was offline for a while works through a backlog nobody reads. A stack that fails to come up now prints its logs before the teardown removes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Opus 5
|
a225b48d0d |
Send the portal the failure count its own tile shows
The portal card's "Failures 24h" had read 0 since failures_24h left HealthSummary: that commit noted nothing read the field, which was true of this repo and not of the portal in index/, where health is an opaque JSON blob with no schema to catch the removal. Restored on the connector rather than in /summary — no screen here reads it, and it is folded from /observability/flows?hours=24, the same endpoint and window the app's Home tile sums, so the two cannot drift apart again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ZeGnqVsf5VHQqvz4HdUhN |
||
|
stroblmeandClaude Opus 5
|
a067ff997d |
Adopt the owner when the enrolling account is gone
An enrolment outlives the database it was made in. Restore a backup, or move to a different one, and the same operator is a different row — `local_user_id` then names nobody, every portal session resolves to no local user, and the machine answers 401 to the only route into it. That is the lockout the welcome frame's owner exists to prevent, and it was prevented only for the case where the row still existed. One superuser is not a guess: it is the account enrolment would have used, so it is adopted and written back. Several is a guess, and this says so instead. Writing it back matters beyond this: a screen paired through the portal borrows the same field, so it was refused for the same reason with no way to say so. Found on the production instance after the move to SQLite, which is exactly this case — DEPLOY.md said to enrol again, and the machine should not need telling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Opus 5
|
10b0ba9e49 |
Add the fluksio CLI: serve, enroll, worker
`pip install fluksio && fluksio serve` on a machine with no Docker, no database and no configuration — which is the case this is for: a node on a cluster where ports cannot be opened. It makes its data directory, its key and an admin account, prints the password once, and serves. Pairing is `fluksio enroll <code> --portal …`, doing what the Settings screen does through the same function, before the engine starts and without one running — a machine nobody can route to has no browser pointed at it either. The portal serves the dashboard, so nothing is served here. Two things had to give way. `fastapi[standard]` pulls a cloud CLI that wants sentry-sdk 2.x while we pinned below it — no pip resolution existed, so the pin is lifted, which the comment beside it had been waiting for and which also lets the Python cap go. And `uv` is now a dependency rather than something to find on PATH: the Modules screen is how a data scientist installs torch, and it was quietly falling back to the engine's own interpreter. The CLI imports nothing from the engine before it has set DATA_DIR — the settings are built on the first import of core.config, and reaching it early put the database in the working directory. There is a test for that now, because the failure is silent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Opus 5
|
961a8f881d |
Keep the engine's state in SQLite, not Postgres
One process owns this database — the image has run a single uvicorn worker for that reason since the four-engines bug — so a file beside the flows is the honest shape for it, and it is what lets `fluksio serve` need no infrastructure at all. Live values, node execution and the work queue never came here anyway; what does is a rollup a minute at a time, a row per cascade and the run history, and WAL keeps the readers going while that one writer works. DATA_DIR is now the one setting that moves everything an installation keeps; the rest derive from it and the images still spell theirs out. The schema is prepared in-process at startup, so the prestart service is gone, and the ten Postgres-only revisions collapse into one portable baseline. Three things only worked because psycopg was casting for us: a token's subject arriving as a string where the column is a UUID, `greatest`, and `date_bin`. The timestamps needed a column type of their own — SQLite stores no offset, and a naive datetime read back either raises against an aware `now` or serialises as local time. Postgres stays in the stack only for Umami, behind the analytics profile. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Opus 5
|
60d7ec81c0 |
Rename the import package app to fluksio
A wheel whose top-level module is `app` collides with anything else in a user's venv, so the package that is about to be published takes the name it is published under. Only the Python package moves; the repo, the Docker WORKDIR and the compose project keep theirs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |