`RedisWorkQueue.stats` read XPENDING, which counts entries delivered to a
consumer and not yet acknowledged — work in progress. Entries sitting in the
stream undelivered were counted nowhere, so an engine hours behind reported
itself idle: on the house, `pending: 4` while the group's lag was 1554.
The group's own `lag` is the missing number. `backlog` now carries it on both
queues (`len(_items)` in memory), leads the health tile, and a sustained one
publishes `engine_degraded` from the timer thread — named with the flow most
of the waiting work belongs to, sampled from the undelivered tail, since that
is the actionable half. It is a summary problem rather than a /utils/health
503: a backlog should not restart the container.
Also drops the keyspace `scan_iter` `stats()` did per poll to count parked
items — it walked every state and idempotency key twice per ten seconds — for
a set the park/unpark path maintains.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BpfSinyCBfjuieikyfMPbf
One process owns this database — the image has run a single uvicorn
worker for that reason since the four-engines bug — so a file beside the
flows is the honest shape for it, and it is what lets `fluksio serve`
need no infrastructure at all. Live values, node execution and the work
queue never came here anyway; what does is a rollup a minute at a time,
a row per cascade and the run history, and WAL keeps the readers going
while that one writer works.
DATA_DIR is now the one setting that moves everything an installation
keeps; the rest derive from it and the images still spell theirs out.
The schema is prepared in-process at startup, so the prestart service is
gone, and the ten Postgres-only revisions collapse into one portable
baseline.
Three things only worked because psycopg was casting for us: a token's
subject arriving as a string where the column is a UUID, `greatest`, and
`date_bin`. The timestamps needed a column type of their own — SQLite
stores no offset, and a naive datetime read back either raises against an
aware `now` or serialises as local time.
Postgres stays in the stack only for Umami, behind the analytics profile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A wheel whose top-level module is `app` collides with anything else in a
user's venv, so the package that is about to be published takes the name
it is published under. Only the Python package moves; the repo, the
Docker WORKDIR and the compose project keep theirs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The milestone is measured on being lighter than Kedro, so make bench-startup
measures it rather than asserting it: 61 ms from submit to result against
1110 ms for kedro run on a pipeline that does the same nothing. The difference
is not orchestration, it is that nothing is booted per run — on a 510-config
sweep that is about nine minutes of pure startup that never happens.
docs/flows/runs.md is the guide: batch flows, sweeps, reporting from inside a
node, artifacts, and the two sanctioned patterns for objects that cannot be
serialized — keep them in one node, or cross at a checkpoint.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
A standalone harness, never part of the test run, that drives a real stack
through the durable path — the webhook, which journals every trigger — and
then stops Redis, kills the engine mid-cascade and restarts the broker
under a live subscription. Latest-value-wins with concurrent cascades means
counter equality is not a promise, so what it asserts is that the queue
drains, that state ends on the last value sent, and that nothing reached
the dead-letter stream.
Every docker verb goes through one helper that checks the compose label
before it acts and refuses anything outside this project, because the
machines this runs on host unrelated services.
Two invariants are deliberately looser than they look. The queue belongs to
the whole stack, so "nothing pending" would be measuring other people's
traffic; the harness waits on the age of the oldest unacknowledged entry
instead, which a stuck item always dominates. And the observability tables
are cleared only after the collector's flush interval has passed, since
deleting a flow publishes an audit event of its own.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017MeiWk3Yq12n2pTvnQWYvt
Trigger hooks are mounted unauthenticated because devices cannot present
a JWT. They now take a secret as a trailing path segment, so a device
needs one URL and no header support. The value never enters the
registered route, only a {secret} template, and is compared with
compare_digest; a mismatch is a bare 404 so the endpoint does not
confirm which hooks exist. Pointing the parameter at the encrypted store
keeps the literal out of flow.json. Hooks without a secret keep working
and now raise a validation issue saying so.
Rotating SECRET_KEY made the stored secrets unreadable for good, since
the Fernet key derives from it. scripts/rotate_secret_key.py re-encrypts
with the new key and refuses if the old one does not decrypt. Now that
recovery exists, an unreadable store fails loudly instead of coming back
empty and leaving flows short of credentials with no visible cause.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkmeRiyeYmVZqJVwuyHq9o
The submodule collapse was only half applied: .gitmodules was deleted but
backend/ and frontend/ were still recorded as gitlinks, so none of their
files were tracked. Replace the gitlinks with the real trees.
Also untrack .env (it carried placeholder secrets) in favour of a tracked
.env.example, drop the committed __pycache__, and narrow the blanket *.png
ignore that would have swallowed design assets.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
stroblmeandClaude Opus 5