Bound the work-queue durability claim to the appendfsync window
Redis runs with appendfsync everysec, so an unclean stop loses up to a second of journaled entries. Name that where the docstrings promised a crash costs nothing, and state the setting and its trade-off on the configuration page. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CL9zvnnvcp1mvA8o7impxk
This commit is contained in:
@@ -1522,7 +1522,8 @@ class Pipeline:
|
|||||||
|
|
||||||
With a work queue attached the event is journaled and the caller
|
With a work queue attached the event is journaled and the caller
|
||||||
returns immediately — that is the path every external trigger takes, so
|
returns immediately — that is the path every external trigger takes, so
|
||||||
a crash mid-cascade loses nothing. Interactive callers (a manual run, a
|
a crash mid-cascade costs at most the last second of journaled work
|
||||||
|
rather than the whole wave. Interactive callers (a manual run, a
|
||||||
draft preview) pass ``durable=False`` and get the old synchronous
|
draft preview) pass ``durable=False`` and get the old synchronous
|
||||||
behaviour, because they are waiting for the result.
|
behaviour, because they are waiting for the result.
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,10 @@ poll — becomes a work item before anything runs. The item is journaled first
|
|||||||
and acknowledged only once the wave it started has quiesced, so an engine that
|
and acknowledged only once the wave it started has quiesced, so an engine that
|
||||||
dies mid-cascade picks the work up again on the way back rather than losing it.
|
dies mid-cascade picks the work up again on the way back rather than losing it.
|
||||||
|
|
||||||
|
Redis journals with ``appendfsync everysec``, so the bound is the last second:
|
||||||
|
an unclean stop of Redis or of its host loses up to a second of entries. An
|
||||||
|
engine that dies while Redis lives loses none of them.
|
||||||
|
|
||||||
Two implementations: Redis Streams, which is what makes the above true, and an
|
Two implementations: Redis Streams, which is what makes the above true, and an
|
||||||
in-memory one for tests and for running without Redis, where "durable" degrades
|
in-memory one for tests and for running without Redis, where "durable" degrades
|
||||||
honestly to "not".
|
honestly to "not".
|
||||||
|
|||||||
@@ -54,7 +54,13 @@ An instance that already has a managed venv keeps it on upgrade under
|
|||||||
| `REDIS_PORT` | `6379` | |
|
| `REDIS_PORT` | `6379` | |
|
||||||
|
|
||||||
Flow state is the last value of every message, node memory, and the run queue.
|
Flow state is the last value of every message, node memory, and the run queue.
|
||||||
Redis here is persistence, not a cache. Run it with append-only persistence on.
|
Redis here is persistence, not a cache. Run it with append-only persistence on
|
||||||
|
(`--appendonly yes`, which the compose stack sets).
|
||||||
|
|
||||||
|
That writes with Redis's default `appendfsync everysec`, so an unclean stop of
|
||||||
|
Redis or of its host loses up to a second of queued work. `appendfsync always`
|
||||||
|
closes the window and cuts engine throughput to about a third, so the default
|
||||||
|
stands.
|
||||||
|
|
||||||
## Identity and access
|
## Identity and access
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user