Bound the work-queue durability claim to the appendfsync window
Redis runs with appendfsync everysec, so an unclean stop loses up to a second of journaled entries. Name that where the docstrings promised a crash costs nothing, and state the setting and its trade-off on the configuration page. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CL9zvnnvcp1mvA8o7impxk
This commit is contained in:
@@ -1522,7 +1522,8 @@ class Pipeline:
|
||||
|
||||
With a work queue attached the event is journaled and the caller
|
||||
returns immediately — that is the path every external trigger takes, so
|
||||
a crash mid-cascade loses nothing. Interactive callers (a manual run, a
|
||||
a crash mid-cascade costs at most the last second of journaled work
|
||||
rather than the whole wave. Interactive callers (a manual run, a
|
||||
draft preview) pass ``durable=False`` and get the old synchronous
|
||||
behaviour, because they are waiting for the result.
|
||||
|
||||
|
||||
@@ -5,6 +5,10 @@ poll — becomes a work item before anything runs. The item is journaled first
|
||||
and acknowledged only once the wave it started has quiesced, so an engine that
|
||||
dies mid-cascade picks the work up again on the way back rather than losing it.
|
||||
|
||||
Redis journals with ``appendfsync everysec``, so the bound is the last second:
|
||||
an unclean stop of Redis or of its host loses up to a second of entries. An
|
||||
engine that dies while Redis lives loses none of them.
|
||||
|
||||
Two implementations: Redis Streams, which is what makes the above true, and an
|
||||
in-memory one for tests and for running without Redis, where "durable" degrades
|
||||
honestly to "not".
|
||||
|
||||
@@ -54,7 +54,13 @@ An instance that already has a managed venv keeps it on upgrade under
|
||||
| `REDIS_PORT` | `6379` | |
|
||||
|
||||
Flow state is the last value of every message, node memory, and the run queue.
|
||||
Redis here is persistence, not a cache. Run it with append-only persistence on.
|
||||
Redis here is persistence, not a cache. Run it with append-only persistence on
|
||||
(`--appendonly yes`, which the compose stack sets).
|
||||
|
||||
That writes with Redis's default `appendfsync everysec`, so an unclean stop of
|
||||
Redis or of its host loses up to a second of queued work. `appendfsync always`
|
||||
closes the window and cuts engine throughput to about a third, so the default
|
||||
stands.
|
||||
|
||||
## Identity and access
|
||||
|
||||
|
||||
Reference in New Issue
Block a user