Give connectors a store that survives restarts, and a way to clear it
`ConnectorNode.store` keeps short strings per connector package in the
engine's database (`connector_state`, migration f8c3e5a91d27), so the nodes of
one package share them and `fluksio serve` needs nothing new: get, set and
delete, at most 64 kB a value and 256 keys a package. A superuser lists the
keys (never the values) and clears them through `/connectors/{node_type}/state`,
or under Stored state in a connector node's panel. Documented in the connector
contract.
This commit is contained in:
@@ -20,7 +20,10 @@ What a connector gets from the base class:
|
||||
reading it;
|
||||
* :meth:`ConnectorNode.save_artifact`, for a device whose readings are bytes —
|
||||
a camera frame, a recorded clip — which travel as a reference on a media
|
||||
port rather than as the message itself.
|
||||
port rather than as the message itself;
|
||||
* :attr:`ConnectorNode.store`, for the few short strings that have to outlive
|
||||
a restart — a certificate pinned on first contact, a pairing token — shared
|
||||
by every node of the connector's package.
|
||||
|
||||
The message schemas and the parameter model are the rest of the contract, and
|
||||
they are the same ones the built-in nodes use. See ``docs/connectors/`` for the
|
||||
@@ -35,6 +38,7 @@ from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from fluksio.flow.connector_store import ConnectorStore, namespace_of
|
||||
from fluksio.flow.nodes import Node
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -92,6 +96,7 @@ class ConnectorNode(Node):
|
||||
"_stop_event",
|
||||
"_last_published",
|
||||
"_artifacts",
|
||||
"_store",
|
||||
"_down",
|
||||
)
|
||||
|
||||
@@ -102,6 +107,7 @@ class ConnectorNode(Node):
|
||||
self._stop_event: asyncio.Event | None = None
|
||||
self._last_published: dict[str, Any] = {}
|
||||
self._artifacts: ArtifactStore | None = None
|
||||
self._store: ConnectorStore | None = None
|
||||
self._down = False
|
||||
|
||||
def _dispatch(self, params: dict[str, Any], **ports: Any) -> dict[str, Any] | None:
|
||||
@@ -166,6 +172,19 @@ class ConnectorNode(Node):
|
||||
[data], name=name, media_type=media_type, volatile=volatile
|
||||
)
|
||||
|
||||
@property
|
||||
def store(self) -> ConnectorStore:
|
||||
"""Short strings this connector keeps across restarts.
|
||||
|
||||
``get(key)``, ``set(key, value)`` and ``delete(key)``, namespaced by the
|
||||
connector's top-level package, so every node of the package sees the
|
||||
same keys. Blocking: call it through ``asyncio.to_thread`` from the
|
||||
event loop. Available before :meth:`start`, unlike the artifact store.
|
||||
"""
|
||||
if self._store is None:
|
||||
self._store = ConnectorStore(namespace_of(type(self)))
|
||||
return self._store
|
||||
|
||||
async def start(self, app: FastAPI | None = None) -> None:
|
||||
self._artifacts = getattr(app.state, "artifact_store", None) if app else None
|
||||
if self.config.poll_interval > 0 and self._stop_event is None:
|
||||
|
||||
Reference in New Issue
Block a user