Assistant node, and the Control tab as one conversation (openpanel V5)

run_turn takes its tools, system prompt and tool caller; the chat passes the
MCP tools as before. The assistant node offers one tool per wired target,
reads its other inputs as context, streams its reply a sentence at a time,
warms the provider on `listening` and says so when the provider is not there.
Completions share the per-server HTTP client. wren replies when nothing reads
`unhandled`, stt shows a note when nothing was heard. /ai/control lists one
conversation per entry node with the messages to watch.
This commit is contained in:
Melvin Strobl
2026-10-19 23:23:25 +02:00
parent 4df7bdb057
commit a286cf4ceb
21 changed files with 937 additions and 184 deletions
+12 -3
View File
@@ -41,6 +41,9 @@ SPOKEN = 256
#: How long after its estimated end a sentence still counts as playing: the
#: device fetches it before it starts.
PLAYING_SLACK_S = 1.0
#: The partial for an utterance that came back empty, often a wake word that
#: was not one, or was lost: shown on the caption, never spoken.
NOTHING_HEARD = "Didn't catch that."
def _check(specs: list[MessageSpec], ports: dict[str, DType], node: str) -> None:
@@ -152,8 +155,9 @@ class SttNode(_Speech):
One realtime session per utterance, opened when ``listening`` turns true
or at the first chunk, and committed when it turns false. A clip goes to
the server whole. Either way the node returns at once, since a device's
next frame waits for it, and publishes once the server answers. An
utterance that fails publishes nothing and is logged.
next frame waits for it, and publishes once the server answers. Nothing
heard, or an utterance lost and logged, is a short note on ``partial``
alone.
"""
__slots__ = ("_heard", "_listening", "_lock", "_session", "_tasks")
@@ -268,12 +272,15 @@ class SttNode(_Speech):
words = text.strip() if final else words + text
if final and words:
await asyncio.to_thread(self._say, partial=words, text=words)
elif final:
await asyncio.to_thread(self._say, partial=NOTHING_HEARD)
elif words:
await asyncio.to_thread(self._say, partial=words)
except Exception as exc:
logger.warning(
"'%s' lost an utterance: %s: %s", self.id, type(exc).__name__, exc
)
await asyncio.to_thread(self._say, partial=NOTHING_HEARD)
finally:
# Ended without a commit, by a timeout or a failure: the next chunk
# opens a session of its own rather than feeding this dead one.
@@ -288,9 +295,11 @@ class SttNode(_Speech):
)
except Exception as exc:
logger.warning("'%s' lost a clip: %s: %s", self.id, type(exc).__name__, exc)
return
text = ""
if text.strip():
await asyncio.to_thread(self._say, partial=text.strip(), text=text.strip())
else:
await asyncio.to_thread(self._say, partial=NOTHING_HEARD)
class TtsNode(_Speech):