Assistant node, and the Control tab as one conversation (openpanel V5)

run_turn takes its tools, system prompt and tool caller; the chat passes the
MCP tools as before. The assistant node offers one tool per wired target,
reads its other inputs as context, streams its reply a sentence at a time,
warms the provider on `listening` and says so when the provider is not there.
Completions share the per-server HTTP client. wren replies when nothing reads
`unhandled`, stt shows a note when nothing was heard. /ai/control lists one
conversation per entry node with the messages to watch.
This commit is contained in:
Melvin Strobl
2026-10-19 23:23:25 +02:00
parent 4df7bdb057
commit a286cf4ceb
21 changed files with 937 additions and 184 deletions
+53 -3
View File
@@ -398,7 +398,9 @@ applied. Keep `apply=false` while checking how the model interprets commands.
`reply` is one sentence saying what was applied, such as "Kitchen light on."
or "Garage light on in 5 minutes.". When nothing was applied, `unhandled`
carries the prompt instead, for something else to answer.
carries the prompt instead, for something else to answer, such as an
[Assistant](#assistant). When no node reads `unhandled`, `reply` says "Sorry,
I can't do that yet." instead, so a spoken command never goes unanswered.
A call with a delay ("in 10m") is held by the engine's work queue, like the
Debounce node's second value: it survives a restart where the queue is in
@@ -415,6 +417,53 @@ They are reported but cannot receive writes unless wired.
The chat's [Control tab](../code/agents.md#control) sends commands by hand.
### Assistant
**`assistant`** — answer a request with the instance's AI provider, acting only
on what is wired to it.
| Input | Type | Notes |
|---|---|---|
| `prompt` | `str` | the request, such as a Wren node's `unhandled`; it runs the node |
| `listening` | `bool` | optional; turning true opens the connection and loads the model while the request is spoken |
| any other | any | read as context when a request arrives, as `outside_temperature: 12.5` |
| Output | Type | Notes |
|---|---|---|
| `reply` | `str` | the answer, one sentence at a time while the model writes it |
| `actions` | `list` | one record per value set: `message`, `value`, `applied`, `delay_s`, `call` |
| any other | `bool`, `int`, `float` or `str` | a **target**: one tool, named after its port, taking a `value` of its type |
| Setting | Default | Notes |
|---|---|---|
| `model` | empty | the model to ask; empty asks the instance's |
| `extra_body` | `{"temperature": 0, "reasoning_effort": "none", "keep_alive": "30m"}` | added to every request as it is |
It asks the provider set under **Settings → Assistant**. As for Wren, the
wiring is the allowlist: the model is offered the targets and nothing else,
never the tools the chat's assistant uses. Its instructions list the context
inputs and what each target holds now. A target is set the moment the model
calls its tool, and a request may take three rounds of tool calls.
Every request ends in a reply:
- When no tool fits or the request makes no sense, the model says so, as
"Sorry, I can't do that.".
- When no provider is set up, the connection is refused, it takes over 3
seconds to connect or 15 seconds to answer, the reply is "The assistant
isn't available right now." and the node reads degraded. A model that has to
be loaded first can take about 10 seconds.
The default `extra_body` is for Ollama. `temperature` makes the same request
do the same thing each time. `reasoning_effort` turns thinking off through
Ollama's OpenAI-compatible API, which ignores Ollama's own `think`.
`keep_alive` keeps the model loaded for 30 minutes after the last request
instead of 5. Drop a field the provider refuses.
In a voice flow, `stt` writes the prompt of a Wren node, Wren's `unhandled` is
the assistant's prompt, both `reply` outputs go to `tts`, and the device's
`listening` warms the assistant.
### Speech to text
**`stt`** — turn a microphone's audio into text, through a speech server.
@@ -435,8 +484,9 @@ utterance is one realtime transcription session (`{url}/v1/realtime`), opened
when `listening` turns true or at the first chunk and committed when it turns
false. A clip is sent whole to `{url}/v1/audio/transcriptions`. The node
returns at once and publishes when the server answers, since a device's next
chunk waits for it. An utterance that fails, or gets no chunk for 10 seconds,
publishes nothing and is logged.
chunk waits for it. An utterance that comes back empty, fails, or gets no
chunk for 10 seconds publishes "Didn't catch that." on `partial` alone, so a
caption shows it and nothing speaks it. A lost one is logged.
`listening` has to hold a value before a chunk wakes the node. Declare it as a
flow input with the initial value `false` when a device may never send it,