Assistant confirms a command from the values it set, without a second round

run_turn takes answer_tools: false ends the turn once the tools have
answered. The assistant node uses it and says what was set ("Kitchen light
off, living room light off."), which saves one model round, about 1.1 s on
the P100. A turn without a call still streams the model's own words.

rebuild-frontend keeps the running stack's FRONTEND_HOST, so the token
issuer no longer flips between the LAN address and app.localhost.
This commit is contained in:
Melvin Strobl
2026-10-20 08:26:14 +02:00
parent a286cf4ceb
commit 1a8f6875c4
5 changed files with 59 additions and 28 deletions
+5 -2
View File
@@ -94,13 +94,16 @@ dev-lan: ## Same, plus the app on http://<host-ip>:$(APP_PORT) (no DNS needed)
# come off the *running* stack rather than from a file, so the rebuilt
# container keeps the hostname and, on a LAN stack, the published port it was
# started with. A published host port is how the lan overlay is recognised,
# because it is the thing only that overlay adds.
# because it is the thing only that overlay adds. Its address is kept too,
# since it is the issuer worker and agent tokens were signed for.
rebuild-frontend: ## Rebuild the app image and restart it (after a UI change)
@domain=$$(docker inspect fluksio-api 2>/dev/null \
| grep -o 'Host(`app\.[^`]*`)' | head -1 | sed 's/Host(`app\.//;s/`)//'); \
port=$$(docker inspect fluksio-api \
--format '{{range $$p, $$c := .NetworkSettings.Ports}}{{range $$c}}{{.HostPort}} {{end}}{{end}}' \
2>/dev/null | awk '{print $$1}'); \
lan=$$(docker inspect fluksio-api --format '{{range .Config.Env}}{{println .}}{{end}}' \
2>/dev/null | sed -n 's/^FRONTEND_HOST=//p'); \
if [ -n "$$port" ]; then \
echo " lan stack detected (published on $$port) — keeping compose.lan.yml"; \
compose="$(COMPOSE_LAN)"; \
@@ -108,7 +111,7 @@ rebuild-frontend: ## Rebuild the app image and restart it (after a UI change)
compose="$(COMPOSE_LOCAL)"; \
fi; \
DOMAIN=$${domain:-$$DOMAIN} ENVIRONMENT=$${ENVIRONMENT:-local} \
APP_PORT=$${port:-$(APP_PORT)} \
APP_PORT=$${port:-$(APP_PORT)} LAN_URL=$$lan \
$$compose up --build -d backend
up: ## Start the production stack
+6 -1
View File
@@ -138,13 +138,16 @@ async def run_turn(
call_tool: ToolCaller,
extra: dict[str, Any] | None = None,
timeout: float | httpx.Timeout = 120.0,
answer_tools: bool = True,
) -> AsyncIterator[dict[str, Any]]:
"""One user turn, yielding progress events as it goes.
A round's text arrives as ``delta`` events while the model writes it, and
once more whole as ``message`` when the round is done. A round is appended
to *messages* only once every tool in it has answered, so a turn cut off
part-way leaves a transcript the provider still accepts.
part-way leaves a transcript the provider still accepts. With
*answer_tools* false the turn ends once its tools have answered, for a
caller that words what they did itself.
"""
messages.append({"role": "user", "content": text})
head = {"role": "system", "content": system}
@@ -193,6 +196,8 @@ async def run_turn(
yield {"type": "tool_result", "name": name, **summarize(result)}
messages.append(assistant)
messages.extend(answers)
if not answer_tools:
return
yield {
"type": "error",
"detail": f"Stopped after {config.max_rounds} rounds of tool calls.",
+27 -12
View File
@@ -6,8 +6,10 @@ offered as one tool named after its port and typed by its message, so the
wiring is the allowlist, as for wren: the MCP authoring tools are never
offered. Every other input is read as context ("outside_temperature: 7.5").
The answer goes out on ``reply`` a sentence at a time while the model writes
it, so a ``tts`` node speaks the first before the last is written.
An answer goes out on ``reply`` a sentence at a time while the model writes
it, so a ``tts`` node speaks the first before the last is written. A change is
confirmed from the values set ("Kitchen light off."), without asking the model
a second time.
"""
from __future__ import annotations
@@ -43,14 +45,12 @@ LISTENING = "listening"
#: Said when there is no provider, or it does not answer.
UNAVAILABLE = "The assistant isn't available right now."
#: Said when the model ends a turn without a word.
DONE, CANT = "Done.", "Sorry, I can't do that."
#: Said when the model ends a turn without a word or a change.
CANT = "Sorry, I can't do that."
#: A server that is not there is given up on after a few seconds, while a
#: model that has to be loaded first takes about ten.
TIMEOUT = httpx.Timeout(15.0, connect=3.0)
#: Rounds of tool calls one request may take.
ROUNDS = 3
#: A target's type, as its tool's parameter.
_SCHEMAS: dict[DType, dict[str, str]] = {
@@ -63,8 +63,8 @@ _SCHEMAS: dict[DType, dict[str, str]] = {
_SYSTEM = """You are the voice of a home: someone spoke to a panel on the wall, \
and you answer out loud.
- To change something, call its tool. The tools are all you can change.
- After a change, say what you did in one short sentence.
- To change something, call its tool without a word: the change is confirmed \
for you. The tools are all you can change.
- If no tool can do what was asked, or you did not understand it, say so in a \
few words, such as "Sorry, I can't do that."
- Answer questions from the readings below.
@@ -74,6 +74,16 @@ few words, such as "Sorry, I can't do that."
_END = re.compile(r"(?<=[.!?])\s+")
def phrase(port: str, value: Any) -> str:
"""A value set, as the reply says it: "kitchen light off"."""
label = port.replace("_", " ")
if isinstance(value, bool):
return f"{label} {'on' if value else 'off'}"
return (
f"{label} to {value:g}" if isinstance(value, float) else f"{label} to {value}"
)
def cut(text: str) -> tuple[list[str], str]:
"""The sentences ``text`` has finished, and the one still being written."""
*done, rest = _END.split(text)
@@ -212,9 +222,7 @@ class AssistantNode(Node):
config = ai_config.load()
if config is None or not config.ready:
return None
return dataclasses.replace(
config, model=self.cfg.model or config.model, max_rounds=ROUNDS
)
return dataclasses.replace(config, model=self.cfg.model or config.model)
def _call(self, work: Coroutine[Any, Any, Any]) -> Future[Any]:
"""Run on the engine's loop, where the provider's connection is kept."""
@@ -298,6 +306,7 @@ class AssistantNode(Node):
call_tool=self._caller(actions),
extra=self.cfg.extra_body,
timeout=TIMEOUT,
answer_tools=False,
):
if event["type"] == "delta":
done, rest = cut(rest + event["content"])
@@ -313,9 +322,15 @@ class AssistantNode(Node):
if problem:
logger.warning("'%s' got no answer: %s", self.id, problem)
if not said:
await say(UNAVAILABLE if problem else DONE if actions else CANT)
await say(UNAVAILABLE if problem else self._confirm(actions) or CANT)
return problem
def _confirm(self, actions: list[dict[str, Any]]) -> str:
"""The values set, as one sentence: "Kitchen light off, setpoint to 22."."""
ports = {spec.name: spec.port for spec in self._targets()}
text = ", ".join(phrase(ports[a["message"]], a["value"]) for a in actions)
return text[:1].upper() + text[1:] + "." if text else ""
def _caller(self, actions: list[dict[str, Any]]) -> agent.ToolCaller:
"""Set the target a call names, and record it in ``actions``."""
specs = {spec.port: spec for spec in self._targets()}
+18 -11
View File
@@ -56,7 +56,7 @@ def test_a_sentence_is_cut_once_it_ends():
assert cut("Lights off! Anything else? ") == (["Lights off!", "Anything else?"], "")
def test_only_wired_targets_are_tools_and_each_sentence_goes_out_as_written(
def test_only_wired_targets_are_tools_a_change_is_confirmed_and_an_answer_streamed(
monkeypatch: pytest.MonkeyPatch,
):
log: list[Any] = []
@@ -69,7 +69,7 @@ def test_only_wired_targets_are_tools_and_each_sentence_goes_out_as_written(
_call("delete_flow", "demo"),
_call("living_room_setpoint", 22),
],
["Kitchen light off", " and heating to 22. It is ", "7.5 degrees outside."],
["It is 7.5", " degrees outside. Wear a ", "jacket."],
]
)
@@ -87,25 +87,21 @@ def test_only_wired_targets_are_tools_and_each_sentence_goes_out_as_written(
monkeypatch.setattr(ai_config, "load", lambda: PROVIDER)
monkeypatch.setattr(client, "stream_completion", completion)
monkeypatch.setattr(AssistantNode, "emit", lambda self, values: log.append(values))
node = _node()
out = _ask(_node(), "turn off the kitchen light and make it warmer")
out = _ask(node, "turn off the kitchen light and make it warmer")
tools = {tool["function"]["name"] for tool in seen[0]["tools"]}
assert tools == {"kitchen_light", "living_room_setpoint"}
system = seen[0]["messages"][0]["content"]
assert "outside_temperature: 7.5" in system
assert seen[0]["extra"]["reasoning_effort"] == "none"
answers = [m["content"] for m in seen[1]["messages"] if m["role"] == "tool"]
assert "error" in json.loads(answers[1])
# Set as called, and the first sentence said before the second is written.
# Set as called, and confirmed without asking the model again.
assert len(seen) == 1
assert log == [
{"kitchen_light": False},
{"living_room_setpoint": 22.0},
"Kitchen light off",
" and heating to 22. It is ",
{"reply": "Kitchen light off and heating to 22."},
"7.5 degrees outside.",
{"reply": "It is 7.5 degrees outside."},
{"reply": "Kitchen light off, living room setpoint to 22."},
]
assert out is not None
assert [a["message"] for a in out["voice.actions"]] == [
@@ -113,6 +109,17 @@ def test_only_wired_targets_are_tools_and_each_sentence_goes_out_as_written(
"voice.setpoint",
]
log.clear()
_ask(node, "what is it like outside")
# The first sentence is said before the second is written.
assert log == [
"It is 7.5",
" degrees outside. Wear a ",
{"reply": "It is 7.5 degrees outside."},
"jacket.",
{"reply": "Wear a jacket."},
]
def test_no_provider_or_one_that_does_not_answer_is_said(
monkeypatch: pytest.MonkeyPatch,
+3 -2
View File
@@ -430,7 +430,7 @@ on what is wired to it.
| Output | Type | Notes |
|---|---|---|
| `reply` | `str` | the answer, one sentence at a time while the model writes it |
| `reply` | `str` | what was set, as "Kitchen light off.", or the answer, one sentence at a time while the model writes it |
| `actions` | `list` | one record per value set: `message`, `value`, `applied`, `delay_s`, `call` |
| any other | `bool`, `int`, `float` or `str` | a **target**: one tool, named after its port, taking a `value` of its type |
@@ -443,7 +443,8 @@ It asks the provider set under **Settings → Assistant**. As for Wren, the
wiring is the allowlist: the model is offered the targets and nothing else,
never the tools the chat's assistant uses. Its instructions list the context
inputs and what each target holds now. A target is set the moment the model
calls its tool, and a request may take three rounds of tool calls.
calls its tool. The reply then names what was set ("Living room light on,
living room setpoint to 22."), so the model is asked once per request.
Every request ends in a reply: