Commit Graph
9 Commits
Author SHA1 Message Date
stroblme c04e4883c2 MCP: mcp 2.3 (MCPServer, transport arguments on streamable_http_app, bearer read in middleware); tests for the 2026-07-28 revision and the resource check 2026-10-28 09:21:01 +01:00
Melvin Strobl 1a8f6875c4 Assistant confirms a command from the values it set, without a second round
run_turn takes answer_tools: false ends the turn once the tools have
answered. The assistant node uses it and says what was set ("Kitchen light
off, living room light off."), which saves one model round, about 1.1 s on
the P100. A turn without a call still streams the model's own words.

rebuild-frontend keeps the running stack's FRONTEND_HOST, so the token
issuer no longer flips between the LAN address and app.localhost.
2026-10-20 08:26:14 +02:00
Melvin Strobl a286cf4ceb Assistant node, and the Control tab as one conversation (openpanel V5)
run_turn takes its tools, system prompt and tool caller; the chat passes the
MCP tools as before. The assistant node offers one tool per wired target,
reads its other inputs as context, streams its reply a sentence at a time,
warms the provider on `listening` and says so when the provider is not there.
Completions share the per-server HTTP client. wren replies when nothing reads
`unhandled`, stt shows a note when nothing was heard. /ai/control lists one
conversation per entry node with the messages to watch.
2026-10-19 23:23:25 +02:00
Melvin Strobl 4df7bdb057 Speech calls keep one connection per server, opened when listening starts
transcribe and speak share an httpx client per speech server with a 300 s
keep-alive, and run on the engine's loop. stt warms it with GET /v1/models
when an utterance opens, so a reply's first sentence finds the connection
open: a new one over a lossy link cost up to a second (openpanel V4).
2026-10-19 22:06:08 +02:00
Melvin Strobl f50d8b9570 tts speaks MP3: a seventh of WAV's bytes over a slow link to the server
A sentence's length for `append` comes from the Xing header LAME writes into
the first frame. The cache key carries the format, so no WAV is reused.
2026-10-19 21:55:17 +02:00
Melvin Strobl 16383ea6e2 Voice nodes (openpanel V4): stt, tts, wren reply/unhandled, socket publish
- `stt`: one realtime transcription session per utterance on the speech
  server's url, opened at `listening` true or the first chunk, committed at
  false; a whole clip goes to /v1/audio/transcriptions. Returns at once and
  publishes `partial` and `text` when the server answers.
- `tts`: one /v1/audio/speech call per sentence, cached by voice and text in
  node state; a reference is marked `append` while earlier ones still play.
- wren: `reply` says what was applied, `unhandled` carries the prompt when
  nothing was. The install hint names the git pin (PyPI's `wren` is another
  package).
- ai/client: transcribe, speak and transcribe_stream beside stream_completion.
- A device socket publishes scalar `in` ports as {"type": "publish"} text,
  in order with its binary frames.
- Pipeline.publish walks only what its values reach: a node downstream of a
  consumer that published nothing no longer runs again (wren re-acted on
  every microphone chunk).
2026-10-19 21:34:02 +02:00
stroblme 1b5a16f636 Assistant streams its answer token by token
ai/client.py streams the completion and puts the chunks back together
(Assembler), covering OpenAI's piecewise tool-call arguments, Ollama's whole
calls, OpenRouter's comment lines, unnumbered calls and in-stream errors.
The turn emits a delta event per piece and a message per round as before,
and the chat grows the last answer as deltas arrive.
2026-10-15 17:37:56 +02:00
stroblme 8f8330c9e0 Assistant answers render as markdown
react-markdown, lazy-loaded with the first answer (37 kB gzip chunk, main
bundle unchanged). No raw HTML, images dropped, links open in a new tab.
The system prompt asks for lists, inline code and fenced code blocks.
2026-10-15 17:18:12 +02:00
stroblme 7240b7b823 Add the assistant: a chat that drives the MCP tools in-process
A provider set under Settings (OpenAI-compatible; key encrypted in
flow-data/ai.json) turns on an Assistant panel in the sidebar. Each turn runs
the FastMCP tools in-process as the person chatting, streams the rounds over
SSE and keeps one transcript per user. The panel floats on any screen and
shares the editor's lane: a settings panel keeps the top half, the chat the
bottom. The MCP client is now built once, for the endpoint and the chat.
2026-09-16 23:28:23 +02:00