Voice nodes (openpanel V4): stt, tts, wren reply/unhandled, socket publish
- `stt`: one realtime transcription session per utterance on the speech
server's url, opened at `listening` true or the first chunk, committed at
false; a whole clip goes to /v1/audio/transcriptions. Returns at once and
publishes `partial` and `text` when the server answers.
- `tts`: one /v1/audio/speech call per sentence, cached by voice and text in
node state; a reference is marked `append` while earlier ones still play.
- wren: `reply` says what was applied, `unhandled` carries the prompt when
nothing was. The install hint names the git pin (PyPI's `wren` is another
package).
- ai/client: transcribe, speak and transcribe_stream beside stream_completion.
- A device socket publishes scalar `in` ports as {"type": "publish"} text,
in order with its binary frames.
- Pipeline.publish walks only what its values reach: a node downstream of a
consumer that published nothing no longer runs again (wren re-acted on
every microphone chunk).
This commit is contained in:
+6
-2
@@ -286,8 +286,12 @@ that family, and the whole message at most `MAX_FRAME_BYTES`. The frame is
|
||||
held in memory and published in one step, named after the message and credited
|
||||
to its device node. A frame that does not
|
||||
qualify is dropped, the reason logged once per socket, and the socket stays
|
||||
open. With `http` it uploads with `PUT /artifacts?volatile=true&name=<message>`
|
||||
and publishes with `POST /messages/<message>`. A device that is unpaired or
|
||||
open. A value of one of its `in` ports wired as `bool`, `int`, `float` or `str`
|
||||
goes as the text message `{"type": "publish", "name": "<message>", "value":
|
||||
...}`, taken in turn with the frames, so a microphone's `listening` false
|
||||
arrives after its last chunk. With `http` it uploads with
|
||||
`PUT /artifacts?volatile=true&name=<message>` and publishes with
|
||||
`POST /messages/<message>`. A device that is unpaired or
|
||||
deleted while connected is closed with 1008, at the latest with its next frame.
|
||||
|
||||
### Live view
|
||||
|
||||
+4
-1
@@ -179,7 +179,10 @@ and ARM, so nothing has to be installed into the image.
|
||||
|
||||
openpanel publishes a 16 kHz mono WAV on its `mic` port after its wake word,
|
||||
and plays any `audio` reference sent to its `speaker` port (WAV, MP3 or AAC).
|
||||
Two Function nodes close that loop with local models.
|
||||
The [Speech to text](../reference/node-types.md#speech-to-text) and
|
||||
[Text to speech](../reference/node-types.md#text-to-speech) nodes close that
|
||||
loop through a speech server on a GPU. Without one, two Function nodes close it
|
||||
with local models.
|
||||
[faster-whisper](https://github.com/SYSTRAN/faster-whisper) turns speech into
|
||||
text and [piper](https://github.com/OHF-Voice/piper1-gpl) turns text into
|
||||
speech.
|
||||
|
||||
Reference in New Issue
Block a user