Fluksio/app
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

wren node: targets from the wiring, on wren 2

Browse Source 
Every output is a target and every input besides prompt a reading. A
targets setting gives descriptions, values, ranges and per-target
guards, model takes a trained pack's artifact digest, min_confidence
defaults to the model's threshold. A command is applied whole or not at
all, and targets the model was not trained on read degraded. The home
path (area/kind ports, groups, brightness, temperature answers) is gone.
This commit is contained in:
stroblme
2026-10-24 21:28:42 +02:00
parent 6892651e76
commit 07fa5a2dc2
5 changed files with 745 additions and 863 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/fluksio/flow/nodes/wren.py
+292 -506
View File
@@ -1,18 +1,17 @@
"""Wren: a typed command in, values for the devices wired to it out.
"""Wren: a typed command in, values for the targets wired to it out.
`Wren <https://github.com/stroblme/wren>`_ is a small language model trained on
one job: read a house — its areas, the devices in each and their states — and
answer a command with calls on those devices. This node is the bridge. Its
output ports *are* the devices, named ``<area>_<kind>``, so the wiring is the
`Wren <https://github.com/stroblme/wren>`_ is a small language model that
turns a short command into calls on a list of targets. This node is the
bridge. Its ports *are* the targets: each output is one a command may set,
each input besides ``prompt`` a reading it may ask about. The wiring is the
allowlist: the model can reach nothing that is not wired to the node.
"""
from __future__ import annotations
import re
import time
from collections.abc import Iterable
from typing import Any, NamedTuple
from typing import TYPE_CHECKING, Any
from pydantic import BaseModel, ConfigDict, Field
from threadpoolctl import threadpool_limits
@@ -20,6 +19,9 @@ from threadpoolctl import threadpool_limits
from fluksio.flow.messages import DType, MessageSpec
from fluksio.flow.nodes.base import Node
if TYPE_CHECKING:
from fastapi import FastAPI
def _wren() -> Any:
"""The model package, which is optional.
@@ -33,120 +35,29 @@ def _wren() -> Any:
except ImportError:
# Not `pip install wren`: that name on PyPI is another package.
raise RuntimeError(
"the wren node needs the model: pip install git+https://github.com/"
"stroblme/wren@eae4d6ff9ded68e36624972fd3f4a8f8d210b0c2"
"the wren node needs the model: pip install "
"git+https://github.com/stroblme/wren"
) from None
if not hasattr(wren, "Target"):
raise RuntimeError(
"the wren node needs wren 2, the installed one is older: pip install "
"--upgrade git+https://github.com/stroblme/wren"
)
return wren
#: The last word of a target port, and the kind of device it stands for.
KIND_WORDS = {
"light": "light",
"lamp": "light",
"brightness": "light",
"setpoint": "heat",
"heating": "heat",
"heat": "heat",
"boost": "heat",
"media": "media",
"music": "media",
"tv": "media",
"shutter": "shutter",
"blind": "shutter",
"cover": "shutter",
"water": "water",
"watering": "water",
"sprinkler": "water",
"switch": "switch",
"plug": "switch",
"socket": "switch",
"appliances": "switch",
}
#: What the node reads, and the types each may be. Only the prompt wakes it.
#: Besides these, ``<area>_temperature`` inputs answer "how warm is it".
INPUTS = {
"prompt": {DType.STR},
"presence": {DType.STR},
"outdoor": {DType.FLOAT, DType.INT},
"dark": {DType.BOOL},
}
#: The input the command arrives on, and the only one that wakes the node.
PROMPT = "prompt"
#: The output the calls are reported on, and the two that say what came of
#: the command. Every other output is a device.
#: the command. Every other output is a target.
ACTIONS = "actions"
REPLY = "reply"
UNHANDLED = "unhandled"
#: The reply when nothing was done and nothing reads ``unhandled``.
CANT = "Sorry, I can't do that yet."
#: The output for the house's brightness, 0 to 100, which the model's
#: brightness calls write whatever area they name.
BRIGHTNESS = "brightness"
_TARGET_TYPES = {DType.BOOL, DType.INT, DType.FLOAT, DType.STR}
#: The tools that act on each kind.
_TOOLS = {
"light": "lights",
"media": "media",
"heat": "heating",
"shutter": "shutters",
"water": "watering",
"switch": "switches",
}
#: What a call sets a target to, by the target's type: (bool, number, text).
#: None is a call that type has no value for, which is then not applied.
#: `heating.set` carries its own number.
_VALUES: dict[str, tuple[bool | None, float | None, str | None]] = {
"lights.on": (True, 100, "on"),
"lights.off": (False, 0, "off"),
"media.play": (True, None, "play"),
"media.stop": (False, None, "stop"),
"heating.boost": (True, None, "boost"),
"heating.off": (False, None, "off"),
"shutters.up": (True, 100, "up"),
"shutters.down": (False, 0, "down"),
"shutters.stop": (None, None, "stop"),
"watering.start": (True, None, "on"),
"watering.stop": (False, None, "off"),
"switches.on": (True, 1, "on"),
"switches.off": (False, 0, "off"),
}
#: The brightness calls, and what "dim" and "brighter" do: a step from the
#: brightness now, or from a usual one when none is known. Dimmed lights
#: stay on.
_LEVELS = ("lights.set", "lights.dim", "lights.brighten")
_STEP, _DIMMEST, _BRIGHTEST, _USUAL = 30, 10, 100, 70
#: An applied call, said: "kitchen light on". `heating.set` says its number.
_SAID = {
"lights.on": "light on",
"lights.off": "light off",
"media.play": "media playing",
"media.stop": "media stopped",
"heating.boost": "heating boosted",
"heating.off": "heating off",
"shutters.up": "shutters up",
"shutters.down": "shutters down",
"shutters.stop": "shutters stopped",
"watering.start": "watering on",
"watering.stop": "watering off",
"switches.on": "switch on",
"switches.off": "switch off",
}
#: What every area held in training: a light, and in the living room, which
#: always existed, media and heating besides. A house missing them reads as
#: unfamiliar and the confidence drops with it, so they are filled in; a call
#: on one that nothing is wired to is reported and not applied. Every area is
#: shown, wired or not, or "the bathroom light" lands on a light that is.
_TRAINED = {"living room": ("light", "media", "heat")}
#: wren's areas outside, whose temperature is the ``outdoor`` reading.
_OUTSIDE = {"garden", "patio", "garage", "driveway", "balcony"}
#: Number words, which wren was trained on as digits.
_ONES = (
"zero one two three four five six seven eight nine ten eleven twelve "
@@ -157,154 +68,6 @@ _NUMBERS = {w: n for n, w in enumerate(_ONES)} | {
w: 20 + 10 * n for n, w in enumerate(_TENS)
}
#: A command for a whole group of devices: "turn off all lights".
_GROUP = re.compile(r"\b(all|every|everything|everywhere)\b")
#: What leaves such a command to the model and the assistant: a question, or
#: an exception to the group.
_NOT_GROUP = re.compile(
r"^(is|are|was|were|do|does|did|can|could|what|which|how|why|when|where|who)\b"
r"|\b(except|but|apart from|other than)\b"
)
#: The kind a group command names, by its word. None named is every kind.
_GROUP_KINDS = {
"light": "light",
"lights": "light",
"lamps": "light",
"music": "media",
"media": "media",
"tv": "media",
"shutters": "shutter",
"blinds": "shutter",
"heating": "heat",
"watering": "water",
"sprinklers": "water",
"switches": "switch",
"plugs": "switch",
"appliances": "switch",
}
#: The call a group command makes on each kind, by its action word.
_GROUP_TOOLS = {
"on": {
"light": "lights.on",
"media": "media.play",
"water": "watering.start",
"switch": "switches.on",
},
"off": {
"light": "lights.off",
"media": "media.stop",
"heat": "heating.off",
"water": "watering.stop",
"switch": "switches.off",
},
"open": {"shutter": "shutters.up"},
"up": {"shutter": "shutters.up"},
"raise": {"shutter": "shutters.up"},
"close": {"shutter": "shutters.down"},
"down": {"shutter": "shutters.down"},
"lower": {"shutter": "shutters.down"},
"stop": {
"media": "media.stop",
"shutter": "shutters.stop",
"water": "watering.stop",
},
}
#: A group call, said: "all lights off".
_GROUP_SAID = {
"lights.on": "lights on",
"lights.off": "lights off",
"media.play": "media playing",
"media.stop": "media stopped",
"heating.off": "heating off",
"shutters.up": "shutters up",
"shutters.down": "shutters down",
"shutters.stop": "shutters stopped",
"watering.start": "watering on",
"watering.stop": "watering off",
"switches.on": "switches on",
"switches.off": "switches off",
}
class _Call(NamedTuple):
"""A call the node makes itself, shaped as the model's."""
tool: str
args: dict[str, Any]
delay_s: float | None = None
def _device(port: str, areas: dict[str, str]) -> tuple[str, str] | None:
"""The (area, kind) a port name stands for, or None."""
for slug, area in areas.items():
word = port.removeprefix(f"{slug}_")
if word != port and word in KIND_WORDS:
return area, KIND_WORDS[word]
return None
def _on(value: Any) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, (int, float)):
return value > 0
return str(value).strip().lower() in {"on", "up", "open", "play", "true"}
def _state(kind: str, values: list[Any]) -> str:
"""A device's state as the model reads it, from what its targets hold."""
if kind == "heat":
temps = [
v for v in values if isinstance(v, (int, float)) and not isinstance(v, bool)
]
return f"{round(temps[0])}C" if temps else "20C"
on = bool(values) and _on(values[0])
if kind == "shutter":
return "up" if on else "down"
return "on" if on else "off"
def _value(call: Any, dtype: DType) -> Any:
"""What ``call`` sets a target of this type to, or None."""
if call.tool == "heating.set":
temp = call.args.get("temp")
flag, number, text = (
None,
temp if isinstance(temp, (int, float)) else None,
None,
)
elif call.tool == "lights.set":
level = call.args.get("level")
flag, number, text = (
None,
level if isinstance(level, (int, float)) else None,
None,
)
else:
flag, number, text = _VALUES.get(call.tool, (None, None, None))
if dtype is DType.BOOL:
return flag
if dtype is DType.STR:
return text
if number is None:
return None
return int(number) if dtype is DType.INT else float(number)
def _said(call: Any, area: str) -> str:
"""An applied call as the reply says it: "garage light on in 5 minutes"."""
if call.tool == "heating.set":
said = f"heating to {_value(call, DType.FLOAT):g} degrees"
elif call.tool == "lights.set":
said = f"brightness at {_value(call, DType.FLOAT):g} percent"
else:
said = _SAID.get(call.tool, call.tool)
if call.delay_s:
minutes = round(call.delay_s / 60)
said += f" in {minutes} minute{'s' * (minutes != 1)}"
# The brightness is the house's, not the area's the call names.
return said if call.tool == "lights.set" else f"{area} {said}"
def _plain(command: str) -> str:
"""The command as the model was trained on it: lower case, no sentence marks.
@@ -343,52 +106,125 @@ def _digits(command: str) -> str:
return re.sub(rf"\b({words})\b", lambda m: str(_NUMBERS[m[1]]), command)
def _named(areas: Iterable[str], command: str) -> list[str]:
"""The areas ``command`` names, in the house's order."""
return [a for a in areas if re.search(rf"\b{re.escape(a)}\b", command)]
def _learnt(target: dict[str, Any]) -> tuple[Any, ...]:
"""What a model learns of a target: all but its unit, which it never sees,
and a range as floats, so [5, 30] is (5.0, 30.0)."""
low_high = target.get("range")
return (
target.get("name"),
target.get("type"),
target.get("values"),
None if low_high is None else tuple(float(n) for n in low_high),
target.get("description", ""),
bool(target.get("readonly", False)),
)
def _asked(house: Any, command: str) -> set[str]:
"""The areas a call may land on: those the command names, or else the
user's and the living room.
def _unset(target: dict[str, Any]) -> Any:
"""The value of a target nothing has set yet: off, 0 within its range, or
its first value."""
if target["type"] == "bool":
return False
if target["type"] == "str":
return (target["values"] or [""])[0]
low, high = target["range"] or (0, 0)
number = min(max(0, low), high)
return int(number) if target["type"] == "int" else float(number)
That is how wren picks the areas it shows as the command's own. The few
it adds besides are there to be told apart, and a call on one of them is a
word the model never learned landing on a neighbour: "switch on the
appliances" turned on the bathroom light at 0.97.
"""
return set(_named(house.areas, command)) or {house.user_area, "living room"}
def _typed(value: Any, dtype: DType) -> Any:
"""A call's value as its port's type: wren may write 21 for a float."""
if dtype is DType.INT:
return int(value)
if dtype is DType.FLOAT:
return float(value)
return value
def _spoken(value: Any, unit: str | None) -> str:
"""A value as the reply says it: "on", "down", "21 °C"."""
if isinstance(value, bool):
return "on" if value else "off"
if isinstance(value, str):
return value.lower()
return f"{value:g}" + (f" {unit}" if unit else "")
def _said(port: str, value: Any, unit: str | None, delay_s: float | None) -> str:
"""An applied call as the reply says it: "living room heating to 21 °C in
5 minutes"."""
to = "" if isinstance(value, (bool, str)) else "to "
said = f"{port.replace('_', ' ')} {to}{_spoken(value, unit)}"
if delay_s:
minutes = delay_s >= 60
n = round(delay_s / 60 if minutes else delay_s)
said += f" in {n} {'minute' if minutes else 'second'}{'s' * (n != 1)}"
return said
def _text(call: Any) -> str:
"""The call as the chat shows it, less the device it names."""
args = ", ".join(f"{k}={v}" for k, v in call.args.items() if k != "id")
return f"{call.tool}({args})"
"""The call as wren writes it, less its delay: ``set(kitchen_light, true)``."""
if call.value is None:
return f"read({call.target})"
value = call.value
if isinstance(value, bool):
value = "true" if value else "false"
elif isinstance(value, float):
value = f"{value:g}"
return f"set({call.target}, {value})"
class TargetInfo(BaseModel):
"""What the model is told of a target besides its port, and how far it is
trusted with it."""
model_config = ConfigDict(extra="forbid")
description: str = Field(
default="",
description='More words for the target, which help pick it: "awning over the patio".',
)
values: list[str] | None = Field(
default=None,
description='The only values a str target takes: ["UP", "DOWN", "STOP"].',
)
range: tuple[float, float] | None = Field(
default=None,
description="The lowest and highest value of a number: [5, 30].",
)
unit: str | None = Field(
default=None,
description='Said after a number in the reply: "°C". The model never sees it.',
)
min_confidence: float | None = Field(
default=None,
ge=0,
le=1,
description="The node's min_confidence for this target alone, such as a "
"higher one for a lock.",
)
pass_on: bool = Field(
default=False,
description="Never set by wren: a command on it goes on to unhandled.",
)
class WrenNode(Node):
"""Turn the command on ``prompt`` into values for the devices wired to it.
"""Turn the command on ``prompt`` into values for the targets wired to it.
Inputs: ``prompt`` (the command, and the only one that wakes the node),
``presence`` (the area the user is in), ``outdoor`` (°C), ``dark``
(otherwise read off the clock: before 07:00 or from 18:00) and any
``<area>_temperature``. A question for a temperature is answered from
that reading, or from ``outdoor`` for an area outside.
Inputs: ``prompt``, the command and the only one that wakes the node, and
any number of readings, which a command may ask about but not set.
The command reaches the model with the ``aliases`` applied and its number
words as digits. A command for all of a kind ("turn off all lights",
"everything off in the office") is not the model's: the node applies it
to every wired device of that kind itself.
Outputs: ``actions``, a list of records, one per call, ``reply``, a
sentence saying what was done or what a reading holds, or that nothing
was done when no node reads ``unhandled``, ``unhandled``, the prompt when
nothing was, and any number of targets a command may set.
Outputs: ``actions``, a list of records, one per call and target,
``reply``, a sentence saying what was applied ("Kitchen light on."), or
that nothing was when no node reads ``unhandled``, ``unhandled``, the
prompt when nothing was, ``brightness``, the house's, for "dim the
lights" or "set the brightness to 30%", and any number of targets
named ``<area>_<kind>``. Targets of one area and kind are one device to
the model; each takes what the call means for its type.
A delayed call is held by the engine's work queue, like the Trigger
node's second value.
A target is named by its port. Its type, its current value and its entry
in ``targets`` are what the model is shown. The command reaches the model
with the ``aliases`` applied and its number words as digits. A delayed
call is held by the engine's work queue, like the Debounce node's second
value.
"""
class Params(BaseModel):
@@ -398,19 +234,34 @@ class WrenNode(Node):
default=False,
description="Write to the targets. Off, the node only reports what it would do.",
)
min_confidence: float = Field(
default=0.93,
model: str = Field(
default="",
pattern=r"^(sha256:[0-9a-f]{64})?$",
description=(
"The trained model, an artifact digest. Empty runs wren's demo "
"model, which knows only its demo home."
),
)
min_confidence: float | None = Field(
default=None,
ge=0,
le=1,
description="Below this, nothing is written; the calls are still reported.",
description="Below this, nothing is written; the calls are still "
"reported. Empty takes the model's own threshold.",
)
targets: dict[str, TargetInfo] = Field(
default_factory=dict,
description="By target port: description, values, range, unit, "
'min_confidence and pass_on, as {"canopy": {"values": ["UP", '
'"DOWN", "STOP"], "description": "awning over the patio"}}.',
)
aliases: dict[str, str] = Field(
default_factory=dict,
description="The house's own words, and words heard wrong, in wren's: "
description="Your own words, and words heard wrong, in the model's: "
'{"traverse": "living room light", "page too": "patio"}.',
)
__slots__ = ("cfg", "devices", "readings")
__slots__ = ("cfg", "targets", "pack", "untrained")
# A call is a command to a device; doing it twice is not doing it once.
idempotent = False
@@ -423,69 +274,64 @@ class WrenNode(Node):
name: str | None = None,
):
self.cfg = self.Params.model_validate(params or {})
wren = _wren()
_wren()
inputs = self._normalize_ports(requires)
outputs = self._normalize_ports(provides)
areas = {area.replace(" ", "_"): area for area in wren.AREAS}
#: The area each ``<area>_temperature`` input reads, to its port.
self.readings: dict[str, str] = {}
for spec in inputs:
reading = areas.get(spec.port.removesuffix("_temperature"))
if reading and spec.port.endswith("_temperature"):
self.readings[reading] = spec.port
allowed = INPUTS["outdoor"] if reading else INPUTS.get(spec.port)
if allowed is None:
raise ValueError(
f"'{spec.port}' is not an input of wren. It reads "
f"{', '.join(INPUTS)} and <area>_temperature."
)
if spec.dtype not in allowed:
raise ValueError(
f"'{spec.port}' is {spec.dtype.value}; wren reads it as "
f"{' or '.join(sorted(d.value for d in allowed))}."
)
if not any(spec.port == "prompt" for spec in inputs):
if not any(spec.port == PROMPT for spec in inputs):
raise ValueError("wren needs a 'prompt' input: the command to act on.")
#: (area, kind) to the ports standing for that device.
self.devices: dict[tuple[str, str], list[str]] = {}
ports: list[tuple[MessageSpec, bool]] = []
for spec in outputs:
if spec.port == ACTIONS:
if spec.dtype is not DType.LIST or spec.item_dtype is not DType.RECORD:
raise ValueError(f"'{ACTIONS}' is a list of records.")
continue
if spec.port in (REPLY, UNHANDLED):
elif spec.port in (REPLY, UNHANDLED):
if spec.dtype is not DType.STR:
raise ValueError(f"'{spec.port}' is a str.")
continue
if spec.port == BRIGHTNESS:
if spec.dtype not in (DType.FLOAT, DType.INT):
raise ValueError(f"'{BRIGHTNESS}' is a float or an int.")
continue
device = _device(spec.port, areas)
if device is None:
raise ValueError(
f"'{spec.port}' names no device wren knows. A target is "
f"<area>_<kind>, the area one of {', '.join(areas)}, the kind "
f"one of {', '.join(KIND_WORDS)}."
)
if device[1] not in wren.KINDS: # switches came with wren's s10
raise ValueError(
f"'{spec.port}' is a {device[1]}, which this wren does not know."
)
else:
ports.append((spec, False))
for spec in inputs:
if spec.port != PROMPT:
ports.append((spec, True))
elif spec.dtype is not DType.STR:
raise ValueError(f"'{PROMPT}' is a str.")
#: Each target by its port, as a model's ``meta`` lists the targets it
#: was trained on.
self.targets: dict[str, dict[str, Any]] = {}
for spec, readonly in ports:
if spec.dtype not in _TARGET_TYPES:
raise ValueError(
f"'{spec.port}' is {spec.dtype.value}; a target is bool, int, "
"float or str."
)
self.devices.setdefault(device, []).append(spec.port)
info = self.cfg.targets.get(spec.port, TargetInfo())
self.targets[spec.port] = {
"name": spec.port,
"type": spec.dtype.value,
"values": info.values,
"range": None if info.range is None else list(info.range),
"description": info.description,
"unit": info.unit,
"readonly": readonly,
}
for port, info in self.cfg.targets.items():
kind = self.targets[port]["type"] if port in self.targets else None
if kind is None:
raise ValueError(f"'targets' names '{port}', which no target port is.")
if info.values is not None and kind != "str":
raise ValueError(f"'{port}' is {kind}; only a str target has values.")
if info.range is not None and kind not in ("int", "float"):
raise ValueError(f"'{port}' is {kind}; only a number has a range.")
# Set on start, from the model and the targets it was trained on.
self.pack: dict[str, Any] | None = None
self.untrained: set[str] = set()
super().__init__(
self._act,
# The prompt is the event; the rest is read when it arrives.
requires=[
spec.model_copy(update={"trigger": spec.port == "prompt"})
spec.model_copy(update={"trigger": spec.port == PROMPT})
for spec in inputs
],
provides=outputs,
@@ -493,130 +339,141 @@ class WrenNode(Node):
name=name or "wren",
)
def _house(
self,
wren: Any,
specs: dict[str, MessageSpec],
presence: str | None,
outdoor: float | None,
dark: bool | None,
) -> Any:
"""The house as the model reads it, each device in its current state."""
names = [specs[port].name for ports in self.devices.values() for port in ports]
current = (
self._pipeline.state.get_present(names)
async def start(self, app: FastAPI | None = None) -> None:
store = getattr(app.state, "artifact_store", None) if app else None
self.pack = self._load(store)
def _load(self, store: Any) -> dict[str, Any] | None:
"""The model, read once, or None when there is none to run.
A target it was not trained on, or one described otherwise since, is
a word it never learnt, so a call on it is never applied.
"""
wren = _wren()
path = wren.runtime.MODEL
if self.cfg.model:
path = store.path(self.cfg.model) if store is not None else None
if path is None:
self.report_health(
"degraded",
f"model {self.cfg.model[:19]}… not found, every command goes on",
)
return None
try:
pack: dict[str, Any] = wren.load_pack(path)
except Exception as exc:
self.report_health(
"degraded", f"model unreadable ({exc}), every command goes on"
)
return None
known = {t.get("name"): _learnt(t) for t in pack["meta"].get("targets", [])}
self.untrained = {
port for port, t in self.targets.items() if known.get(port) != _learnt(t)
}
if self.untrained:
self.report_health(
"degraded",
f"the model needs retraining for {', '.join(sorted(self.untrained))}; "
"commands on them go on",
)
else:
self.report_health("ok")
return pack
def _current(self, readings: dict[str, Any]) -> dict[str, Any]:
"""Each target's value now, by port: an output's from the state, a
reading's from its input."""
ports = {
spec.name: spec.port
for spec in self.output_ports
if spec.name and spec.port in self.targets
}
present = (
self._pipeline.state.get_present(list(ports))
if self._pipeline is not None
else {}
)
areas: dict[str, list[Any]] = {}
for area in wren.AREAS:
wired = {kind for (a, kind) in self.devices if a == area}
kinds = wired | set(_TRAINED.get(area, ("light",)))
areas[area] = []
for kind in (k for k in wren.KINDS if k in kinds):
ports = self.devices.get((area, kind), [])
held = [
current[specs[p].name] for p in ports if specs[p].name in current
]
# The id is ours to choose: the model is shown aliases anyway.
areas[area].append(
wren.Device(kind, f"{area}:{kind}", _state(kind, held))
)
clock = time.localtime()
where = str(presence or "").strip().lower().replace("_", " ")
return wren.House(
time=time.strftime("%H:%M", clock),
# The range the model was trained on; past it a number is unfamiliar.
outdoor_c=min(max(float(20 if outdoor is None else outdoor), -5), 30),
dark=clock.tm_hour < 7 or clock.tm_hour >= 18 if dark is None else dark,
user_area=where if where in wren.AREAS else None,
areas=areas,
)
return {ports[name]: value for name, value in present.items()} | readings
def _bar(self, port: str, threshold: float) -> float:
"""The confidence a call on ``port`` needs: the target's own, the
node's, or else the model's."""
info = self.cfg.targets.get(port, TargetInfo())
bars = (info.min_confidence, self.cfg.min_confidence, threshold)
return next(bar for bar in bars if bar is not None)
def _act(
self,
params: dict[str, Any],
prompt: str = "",
presence: str | None = None,
outdoor: float | None = None,
dark: bool | None = None,
**readings: float | None,
self, params: dict[str, Any], prompt: str = "", **readings: Any
) -> dict[str, Any] | None:
if not prompt.strip():
# An empty command is nothing to do, which is what a flow input
# starts as.
return None
wren = _wren()
specs = {spec.port: spec for spec in self.output_ports}
house = self._house(wren, specs, presence, outdoor, dark)
prompt = _aliased(prompt, self.cfg.aliases)
command = _digits(_plain(prompt))
asked = _asked(house, command)
group = self._group(command, house.areas)
if group is not None:
calls, confidence, asked = group, 1.0, set(house.areas)
else:
current = self._current(readings)
calls: list[Any] = []
threshold = 1.0
if self.pack is not None:
wren = _wren()
targets = [
wren.Target(
**t, value=_unset(t) if current.get(p) is None else current[p]
)
for p, t in self.targets.items()
]
# One BLAS thread answers in about half the time for a model
# this small. Scoped to the call, so numpy elsewhere in the
# process keeps its threads.
with threadpool_limits(1):
calls, confidence = wren.act(command, house)
confidence = round(float(confidence), 3)
write = self.cfg.apply and confidence >= self.cfg.min_confidence
calls = wren.act(command, targets, model=self.pack)
threshold = self.pack["meta"].get("threshold", threshold)
# All of a command or none of it: applying a part leaves half of a
# duration ("off for 7 minutes": off now, back on later), or lets the
# assistant do again what was applied. `apply` only holds back writes:
# a reading is answered either way.
sure = all(
call.confidence >= self._bar(call.target, threshold)
and call.target not in self.untrained
and not self.cfg.targets.get(call.target, TargetInfo()).pass_on
# A reading nothing has written yet has no answer.
and (call.value is not None or current.get(call.target) is not None)
for call in calls
)
specs = {spec.port: spec for spec in [*self.input_ports, *self.output_ports]}
writes: dict[str, Any] = {}
actions: list[dict[str, Any]] = []
said: list[str] = []
answers: list[str] = []
for call in calls:
area = str(call.args.get("area", ""))
if (
call.tool == "sensors.temperature"
and area in asked
and confidence >= self.cfg.min_confidence
and (answer := self._temperature(area, outdoor, readings))
):
answers.append(answer)
area, _, kind = str(call.args.get("id", "")).partition(":")
ports = self.devices.get((area, kind), [])
if call.tool.split(".")[0] != _TOOLS.get(kind) or area not in asked:
ports = []
if call.tool in _LEVELS:
ports = []
if BRIGHTNESS in specs:
ports = [BRIGHTNESS]
call = self._leveled(call, specs[BRIGHTNESS].name)
hits = [
(specs[port], value)
for port in ports
if (value := _value(call, specs[port].dtype)) is not None
]
record = {
"message": "",
"value": None,
"applied": False,
"delay_s": call.delay_s,
"call": _text(call),
"confidence": confidence,
}
if not hits:
actions.append(record)
done = False
for spec, value in hits:
applied = write
spec, unit = specs[call.target], self.targets[call.target]["unit"]
applied = sure if call.value is None else sure and self.cfg.apply
if call.value is None:
value = current.get(call.target)
if applied:
words = call.target.replace("_", " ").capitalize()
answers.append(f"{words} is {_spoken(value, unit)}.")
else:
value = _typed(call.value, spec.dtype)
if applied and call.delay_s:
applied = self._pipeline is not None and self._pipeline.defer(
self, self._to_messages({spec.port: value}) or {}, call.delay_s
)
elif applied:
writes[spec.port] = value
actions.append(
record | {"message": spec.name, "value": value, "applied": applied}
)
done = done or applied
if done:
said.append(_said(call, area))
if group is not None and said:
said = self._group_said(group, command, house.areas)
if applied:
said.append(_said(call.target, value, unit, call.delay_s))
actions.append(
{
"message": spec.name,
"value": value,
"applied": applied,
"delay_s": call.delay_s,
"call": _text(call),
"confidence": round(float(call.confidence), 3),
}
)
wired = {spec.port for spec in self.output_ports}
if ACTIONS in wired:
@@ -632,77 +489,6 @@ class WrenNode(Node):
writes[REPLY] = CANT
return writes or None
def _leveled(self, call: Any, name: str) -> _Call:
"""A brightness call as the level it sets: "dim" is 30 below now."""
level = call.args.get("level")
if call.tool != "lights.set":
now = (
self._pipeline.state.get_present([name]).get(name)
if self._pipeline is not None
else None
)
if not isinstance(now, (int, float)) or isinstance(now, bool):
now = _USUAL
if call.tool == "lights.dim":
level = max(now - _STEP, min(now, _DIMMEST))
else:
level = min(now + _STEP, _BRIGHTEST)
elif isinstance(level, (int, float)):
level = min(max(level, 0), _BRIGHTEST)
return _Call("lights.set", {"level": level}, call.delay_s)
def _group(self, command: str, areas: Iterable[str]) -> list[_Call] | None:
"""The calls a command for a whole group means, one per wired device,
or None when it is not one: "turn off all lights" is every light.
Not the model's to answer: it makes one call of it, and "turn off all
lights" switched a living room light on at 0.98.
"""
action = re.search(rf"\b({'|'.join(_GROUP_TOOLS)})\b", command)
if not _GROUP.search(command) or _NOT_GROUP.search(command) or not action:
return None
tools = _GROUP_TOOLS[action[1]]
kinds = {_GROUP_KINDS[w] for w in command.split() if w in _GROUP_KINDS}
named = _named(areas, command)
delay = re.search(r"\bin (\d+) minutes?\b", command)
seconds = int(delay[1]) * 60.0 if delay else None
calls = [
_Call(tools[kind], {"id": f"{area}:{kind}"}, seconds)
for area, kind in self.devices
if kind in tools
and (not kinds or kind in kinds)
and (not named or area in named)
]
return calls or None
@staticmethod
def _group_said(
calls: list[_Call], command: str, areas: Iterable[str]
) -> list[str]:
"""A group command's reply: "all lights off", "lights off in the office"."""
said = list(dict.fromkeys(_GROUP_SAID[call.tool] for call in calls))
named = _named(areas, command)
if named:
said[-1] += " in the " + " and the ".join(named)
else:
said[0] = "all " + said[0]
if calls[0].delay_s:
minutes = round(calls[0].delay_s / 60)
said[-1] += f" in {minutes} minute{'s' * (minutes != 1)}"
return said
def _temperature(
self, area: str, outdoor: float | None, readings: dict[str, float | None]
) -> str | None:
""" "It's 8 degrees outside.", from the reading wired for the area."""
value = readings.get(self.readings.get(area, ""))
where = f"in the {area}"
if value is None and area in _OUTSIDE:
value, where = outdoor, "outside"
if not isinstance(value, (int, float)) or isinstance(value, bool):
return None
return f"It's {round(value)} degrees {where}."
def _passed_on(self) -> bool:
"""Whether a node reads ``unhandled``, such as an assistant."""
name = next((s.name for s in self.output_ports if s.port == UNHANDLED), "")
backend/tests/flow/conftest.py
+35 -22
View File
@@ -1,5 +1,7 @@
from collections.abc import Generator
from dataclasses import dataclass
from pathlib import Path
from types import SimpleNamespace
from typing import Any
import pytest
@@ -12,40 +14,51 @@ def db() -> Generator[None, None, None]:
@dataclass
class _Device:
kind: str
id: str
state: str
class FakeTarget:
name: str
type: str
value: Any
values: list[str] | None = None
range: list[float] | None = None
description: str = ""
unit: str | None = None
readonly: bool = False
@dataclass
class _House:
time: str
outdoor_c: float
dark: bool
user_area: str | None
areas: dict[str, list[_Device]]
class FakeCall:
target: str
value: Any = None
delay_s: float | None = None
confidence: float = 1.0
class FakeWren:
"""The model package's surface, answering whatever a test sets ``answer`` to."""
"""The model package's surface: ``act`` answers whatever a test sets
``answer`` to, and ``load_pack`` returns ``pack``."""
AREAS = [
*("living room", "kitchen", "bathroom", "bedroom", "office"),
*("garden", "patio", "garage", "driveway", "balcony"),
]
KINDS = ["light", "media", "heat", "shutter", "water", "switch"]
Device = _Device
House = _House
Target = FakeTarget
Call = FakeCall
runtime = SimpleNamespace(MODEL=Path("demo.bin"))
def __init__(self) -> None:
self.answer: tuple[list[Any], float] = ([], 1.0)
self.houses: list[_House] = []
self.answer: list[FakeCall] = []
self.pack: dict[str, Any] = {"meta": {"threshold": 0.9, "targets": []}}
self.loaded: list[Path] = []
self.commands: list[str] = []
self.targets: list[list[FakeTarget]] = []
self.models: list[Any] = []
def act(self, command: str, house: _House) -> tuple[list[Any], float]:
self.houses.append(house)
def load_pack(self, path: Path) -> dict[str, Any]:
self.loaded.append(path)
return self.pack
def act(
self, command: str, targets: list[FakeTarget], *, model: Any = None
) -> list[FakeCall]:
self.commands.append(command)
self.targets.append(targets)
self.models.append(model)
return self.answer
backend/tests/flow/test_wren.py
+343 -244
View File
@@ -4,79 +4,96 @@ The model is an optional package, so ``fake_wren`` (conftest) answers in its
place: what the node does with an answer is what is under test here.
"""
from dataclasses import dataclass, field
import asyncio
from types import SimpleNamespace
from typing import Any
import pytest
from fluksio.flow.artifacts import ArtifactStore
from fluksio.flow.messages import DType, MessageSpec
from fluksio.flow.nodes.base import Node
from fluksio.flow.nodes.wren import CANT, WrenNode, _plain, _value
from fluksio.flow.nodes.wren import CANT, WrenNode, _plain
from fluksio.flow.pipeline import Pipeline
@dataclass
class FakeCall:
tool: str
args: dict[str, Any] = field(default_factory=dict)
delay_s: float | None = None
PROMPT = MessageSpec(name="command", port="prompt", dtype=DType.STR)
OUTDOOR = MessageSpec(name="outdoor", port="outdoor_temperature", dtype=DType.FLOAT)
LIGHT = MessageSpec(name="kitchen_light", dtype=DType.BOOL)
def _node(*targets: tuple[str, DType], **params: object) -> WrenNode:
def _node(
fake_wren,
*targets: tuple[str, DType],
inputs=(),
saying=False,
trained=None,
health=None,
app=None,
**params: Any,
) -> WrenNode:
"""A started node, its model trained on its own targets unless ``trained``
lists others."""
node = WrenNode(
requires=[MessageSpec(name="command", port="prompt", dtype=DType.STR)],
requires=[PROMPT, *inputs],
provides=[
MessageSpec(name="actions", dtype=DType.LIST),
*(MessageSpec(name=port, dtype=dtype) for port, dtype in targets),
*(
MessageSpec(name=port, dtype=DType.STR)
for port in ("reply", "unhandled")
if saying
),
],
params=params,
)
node.assign_flow("house", "control")
if health is not None:
node._on_health = lambda _node, status, detail: health.append((status, detail))
fake_wren.pack["meta"]["targets"] = (
list(node.targets.values()) if trained is None else trained
)
asyncio.run(node.start(app))
return node
def test_targets_of_one_area_and_kind_are_one_device(fake_wren):
node = _node(
("living_room_setpoint", DType.FLOAT),
("living_room_boost", DType.BOOL),
("kitchen_lamp", DType.BOOL),
("office_plug", DType.BOOL),
)
assert node.devices == {
("living room", "heat"): ["living_room_setpoint", "living_room_boost"],
("kitchen", "light"): ["kitchen_lamp"],
("office", "switch"): ["office_plug"],
}
def _record(port: str, value: Any, call: str, **more: Any) -> dict[str, Any]:
return {
"message": f"house.{port}",
"value": value,
"applied": True,
"delay_s": None,
"call": call,
"confidence": 1.0,
} | more
def test_a_kind_the_model_does_not_know_is_refused(fake_wren, monkeypatch):
monkeypatch.setattr(fake_wren, "KINDS", ["light"])
with pytest.raises(ValueError, match="switch"):
_node(("office_plug", DType.BOOL))
@pytest.mark.parametrize("port", ["hallway_light", "kitchen_fridge", "light"])
def test_a_port_naming_no_trained_device_is_refused(fake_wren, port):
with pytest.raises(ValueError, match="living_room") as refused:
_node((port, DType.BOOL))
assert port in str(refused.value)
def test_a_prompt_is_required(fake_wren):
with pytest.raises(ValueError, match="prompt"):
WrenNode(provides=[MessageSpec(name="kitchen_light", dtype=DType.BOOL)])
@pytest.mark.parametrize(
("requires", "provides", "params", "refusal"),
[
([], [LIGHT], {}, "prompt"),
([PROMPT], [MessageSpec(name="plan", dtype=DType.JSON)], {}, "plan"),
([PROMPT], [LIGHT], {"targets": {"hall_light": {}}}, "hall_light"),
(
[PROMPT],
[LIGHT],
{"targets": {"kitchen_light": {"values": ["ON"]}}},
"values",
),
([PROMPT], [LIGHT], {"targets": {"kitchen_light": {"range": [0, 1]}}}, "range"),
],
)
def test_a_wiring_wren_cannot_act_on_is_refused(
fake_wren, requires, provides, params, refusal
):
with pytest.raises(ValueError, match=refusal):
WrenNode(requires=requires, provides=provides, params=params)
def test_only_the_prompt_wakes_it(fake_wren):
node = WrenNode(
requires=[
MessageSpec(name="command", port="prompt", dtype=DType.STR),
MessageSpec(name="outside", port="outdoor", dtype=DType.FLOAT),
],
)
node = WrenNode(requires=[PROMPT, OUTDOOR])
assert {spec.port: spec.trigger for spec in node.input_ports} == {
"prompt": True,
"outdoor": False,
"outdoor_temperature": False,
}
@@ -85,83 +102,284 @@ def test_the_model_reads_the_command_plain():
assert _plain("Heat it to 22.5, please!") == "heat it to 22.5 please"
@pytest.mark.parametrize(
("tool", "args", "dtype", "value"),
[
("lights.on", {}, DType.BOOL, True),
("lights.on", {}, DType.FLOAT, 100.0),
("lights.off", {}, DType.STR, "off"),
("heating.set", {"temp": 22}, DType.FLOAT, 22.0),
("heating.set", {"temp": 22}, DType.BOOL, None),
("heating.boost", {}, DType.BOOL, True),
("heating.boost", {}, DType.INT, None),
("switches.off", {}, DType.INT, 0),
("sensors.temperature", {"area": "kitchen"}, DType.FLOAT, None),
],
)
def test_a_call_means_what_its_target_type_can_hold(tool, args, dtype, value):
assert _value(FakeCall(tool, args), dtype) == value
def test_applied_calls_reach_their_targets(fake_wren):
def test_the_targets_reach_the_model_with_their_values_and_settings(fake_wren):
node = _node(
("living_room_setpoint", DType.FLOAT),
("living_room_boost", DType.BOOL),
fake_wren,
("kitchen_light", DType.BOOL),
("living_room_heating", DType.FLOAT),
("kitchen_shutter", DType.STR),
inputs=[OUTDOOR],
targets={
"living_room_heating": {"range": [5, 30], "unit": "°C"},
"kitchen_shutter": {"values": ["UP", "DOWN"]},
"outdoor_temperature": {"description": "temperature outside"},
},
)
Pipeline([node]).state["house.kitchen_light"] = True
node.execute({"house.command": "warm it up", "house.outdoor": 6.2})
assert fake_wren.loaded == [fake_wren.runtime.MODEL]
assert fake_wren.models == [fake_wren.pack]
# Targets nothing has set yet are off, 0 within their range, or their
# first value.
target = fake_wren.Target
assert fake_wren.targets[0] == [
target("kitchen_light", "bool", True),
target("living_room_heating", "float", 5.0, range=[5.0, 30.0], unit="°C"),
target("kitchen_shutter", "str", "UP", values=["UP", "DOWN"]),
target(
"outdoor_temperature",
"float",
6.2,
description="temperature outside",
readonly=True,
),
]
def test_applied_calls_reach_their_targets_and_the_reply_says_so(fake_wren):
node = _node(
fake_wren,
("kitchen_light", DType.BOOL),
("living_room_heating", DType.FLOAT),
saying=True,
apply=True,
targets={"living_room_heating": {"range": [5, 30], "unit": "°C"}},
)
Pipeline([node]).state["house.living_room_setpoint"] = 21.0
fake_wren.answer = (
[
FakeCall("heating.set", {"id": "living room:heat", "temp": 23}),
# The living room's light is filled in for the model, not wired.
FakeCall("lights.on", {"id": "living room:light"}),
# 0.95 clears the model's own threshold, 0.9 here.
fake_wren.answer = [
fake_wren.Call("kitchen_light", True, confidence=0.95),
fake_wren.Call("living_room_heating", 21),
]
out = node.execute({"house.command": "kitchen light on and the heating to 21"})
assert out == {
"house.kitchen_light": True,
"house.living_room_heating": 21.0,
"house.reply": "Kitchen light on, living room heating to 21 °C.",
"house.actions": [
_record("kitchen_light", True, "set(kitchen_light, true)", confidence=0.95),
_record("living_room_heating", 21.0, "set(living_room_heating, 21)"),
],
0.98,
)
out = node.execute({"house.command": "set the heating to 23 degrees"})
assert out is not None
assert out["house.living_room_setpoint"] == 23.0
assert "house.living_room_boost" not in out
assert [(a["message"], a["applied"]) for a in out["house.actions"]] == [
("house.living_room_setpoint", True),
("", False),
]
# Areas nothing is wired to are shown too, so a call can name them.
assert set(fake_wren.houses[0].areas) == set(fake_wren.AREAS)
living = fake_wren.houses[0].areas["living room"]
assert [(d.kind, d.state) for d in living] == [
("light", "off"),
("media", "off"),
("heat", "21C"),
]
}
assert isinstance(out["house.living_room_heating"], float)
@pytest.mark.parametrize(
("params", "confidence"),
[({}, 0.99), ({"apply": True}, 0.5), ({"apply": True, "min_confidence": 1}, 0.99)],
[
({}, 0.99),
# Below the model's own threshold, 0.9 here.
({"apply": True}, 0.85),
({"apply": True, "min_confidence": 1}, 0.99),
],
)
def test_nothing_is_written_unless_applied_and_sure(fake_wren, params, confidence):
node = _node(("kitchen_light", DType.BOOL), **params)
fake_wren.answer = ([FakeCall("lights.on", {"id": "kitchen:light"})], confidence)
node = _node(fake_wren, ("kitchen_light", DType.BOOL), **params)
fake_wren.answer = [fake_wren.Call("kitchen_light", True, confidence=confidence)]
out = node.execute({"house.command": "turn on the lights in the kitchen"})
out = node.execute({"house.command": "turn on the kitchen light"})
assert out == {
"house.actions": [
{
"message": "house.kitchen_light",
"value": True,
"applied": False,
"delay_s": None,
"call": "lights.on()",
"confidence": confidence,
}
_record(
"kitchen_light",
True,
"set(kitchen_light, true)",
applied=False,
confidence=confidence,
)
]
}
@pytest.mark.parametrize(
("params", "applied"),
[
({}, True),
# One call short of its target's bar stops the whole command.
({"targets": {"kitchen_light": {"min_confidence": 0.99}}}, False),
({"targets": {"kitchen_light": {"pass_on": True}}}, False),
({"min_confidence": 0.99}, False),
# A target's own bar is taken over the node's.
(
{
"min_confidence": 0.99,
"targets": {
"kitchen_light": {"min_confidence": 0.9},
"garden_light": {"min_confidence": 0.9},
},
},
True,
),
],
)
def test_a_command_is_applied_whole_or_not_at_all(fake_wren, params, applied):
node = _node(
fake_wren,
("kitchen_light", DType.BOOL),
("garden_light", DType.BOOL),
saying=True,
apply=True,
**params,
)
fake_wren.answer = [
fake_wren.Call("garden_light", True, confidence=0.95),
fake_wren.Call("kitchen_light", True, confidence=0.95),
]
command = "garden and kitchen lights on"
out = node.execute({"house.command": command})
assert out is not None
assert [a["applied"] for a in out.pop("house.actions")] == [applied, applied]
assert out == (
{
"house.garden_light": True,
"house.kitchen_light": True,
"house.reply": "Garden light on, kitchen light on.",
}
if applied
else {"house.unhandled": command, "house.reply": CANT}
)
def test_a_target_the_model_was_not_trained_on_is_never_applied(fake_wren):
trained = [
# As the pack writes it: [5, 30] is the node's (5.0, 30.0), and the
# unit is not the model's to know.
{
"name": "living_room_heating",
"type": "float",
"values": None,
"range": [5, 30],
"description": "",
"unit": None,
"readonly": False,
},
# Described otherwise since.
{
"name": "canopy",
"type": "str",
"values": ["UP", "DOWN", "STOP"],
"range": None,
"description": "awning",
"unit": None,
"readonly": False,
},
]
health: list[tuple[str, str | None]] = []
node = _node(
fake_wren,
("living_room_heating", DType.FLOAT),
("canopy", DType.STR),
("garden_light", DType.BOOL),
saying=True,
trained=trained,
health=health,
apply=True,
targets={
"living_room_heating": {"range": [5, 30], "unit": "°C"},
"canopy": {
"values": ["UP", "DOWN", "STOP"],
"description": "awning over the patio",
},
},
)
((status, detail),) = health
assert status == "degraded"
assert detail is not None and "retraining for canopy, garden_light" in detail
fake_wren.answer = [fake_wren.Call("canopy", "DOWN")]
out = node.execute({"house.command": "close the awning"})
assert out is not None
assert [a["applied"] for a in out["house.actions"]] == [False]
assert out["house.unhandled"] == "close the awning"
def test_a_read_is_answered_from_the_target_value(fake_wren):
node = _node(
fake_wren,
inputs=[OUTDOOR],
saying=True,
# Not applied: a reading is answered anyway, as it writes nothing.
targets={"outdoor_temperature": {"unit": "°C"}},
)
fake_wren.answer = [fake_wren.Call("outdoor_temperature")]
command = "how warm is it outside"
out = node.execute({"house.command": command, "house.outdoor": 6.2})
assert out == {
"house.reply": "Outdoor temperature is 6.2 °C.",
"house.actions": [
_record("outdoor", 6.2, "read(outdoor_temperature)"),
],
}
# Nothing has written the reading yet: there is no answer to give.
out = node.execute({"house.command": command})
assert out is not None
assert out["house.unhandled"] == command
def test_a_delayed_call_is_held_by_the_work_queue(fake_wren, monkeypatch):
node = _node(fake_wren, ("kitchen_light", DType.BOOL), saying=True, apply=True)
deferred: list[tuple[dict[str, Any], float]] = []
def defer(self, node, outputs, seconds):
deferred.append((outputs, seconds))
return True
monkeypatch.setattr(Pipeline, "defer", defer)
Pipeline([node])
fake_wren.answer = [
fake_wren.Call("kitchen_light", False),
fake_wren.Call("kitchen_light", True, delay_s=420),
]
out = node.execute({"house.command": "kitchen light off for 7 minutes"})
assert deferred == [({"house.kitchen_light": True}, 420)]
assert out is not None
assert out["house.kitchen_light"] is False
assert out["house.reply"] == "Kitchen light off, kitchen light on in 7 minutes."
def test_a_trained_model_is_read_from_the_artifact_store(fake_wren, tmp_path):
store = ArtifactStore(tmp_path)
digest = store.put([b"WREN"])["digest"]
app = SimpleNamespace(state=SimpleNamespace(artifact_store=store))
health: list[tuple[str, str | None]] = []
_node(
fake_wren, ("kitchen_light", DType.BOOL), app=app, health=health, model=digest
)
assert fake_wren.loaded == [store.path(digest)]
assert health == [("ok", None)]
# One the store does not have: every command goes on.
node = _node(
fake_wren,
("kitchen_light", DType.BOOL),
saying=True,
app=app,
health=health,
apply=True,
model="sha256:" + "0" * 64,
)
assert health[-1][0] == "degraded"
out = node.execute({"house.command": "kitchen light on"})
assert out == {
"house.actions": [],
"house.unhandled": "kitchen light on",
"house.reply": CANT,
}
assert fake_wren.commands == []
def test_the_model_runs_on_one_blas_thread(fake_wren, monkeypatch):
"""About twice as fast for a model this small; numpy is loaded by `mlp`."""
from threadpoolctl import threadpool_info
@@ -169,163 +387,44 @@ def test_the_model_runs_on_one_blas_thread(fake_wren, monkeypatch):
threads: list[int] = []
act = fake_wren.act
def counted(command: str, house: Any) -> tuple[list[Any], float]:
def counted(command: str, targets: Any, *, model: Any = None) -> Any:
threads.extend(
pool["num_threads"]
for pool in threadpool_info()
if pool["user_api"] == "blas"
)
return act(command, house)
return act(command, targets, model=model)
monkeypatch.setattr(fake_wren, "act", counted)
_node(("kitchen_lamp", DType.BOOL)).execute({"house.command": "lights on"})
node = _node(fake_wren, ("kitchen_light", DType.BOOL))
node.execute({"house.command": "lights on"})
assert threads
assert set(threads) == {1}
def test_a_reply_says_what_was_applied_and_the_rest_is_unhandled(fake_wren):
node = WrenNode(
requires=[MessageSpec(name="command", port="prompt", dtype=DType.STR)],
provides=[
MessageSpec(name="kitchen_light", dtype=DType.BOOL),
MessageSpec(name="reply", dtype=DType.STR),
MessageSpec(name="unhandled", dtype=DType.STR),
],
params={"apply": True},
def test_own_words_and_number_words_reach_the_model_as_its_own(fake_wren):
node = _node(
fake_wren,
("garden_light", DType.BOOL),
saying=True,
apply=True,
aliases={"Page Too": "garden"},
)
node.assign_flow("house", "control")
fake_wren.answer = ([FakeCall("lights.on", {"id": "kitchen:light"})], 0.99)
out = node.execute({"house.command": "turn on the kitchen light"})
assert out == {"house.kitchen_light": True, "house.reply": "Kitchen light on."}
# Nothing wired for the call, or nothing to call at all. Nothing reads
# what is left, so the node says so itself.
asked = "what is the weather tomorrow"
for calls in ([FakeCall("lights.on", {"id": "garage:light"})], []):
fake_wren.answer = (calls, 0.99)
out = node.execute({"house.command": asked})
assert out == {"house.unhandled": asked, "house.reply": CANT}
# An assistant reading it answers instead.
unhandled = MessageSpec(name="house.unhandled", port="unhandled", dtype=DType.STR)
Pipeline([node, Node(lambda params, unhandled: None, requires=[unhandled])])
assert node.execute({"house.command": asked}) == {"house.unhandled": asked}
def test_a_call_on_an_area_the_command_does_not_name_is_not_applied(fake_wren):
"""A word the model never learned lands on a neighbour it was shown."""
node = _node(("bathroom_light", DType.BOOL), apply=True)
fake_wren.answer = ([FakeCall("lights.on", {"id": "bathroom:light"})], 0.97)
out = node.execute({"house.command": "switch on the appliances"})
assert out is not None
assert "house.bathroom_light" not in out
assert [a["applied"] for a in out["house.actions"]] == [False]
def _speaking(*targets: tuple[str, DType], inputs=(), **params: object) -> WrenNode:
node = WrenNode(
requires=[MessageSpec(name="command", port="prompt", dtype=DType.STR), *inputs],
provides=[
*(MessageSpec(name=port, dtype=dtype) for port, dtype in targets),
MessageSpec(name="reply", dtype=DType.STR),
MessageSpec(name="unhandled", dtype=DType.STR),
],
params={"apply": True, **params},
)
node.assign_flow("house", "control")
return node
def test_the_house_words_and_number_words_reach_the_model_as_its_own(fake_wren):
node = _speaking(("patio_light", DType.BOOL), aliases={"Page Too": "patio"})
fake_wren.answer = ([], 0.99)
out = node.execute({"house.command": "Turn on the page too light in five minutes."})
assert fake_wren.commands == ["turn on the patio light in 5 minutes"]
# The assistant reads the house's words too, as they were said otherwise.
assert fake_wren.commands == ["turn on the garden light in 5 minutes"]
# The assistant reads the own words too, as they were said otherwise.
passed = "Turn on the garden light in five minutes."
assert out == {
"house.unhandled": "Turn on the patio light in five minutes.",
"house.actions": [],
"house.unhandled": passed,
"house.reply": CANT,
}
def test_a_command_for_all_of_a_kind_reaches_every_wired_one(fake_wren):
lights = [(f"{area}_light", DType.BOOL) for area in ("kitchen", "office")]
node = _speaking(*lights, ("office_shutter", DType.STR))
out = node.execute({"house.command": "Turn off all lights."})
assert out == {
"house.kitchen_light": False,
"house.office_light": False,
"house.reply": "All lights off.",
}
out = node.execute({"house.command": "turn everything off in the office"})
assert out == {
"house.office_light": False,
"house.reply": "Lights off in the office.",
}
assert fake_wren.commands == [] # the node's, not the model's
# A question or an exception is the model's again.
fake_wren.answer = ([], 0.99)
node.execute({"house.command": "are all lights off"})
node.execute({"house.command": "turn off all lights but the kitchen"})
assert len(fake_wren.commands) == 2
def test_a_question_for_a_temperature_is_answered_from_its_reading(fake_wren):
node = _speaking(
inputs=[
MessageSpec(name="outside", port="outdoor", dtype=DType.FLOAT),
MessageSpec(
name="kitchen_temp", port="kitchen_temperature", dtype=DType.FLOAT
),
]
)
assert node.readings == {"kitchen": "kitchen_temperature"}
readings = {"house.outside": 7.6, "house.kitchen_temp": 21.2}
for area, said in [("garden", "outside"), ("kitchen", "in the kitchen")]:
fake_wren.answer = ([FakeCall("sensors.temperature", {"area": area})], 0.99)
command = f"what is the temperature in the {area}"
out = node.execute({"house.command": command, **readings})
degrees = 8 if area == "garden" else 21
assert out == {"house.reply": f"It's {degrees} degrees {said}."}
# No reading for the bedroom, or a model unsure of it: the assistant's.
for area, confidence in [("bedroom", 0.99), ("kitchen", 0.5)]:
fake_wren.answer = (
[FakeCall("sensors.temperature", {"area": area})],
confidence,
)
command = f"how warm is it in the {area}"
out = node.execute({"house.command": command, **readings})
assert out == {"house.unhandled": command, "house.reply": CANT}
def test_a_brightness_call_sets_the_house_brightness(fake_wren):
node = _speaking(("brightness", DType.INT))
# Whatever area the call names: the brightness is the house's.
for call, level in [
(FakeCall("lights.set", {"id": "kitchen:light", "level": 30}), 30),
(FakeCall("lights.dim", {"id": "kitchen:light"}), 40),
(FakeCall("lights.brighten", {"id": "bedroom:light"}), 100),
]:
fake_wren.answer = ([call], 0.95)
out = node.execute({"house.command": "dim the lights in the kitchen"})
assert out == {
"house.brightness": level,
"house.reply": f"Brightness at {level} percent.",
}
# Nothing wired for it: the assistant's.
node = _speaking(("kitchen_light", DType.BOOL))
fake_wren.answer = ([FakeCall("lights.dim", {"id": "kitchen:light"})], 0.95)
out = node.execute({"house.command": "dim the lights"})
assert out == {"house.unhandled": "dim the lights", "house.reply": CANT}
# An assistant reading `unhandled` answers instead.
unhandled = MessageSpec(name="house.unhandled", port="unhandled", dtype=DType.STR)
Pipeline([node, Node(lambda params, unhandled: None, requires=[unhandled])])
out = node.execute({"house.command": "Turn on the page too light in five minutes."})
assert out == {"house.actions": [], "house.unhandled": passed}
docs/code/agents.md
+3 -3
View File
@@ -77,6 +77,6 @@ which is where a `stt` node writes what a device heard, so typed and spoken
commands are one conversation: the words heard so far, the command, each
reply, and the reported actions with their target message, value and whether
each was applied. A node whose prompt is another's `unhandled` answers in that
node's conversation. Wren's `apply` and `min_confidence` settings decide
whether it writes to devices. Control conversations stay in the browser and
clear on reload.
node's conversation. Wren's `apply`, `min_confidence` and `targets` settings
decide whether it writes to its targets. Control conversations stay in the
browser and clear on reload.
docs/reference/node-types.md
+72 -88
View File
@@ -340,113 +340,97 @@ use a [batch flow](../concepts/runs.md) and a Function node.
### Wren
**`wren`** — turn a typed command into values for the devices wired to it.
**`wren`** — turn a typed command into values for the targets wired to it.
[Wren](https://github.com/stroblme/wren) is an optional local language model
for device commands. Install it in the engine's environment with
`pip install git+https://github.com/stroblme/wren@eae4d6ff9ded68e36624972fd3f4a8f8d210b0c2`.
The `wren` package on PyPI is an unrelated one. Without the model the node
reports a missing dependency.
that turns short commands into calls on a fixed list of targets. Install it in
the engine's environment with
`pip install git+https://github.com/stroblme/wren`. The `wren` package on PyPI
is an unrelated one. Without the model the node reports a missing dependency.
[Wren's own docs](https://stroblme.github.io/wren/) explain how to describe
targets, train a model for them, and what a model can and cannot do.
| Input | Type | Notes |
|---|---|---|
| `prompt` | `str` | the command, and the only input that runs the node |
| `presence` | `str` | the area the user is in; anything else reads as away |
| `outdoor` | `float` or `int` | °C; 20 without it, clamped to −5…30 |
| `dark` | `bool` | without it, dark before 07:00 and from 18:00 |
| `<area>_temperature` | `float` or `int` | °C, read when the command asks how warm the area is |
| any other | `bool`, `int`, `float` or `str` | a **reading**: a target a command may ask about but not set |
The outputs are `actions`, a list of records, `reply`, `unhandled`,
`brightness`, and any number of **targets**. A
target is an output named `<area>_<kind>` and bound to the message its device
listens on: `living_room_setpoint` bound to `home.setpoint` is the living
room's heating. The wiring is the allowlist, so the model can write nothing
that is not wired to the node.
| Output | Type | Notes |
|---|---|---|
| `actions` | `list` | one record per call |
| `reply` | `str` | what was set or read, as "Kitchen light on." |
| `unhandled` | `str` | the prompt, when the node did nothing with it |
| any other | `bool`, `int`, `float` or `str` | a **target** a command may set |
- **area**: `living_room`, `kitchen`, `bathroom`, `bedroom`, `office`,
`garden`, `patio`, `garage`, `driveway`, `balcony`
- **kind**: `light`, `lamp`, `brightness` for a light; `setpoint`, `heating`,
`heat`, `boost` for heating; `media`, `music`, `tv`; `shutter`, `blind`,
`cover`; `water`, `watering`, `sprinkler`; `switch`, `plug`, `socket`,
`appliances` for a switch, which only wren's s10 model knows
A port naming anything else, or a kind the installed model lacks, is refused
when the flow is built. A port is
named after its message unless it is given a name of its own, in the node
panel's port name field or as `{"name": "setpoint", "port":
"living_room_setpoint"}` through the flow API.
Targets of one area and kind are one device to the model, and each takes what
a call means for its type. A target's current value is the state the model
reads.
| Call | `bool` | number | `str` |
|---|---|---|---|
| `lights.on` / `off` | `true` / `false` | `100` / `0` | `on` / `off` |
| `media.play` / `stop` | `true` / `false` | — | `play` / `stop` |
| `heating.set` | — | the temperature | — |
| `heating.boost` / `off` | `true` / `false` | — | `boost` / `off` |
| `shutters.up` / `down` | `true` / `false` | `100` / `0` | `up` / `down` |
| `shutters.stop` | — | — | `stop` |
| `watering.start` / `stop` | `true` / `false` | — | `on` / `off` |
| `switches.on` / `off` | `true` / `false` | `1` / `0` | `on` / `off` |
| `lights.set` / `dim` / `brighten` | — | the level, on `brightness` only | — |
A target is named by its port and bound to the message it stands for:
`living_room_heating` bound to `home.setpoint` is the living room's heating.
Its name, type and current value are what the model is shown, so the wiring
is the allowlist and the model can reach nothing that is not wired to the
node. A target nothing has set yet is shown as off, as 0 within its range, or
as its first value. A port is named after its message unless it is given a
name of its own, in the node panel's port name field or as
`{"name": "setpoint", "port": "living_room_heating"}` through the flow API.
| Setting | Default | Notes |
|---|---|---|
| `apply` | `false` | write to the targets; off, the node only reports |
| `min_confidence` | `0.93` | below this nothing is written. wren's s10 model is less sure of itself, use `0.8` with it |
| `aliases` | empty | the house's own words, or words the recogniser hears wrong, in the model's: `{"traverse": "living room light", "page too": "patio"}` |
| `model` | empty | the trained model, an artifact digest; empty runs wren's demo model |
| `min_confidence` | empty | the confidence a call needs; empty takes the model's own threshold |
| `targets` | empty | by target port, what the model is told of it and how far it is trusted (below) |
| `aliases` | empty | your own words, or words the recogniser hears wrong, in the model's: `{"traverse": "living room light", "page too": "patio"}` |
Each record in `actions` contains `message`, `value`, `applied`, `delay_s`,
`call`, and `confidence`. Unwired calls have an empty `message` and are not
applied. Keep `apply=false` while checking how the model interprets commands.
Every field of a `targets` entry is optional:
`reply` is one sentence saying what was applied, such as "Kitchen light on."
or "Garage light on in 5 minutes.". When nothing was applied, `unhandled`
carries the prompt instead, for something else to answer, such as an
[Assistant](#assistant). When no node reads `unhandled`, `reply` says "Sorry,
I can't do that yet." instead, so a spoken command never goes unanswered.
| Field | Notes |
|---|---|
| `description` | more words for the target, which help pick it: `"awning over the patio"` |
| `values` | the only values a `str` target takes: `["UP", "DOWN", "STOP"]` |
| `range` | the lowest and highest value of a number: `[5, 30]` |
| `unit` | said after a number in the reply, such as `"°C"`; the model never sees it |
| `min_confidence` | this target's own bar, used over the node's, such as a higher one for a lock |
| `pass_on` | wren never sets it, and a command on it goes on to `unhandled` |
`brightness` is the house's, a `float` or `int` from 0 to 100. No wren model
makes brightness calls yet. For one that does, the node writes them to
`brightness` whatever area they name: "set the brightness in the kitchen to 30%" sets the
house to 30. "Dim" and "brighter" move it 30 from its current value (70
when nothing has set it), and a dim stops at 10. The reply is "Brightness at
40 percent.". Without the output these calls go on to `unhandled`.
A target that is not a `bool`, `int`, `float` or `str`, a `targets` entry
naming no target, `values` on a target that is not a `str` and a `range` on
one that is not a number are refused when the flow is built.
A call with a delay ("in 10m") is held by the engine's work queue, like the
Debounce node's second value: it survives a restart where the queue is in
A command is applied whole or not at all. Its calls are written only when
`apply` is on, every call reaches its target's bar, and none is on a
`pass_on` target or on one the model was not trained on. Applying part of a
command would leave half of a duration ("off for 7 minutes" is off now and on
again later), or let the assistant do again what was already done. Otherwise
nothing is written and `unhandled` carries the prompt, for something else to
answer, such as an [Assistant](#assistant). When no node reads `unhandled`,
`reply` says "Sorry, I can't do that yet." instead, so a spoken command never
goes unanswered.
`reply` says what was set, as "Kitchen light on, living room heating to 21 °C
in 5 minutes.", and answers a read from the target's current value, as
"Outdoor temperature is 6.2 °C.", with `apply` off too, since a read writes
nothing. A read of a target nothing has written yet goes on to `unhandled`. Each record in `actions` contains `message`, `value`,
`applied`, `delay_s`, `call` as wren writes it, such as
`set(kitchen_light, true)`, and that call's `confidence`. Keep `apply=false`
while checking how the model reads commands.
A call with a delay ("in 10 minutes") is held by the engine's work queue, like
the Debounce node's second value: it survives a restart where the queue is in
Redis, and not otherwise.
The model sees every area, wired or not, each with at least a light, so "the
bathroom light" is not taken for a light that is wired elsewhere. Such a call
is reported but not applied, and goes on to `unhandled`. So is a call on an
area the command does not name, unless it is the user's area or, with no area
named, the living room: "switch on the appliances" is a word the pinned model
never learned, and it lands on whichever light it was shown. The command is read
in lower case without sentence marks and with number words as digits ("in
five minutes" as "in 5 minutes"), as the model was trained. `aliases` are
applied before, whole words in any case, and `unhandled` carries the prompt
with them.
A model knows the targets it was trained on: their names, types, `values`,
`range`, descriptions and which are readings. A target wired since, or
described otherwise since, is one it never learnt. The node then reads
degraded, naming such targets and asking for the model to be retrained, and a
call on one is never applied. wren's demo model knows the 20 targets of its
[demo home](https://github.com/stroblme/wren/blob/main/wren/home.json), so
it runs without that warning only on ports named and described as there. A
`model` the artifact store does not have or cannot read also reads degraded,
and every command goes on to `unhandled`. A model trained with wren goes into
the store with `PUT /artifacts` (see the [API](../code/api.md#artifacts)).
A command for all of a kind, such as "turn off all lights", "switch every
light on in 5 minutes" or "everything off in the office", is not the
model's. The node applies it to every wired target of that kind, within the
areas it names, and replies "All lights off." or "Lights off in the
office.". A question ("are all lights off") or an exception ("all but the
kitchen") goes to the model as any other command.
A question for an area's temperature ("how warm is it in the kitchen") is
answered from its `<area>_temperature` input, or from `outdoor` for an area
outside, as "It's 8 degrees outside.". Without a reading it goes on to
`unhandled`.
!!! note "What the current model cannot do"
Commands are English, such as "turn on the lights in the kitchen".
The model uses the areas above, one device per kind and area, on/off
actions, and whole-minute delays. It does not dim lights or set colors.
The command is read in lower case without sentence marks and with number words
as digits ("in five minutes" as "in 5 minutes"), as the model was trained.
`aliases` are applied before, whole words in any case, and `unhandled` carries
the prompt with them.
The chat's [Control tab](../code/agents.md#control) sends commands by hand.