wren node: untrained targets left out of the prompt, a read beside a held-back set goes on whole
Docs / docs (push) Successful in 27s
Playwright Tests / test-playwright (1, 2) (push) Failing after 47s
Playwright Tests / test-playwright (2, 2) (push) Failing after 36s
pre-commit / pre-commit (push) Failing after 2m55s
Test Backend / test-backend (3.12) (push) Failing after 5m12s
Test Backend / test-backend (3.14) (push) Failing after 5m16s
Compose Smoke Test / test-compose (push) Failing after 36s
Playwright Tests / merge-reports (push) Failing after 2m33s
Docs / docs (push) Successful in 27s
Playwright Tests / test-playwright (1, 2) (push) Failing after 47s
Playwright Tests / test-playwright (2, 2) (push) Failing after 36s
pre-commit / pre-commit (push) Failing after 2m55s
Test Backend / test-backend (3.12) (push) Failing after 5m12s
Test Backend / test-backend (3.14) (push) Failing after 5m16s
Compose Smoke Test / test-compose (push) Failing after 36s
Playwright Tests / merge-reports (push) Failing after 2m33s
This commit is contained in:
@@ -445,11 +445,14 @@ class WrenNode(Node):
|
||||
threshold = 1.0
|
||||
if self.pack is not None:
|
||||
wren = _wren()
|
||||
# A target the model never learnt is left out: shown, it is a line
|
||||
# unlike any in training, which shifts the answers on the others.
|
||||
targets = [
|
||||
wren.Target(
|
||||
**t, value=_unset(t) if current.get(p) is None else current[p]
|
||||
)
|
||||
for p, t in self.targets.items()
|
||||
if p not in self.untrained
|
||||
]
|
||||
# One BLAS thread answers in about half the time for a model
|
||||
# this small. Scoped to the call, so numpy elsewhere in the
|
||||
@@ -470,6 +473,9 @@ class WrenNode(Node):
|
||||
and (call.value is not None or current.get(call.target) is not None)
|
||||
for call in calls
|
||||
)
|
||||
# A reading answered beside a set that is not written would drop the
|
||||
# set unsaid, so the whole command goes on instead.
|
||||
held = not self.cfg.apply and any(call.value is not None for call in calls)
|
||||
specs = {spec.port: spec for spec in [*self.input_ports, *self.output_ports]}
|
||||
writes: dict[str, Any] = {}
|
||||
actions: list[dict[str, Any]] = []
|
||||
@@ -477,7 +483,7 @@ class WrenNode(Node):
|
||||
answers: list[str] = []
|
||||
for call in calls:
|
||||
spec, unit = specs[call.target], self.targets[call.target]["unit"]
|
||||
applied = sure if call.value is None else sure and self.cfg.apply
|
||||
applied = sure and (not held if call.value is None else self.cfg.apply)
|
||||
if call.value is None:
|
||||
value = current.get(call.target)
|
||||
if applied:
|
||||
|
||||
@@ -295,6 +295,8 @@ def test_a_target_the_model_was_not_trained_on_is_never_applied(fake_wren):
|
||||
|
||||
fake_wren.answer = [fake_wren.Call("canopy", "DOWN")]
|
||||
out = node.execute({"house.command": "close the awning"})
|
||||
# The model is not shown the targets it never learnt.
|
||||
assert [t.name for t in fake_wren.targets[0]] == ["living_room_heating"]
|
||||
assert out is not None
|
||||
assert [a["applied"] for a in out["house.actions"]] == [False]
|
||||
assert out["house.unhandled"] == "close the awning"
|
||||
@@ -325,6 +327,24 @@ def test_a_read_is_answered_from_the_target_value(fake_wren):
|
||||
assert out["house.unhandled"] == command
|
||||
|
||||
|
||||
def test_a_read_beside_a_set_not_written_goes_on_whole(fake_wren):
|
||||
node = _node(
|
||||
fake_wren, ("kitchen_light", DType.BOOL), inputs=[OUTDOOR], saying=True
|
||||
)
|
||||
fake_wren.answer = [
|
||||
fake_wren.Call("outdoor_temperature"),
|
||||
fake_wren.Call("kitchen_light", True),
|
||||
]
|
||||
command = "how warm is it outside and turn on the kitchen light"
|
||||
|
||||
out = node.execute({"house.command": command, "house.outdoor": 6.2})
|
||||
|
||||
assert out is not None
|
||||
assert [a["applied"] for a in out["house.actions"]] == [False, False]
|
||||
assert out["house.unhandled"] == command
|
||||
assert "house.reply" not in out or out["house.reply"] == CANT
|
||||
|
||||
|
||||
def test_a_delayed_call_is_held_by_the_work_queue(fake_wren, monkeypatch):
|
||||
node = _node(fake_wren, ("kitchen_light", DType.BOOL), saying=True, apply=True)
|
||||
deferred: list[tuple[dict[str, Any], float]] = []
|
||||
|
||||
@@ -409,7 +409,8 @@ goes unanswered.
|
||||
`reply` says what was set, as "Kitchen light on, living room heating to 21 °C
|
||||
in 5 minutes.", and answers a read from the target's current value, as
|
||||
"Outdoor temperature is 6.2 °C.", with `apply` off too, since a read writes
|
||||
nothing. A read of a target nothing has written yet goes on to `unhandled`. Each record in `actions` contains `message`, `value`,
|
||||
nothing. A read beside a set that `apply` holds back is not answered, so the
|
||||
whole command goes on. A read of a target nothing has written yet goes on to `unhandled`. Each record in `actions` contains `message`, `value`,
|
||||
`applied`, `delay_s`, `call` as wren writes it, such as
|
||||
`set(kitchen_light, true)`, and that call's `confidence`. Keep `apply=false`
|
||||
while checking how the model reads commands.
|
||||
@@ -421,8 +422,8 @@ Redis, and not otherwise.
|
||||
A model knows the targets it was trained on: their names, types, `values`,
|
||||
`range`, descriptions and which are readings. A target wired since, or
|
||||
described otherwise since, is one it never learnt. The node then reads
|
||||
degraded, naming such targets and asking for the model to be retrained, and a
|
||||
call on one is never applied. wren's demo model knows the 20 targets of its
|
||||
degraded, naming such targets and asking for the model to be retrained. They
|
||||
are left out of the model's prompt, so a command on one goes on. wren's demo model knows the 20 targets of its
|
||||
[demo home](https://github.com/stroblme/wren/blob/main/wren/home.json), so
|
||||
it runs without that warning only on ports named and described as there. A
|
||||
`model` the artifact store does not have or cannot read also reads degraded,
|
||||
|
||||
Reference in New Issue
Block a user