wren sees every area and reads the command plain
Docs / docs (push) Successful in 41s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m51s
Playwright Tests / test-playwright (2, 2) (push) Failing after 31s
pre-commit / pre-commit (push) Failing after 2m34s
Test Backend / test-backend (3.12) (push) Failing after 5m4s
Test Backend / test-backend (3.14) (push) Failing after 4m43s
Compose Smoke Test / test-compose (push) Failing after 27s
Playwright Tests / merge-reports (push) Failing after 2m31s
Docs / docs (push) Successful in 41s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m51s
Playwright Tests / test-playwright (2, 2) (push) Failing after 31s
pre-commit / pre-commit (push) Failing after 2m34s
Test Backend / test-backend (3.12) (push) Failing after 5m4s
Test Backend / test-backend (3.14) (push) Failing after 4m43s
Compose Smoke Test / test-compose (push) Failing after 27s
Playwright Tests / merge-reports (push) Failing after 2m31s
Shown only the wired areas, the model took "turn on the bathroom light" for the living room's light and "the office light" for the kitchen's, both above 0.93: 5 of the 28 recorded 7B commands switched a wrong light. Every area is shown now, with a light each, so such a call names an unwired device and goes on to `unhandled`. With every area shown, a transcript's "Turn on the kitchen light." fell to 0.80. The command is passed lower case without sentence marks, as the model was trained, which brings it to 0.99. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ljdk7nDaguh9Uhi3zhcSjn
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
5235ac59c2
commit
54bc6b1512
@@ -11,7 +11,7 @@ import pytest
|
||||
|
||||
from fluksio.flow.messages import DType, MessageSpec
|
||||
from fluksio.flow.nodes.base import Node
|
||||
from fluksio.flow.nodes.wren import CANT, WrenNode, _value
|
||||
from fluksio.flow.nodes.wren import CANT, WrenNode, _plain, _value
|
||||
from fluksio.flow.pipeline import Pipeline
|
||||
|
||||
|
||||
@@ -72,6 +72,11 @@ def test_only_the_prompt_wakes_it(fake_wren):
|
||||
}
|
||||
|
||||
|
||||
def test_the_model_reads_the_command_plain():
|
||||
assert _plain("Turn on the kitchen light.") == "turn on the kitchen light"
|
||||
assert _plain("Heat it to 22.5, please!") == "heat it to 22.5 please"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("tool", "args", "dtype", "value"),
|
||||
[
|
||||
@@ -114,6 +119,8 @@ def test_applied_calls_reach_their_targets(fake_wren):
|
||||
("house.living_room_setpoint", True),
|
||||
("", False),
|
||||
]
|
||||
# Areas nothing is wired to are shown too, so a call can name them.
|
||||
assert set(fake_wren.houses[0].areas) == set(fake_wren.AREAS)
|
||||
living = fake_wren.houses[0].areas["living room"]
|
||||
assert [(d.kind, d.state) for d in living] == [
|
||||
("light", "off"),
|
||||
|
||||
Reference in New Issue
Block a user