Close the nine open SDK tasks: one engine per directory, a tabbed dashboard, re-pairing, run recovery

serve: refuse a second engine for one data directory whatever port it was
asked for, using the pidfile and a token this directory signed. The check
runs before the database is touched and before the credential is written,
which is what left every later CLI call pointing at a dead port.

The terminal dashboard is three tabs (Overview, Runs, Logs) with the toolbar
following the focused pane, the engine's output goes to serve.log rather than
down a pipe, and closing the screen stops both reader threads so the prompt
comes back. It adopts a running engine on every start, so stop/start and
restart work on one it did not start, and a stop waits for the process to be
gone before the next start. Enrolment reports itself in the modal.

enroll: a new claim code replaces the pairing instead of being refused. The
code is redeemed before anything is written, mappings to a portal being left
are cleared, and a running engine redials when the stored enrolment changes.

runs: an engine re-queues the runs left `queued` by the one before it, and
`fluksio retry <id>` / `retry --group <sweep>` submits an interrupted run
again with the same inputs and group, recorded through Run.parent_id.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U9BoNGq6V9MdRWAte7JBuC

backend/tests/api/routes/test_runs.py

@@ -492,6 +492,107 @@ def test_the_seed_is_recorded_the_same_way_however_it_arrived():
             session.commit()
 
 
+def test_a_retry_is_a_new_run_that_names_the_one_it_repeats():
+    """The way back from a run an engine restart interrupted.
+
+    Re-issuing the whole sweep is the blunt version; this keeps the group, so
+    what was missing is filled in rather than run twice.
+    """
+    flow = FlowDef(
+        name="study",
+        mode="batch",
+        inputs=[
+            FlowInput(spec=MessageSpec(name="lr", dtype=DType.FLOAT), initial=0.01)
+        ],
+    )
+    queue = _Collect()
+    service = RunService(controller=_OneFlow(flow), queue=queue)
+    made = []
+    with Session(db_engine) as session:
+        session.add(
+            Run(
+                id="abandoned-1",
+                flow="study",
+                status="abandoned",
+                params={"lr": 0.3},
+                group_id="sweep-9",
+                created_at=datetime.now(UTC),
+            )
+        )
+        session.add(
+            Run(
+                id="going-1",
+                flow="study",
+                status="running",
+                created_at=datetime.now(UTC),
+            )
+        )
+        session.commit()
+    try:
+        again = service.retry("abandoned-1", actor="someone@example.com")
+        made.append(again.id)
+        assert again.parent_id == "abandoned-1"
+        assert (again.params, again.group_id, again.cause) == (
+            {"lr": 0.3},
+            "sweep-9",
+            "retry",
+        )
+        assert queue.items[-1].run_id == again.id
+
+        # A run that has not finished is cancelled, not retried.
+        with pytest.raises(RunRejected):
+            service.retry("going-1")
+        with pytest.raises(RunRejected):
+            service.retry("never-existed")
+    finally:
+        with Session(db_engine) as session:
+            for run in session.exec(
+                select(Run).where(col(Run.id).in_([*made, "abandoned-1", "going-1"]))
+            ).all():
+                session.delete(run)
+            session.commit()
+
+
+def test_runs_left_queued_are_woken_by_the_next_engine():
+    """The row is the record, and the work item is not part of its transaction.
+
+    An in-memory queue loses the item with the process and a stream item
+    nobody claimed is nobody's, so without this a run submitted just before a
+    restart waits for an engine that will never be told about it.
+    """
+    queue = _Collect()
+    service = RunService(controller=_Unusable(), queue=queue)
+    with Session(db_engine) as session:
+        session.add(
+            Run(
+                id="orphan-1",
+                flow="study",
+                status="queued",
+                created_at=datetime.now(UTC),
+            )
+        )
+        session.add(
+            Run(
+                id="finished-1",
+                flow="study",
+                status="ok",
+                created_at=datetime.now(UTC),
+            )
+        )
+        session.commit()
+    try:
+        service._requeue_queued()
+        assert [item.run_id for item in queue.items] == ["orphan-1"]
+        assert queue.items[0].flow == "study"
+    finally:
+        with Session(db_engine) as session:
+            for run in session.exec(
+                select(Run).where(col(Run.id).in_(["orphan-1", "finished-1"]))
+            ).all():
+                session.delete(run)
+            session.commit()
+
+
 def test_a_key_nobody_used_submits_normally(
     client, superuser_token_headers, monkeypatch
 ):