Let a worker say what machine it is

A worker reported its labels and nothing about the machine behind them, so the
engine could route a node to a GPU box but not tell whether that box had a GPU
free. Inventory — cores, GPUs, memory — now arrives with the hello frame, and
the run frame carries back what the engine allocated for that call.

Which is protocol 2 on both ends. GPUs are never probed: asking a vendor tool
would make the one dependency two, so a GPU is what the batch job says it was
given or what --gpus says. A worker that reports nothing still attaches and is
scheduled by its label alone.

Two things a job scheduler needs: --max-idle stops a worker started for one job
rather than letting it hold its allocation to the walltime, and a refusal is now
fatal instead of a reconnect loop that reads as a hang in a job's log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A6HeySA27EkGANZN95QySW
This commit is contained in:
2026-08-27 08:29:35 +02:00
co-authored by Claude Opus 5
parent 0ffcabfdb9
commit 1a9753fa9d
7 changed files with 359 additions and 36 deletions
+13 -1
View File
@@ -21,7 +21,12 @@ from pydantic import BaseModel, Field
from fluksio.api.deps import get_current_active_superuser, get_current_user
from fluksio.core import security
from fluksio.flow.remote import PROTOCOL, RemoteWorker, RemoteWorkerHub
from fluksio.flow.remote import (
PROTOCOL,
RemoteWorker,
RemoteWorkerHub,
WorkerInventory,
)
logger = logging.getLogger(__name__)
@@ -40,6 +45,9 @@ class WorkerInfo(BaseModel):
last_seen: float = 0.0
python: str = ""
venv_digest: str = ""
cpus: int = 1
gpus: int = 0
ram_mb: int | None = None
class TokenRequest(BaseModel):
@@ -74,6 +82,9 @@ def read_workers(request: Request) -> Any:
last_seen=worker.last_seen,
python=str(worker.info.get("python") or ""),
venv_digest=str(worker.info.get("venv_digest") or ""),
cpus=worker.inventory.cpus,
gpus=worker.inventory.gpus,
ram_mb=worker.inventory.ram_mb,
)
for worker in _hub(request.app).workers()
]
@@ -163,6 +174,7 @@ async def attach(websocket: WebSocket, token: str = "") -> None:
"python": hello.get("python"),
"venv_digest": hello.get("venv_digest"),
},
inventory=WorkerInventory.from_hello(hello.get("inventory")),
)
hub.attach(worker)
await websocket.send_json({"op": "welcome", "protocol": PROTOCOL, "name": name})