Service node: a server kept running on a worker (openpanel V2)
- `service` node type: command, port, worker label, health path, env and
weights (an artifact digest the worker fetches as {weights}). Its flow
starts it; it restarts with backoff, reads degraded while starting or
without a worker and down after three starts that never got healthy, and
publishes http://<worker>:<port> once /health answers 200, "" when it ends.
- Worker protocol 3: a `serve` call, every call in its own process group,
killed when the engine ends the call, the socket drops or the agent gets
SIGTERM. `--service-host` overrides the address the engine sees.
- Services and compiles take no `--parallel` slot, and a flow's detail is
built off the event loop, so a save no longer stalls the engine.
This commit is contained in:
@@ -55,6 +55,7 @@ provides the same agent as `fluksio worker`.
|
||||
| `--python` | worker's interpreter | Python used for node code |
|
||||
| `--parallel` | `1` | concurrent calls |
|
||||
| `--artifact-url` | derived from `--url` | artifact endpoint, if hosted separately |
|
||||
| `--service-host` | address the engine sees | host in a service's url, when the engine reaches this machine elsewhere, such as through a tunnel |
|
||||
| `--cpus` | detected | advertised CPU count |
|
||||
| `--gpus` | detected | advertised GPU count; explicit `0` means none |
|
||||
| `--ram-mb` | detected | advertised memory in MB |
|
||||
@@ -99,6 +100,25 @@ in-flight calls. While executing, the agent sends heartbeats every ten seconds;
|
||||
90 seconds without contact fails the call. Heartbeats do not reset the node's
|
||||
own silence timeout.
|
||||
|
||||
Every call runs in a process group of its own. The agent kills the group when
|
||||
the engine ends the call, when the connection drops and when the agent stops,
|
||||
so nothing a node started keeps running. If the agent itself is killed, its
|
||||
service manager has to clean up: systemd's default `KillMode=control-group`
|
||||
does.
|
||||
|
||||
## Services
|
||||
|
||||
A [Service node](../reference/node-types.md#service) runs a server on a worker
|
||||
for as long as its flow runs, such as a speech model that should stay loaded.
|
||||
It is one long call that holds none of the worker's `--parallel` slots and no
|
||||
resources, so it does not block calls or hold a GPU against a training run.
|
||||
The engine checks the server's health over HTTP and restarts it whenever it
|
||||
stops, and its url names the address the engine sees the worker connect from.
|
||||
Set `--service-host` on the worker where that address is wrong.
|
||||
|
||||
A service's `weights` setting holds an artifact digest. The worker fetches it
|
||||
as node code fetches an artifact and passes the local path as `{weights}`.
|
||||
|
||||
## Resources and artifacts
|
||||
|
||||
Workers report CPU, GPU and memory inventory, and scheduling checks both labels
|
||||
|
||||
Reference in New Issue
Block a user