Run node code on the venv Fluksio was installed into
The workflow this serves: make a venv, install what you work with, then `pip install fluksio` into the same one. Building a second environment beside it was exactly wrong — the packages the nodes need are already here, and the Modules screen was asking for them a second time. `NODE_VENV=auto` (the default) adopts that venv. It declines in the three cases where adopting would be wrong: `managed` says otherwise, a managed venv already exists and may hold packages somebody installed on purpose, or the engine is not running from a venv at all. The images set `managed`, since the venv in them holds the app and nothing of anybody else's. An adopted venv is never written to. `uv pip sync` makes a venv hold exactly the manifest, so pointed at somebody's own environment it uninstalls their work and the engine with it — `sync()` refuses outright and `reconcile()` returns before it can be called at startup, which is where that would have happened first. The Modules screen lists what is installed and drops its editor; `pip` is how that environment changes. `fluksio serve` now names the interpreter node code runs on, which is the thing a data scientist most needs to know at that moment. `fluksio-worker` already defaulted `--python` to its own interpreter, so a GPU box works the same way — that was only ever undocumented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
This commit is contained in:
@@ -4,7 +4,11 @@ Point it at the engine on the box with the GPU and it dials in::
|
||||
|
||||
pip install fluksio-worker
|
||||
fluksio-worker --url wss://api.example.com/api/v1/workers/attach \\
|
||||
--token "$FLUKSIO_WORKER_TOKEN" --labels gpu --python /opt/venv/bin/python
|
||||
--token "$FLUKSIO_WORKER_TOKEN" --labels gpu
|
||||
|
||||
Install it into the environment the training code already runs in and node
|
||||
code runs on that: ``--python`` defaults to the interpreter this was started
|
||||
with. Point it elsewhere only when the two are meant to differ.
|
||||
|
||||
It connects *out*, so the engine needs no route back and nothing has to expose
|
||||
Redis. What it then does is what the engine's own worker pool does: hold a few
|
||||
@@ -274,7 +278,10 @@ def main(argv: list[str] | None = None) -> int:
|
||||
parser.add_argument(
|
||||
"--python",
|
||||
default=sys.executable,
|
||||
help="the interpreter node code runs on; point it at the venv with torch",
|
||||
help=(
|
||||
"the interpreter node code runs on (default: the one running this, "
|
||||
"so installing into the venv with torch in it is enough)"
|
||||
),
|
||||
)
|
||||
parser.add_argument("--parallel", type=int, default=1)
|
||||
parser.add_argument("--artifact-url", default="")
|
||||
|
||||
Reference in New Issue
Block a user