stroblmeandClaude Fable 5
fea57064f9
The workflow this serves: make a venv, install what you work with, then `pip install fluksio` into the same one. Building a second environment beside it was exactly wrong — the packages the nodes need are already here, and the Modules screen was asking for them a second time. `NODE_VENV=auto` (the default) adopts that venv. It declines in the three cases where adopting would be wrong: `managed` says otherwise, a managed venv already exists and may hold packages somebody installed on purpose, or the engine is not running from a venv at all. The images set `managed`, since the venv in them holds the app and nothing of anybody else's. An adopted venv is never written to. `uv pip sync` makes a venv hold exactly the manifest, so pointed at somebody's own environment it uninstalls their work and the engine with it — `sync()` refuses outright and `reconcile()` returns before it can be called at startup, which is where that would have happened first. The Modules screen lists what is installed and drops its editor; `pip` is how that environment changes. `fluksio serve` now names the interpreter node code runs on, which is the thing a data scientist most needs to know at that moment. `fluksio-worker` already defaulted `--python` to its own interpreter, so a GPU box works the same way — that was only ever undocumented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU
69 lines
2.6 KiB
Docker
69 lines
2.6 KiB
Docker
FROM python:3.10
|
|
|
|
ENV PYTHONUNBUFFERED=1
|
|
|
|
# Install uv
|
|
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#installing-uv
|
|
COPY --from=ghcr.io/astral-sh/uv:0.9.26 /uv /uvx /bin/
|
|
|
|
# Compile bytecode
|
|
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#compiling-bytecode
|
|
ENV UV_COMPILE_BYTECODE=1
|
|
|
|
# uv Cache
|
|
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#caching
|
|
ENV UV_LINK_MODE=copy
|
|
|
|
WORKDIR /app/
|
|
|
|
# Place executables in the environment at the front of the path
|
|
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#using-the-environment
|
|
ENV PATH="/app/.venv/bin:$PATH"
|
|
|
|
# /app/.venv holds the app and nothing of anybody else's, so there is nothing
|
|
# here to adopt: node code gets a venv of its own on the data volume, which is
|
|
# what the Modules screen installs into. A `pip install fluksio` into an
|
|
# environment somebody already works in is the case that adopts instead.
|
|
ENV NODE_VENV=managed
|
|
|
|
# Install dependencies
|
|
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#intermediate-layers
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
--mount=type=bind,source=uv.lock,target=uv.lock \
|
|
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
|
|
uv sync --frozen --no-install-workspace --package fluksio
|
|
|
|
COPY ./backend/scripts /app/backend/scripts
|
|
|
|
COPY ./backend/pyproject.toml ./backend/alembic.ini /app/backend/
|
|
|
|
COPY ./backend/fluksio /app/backend/fluksio
|
|
|
|
# The worker is a workspace member of its own, so the engine's sync needs it
|
|
# present to resolve the dependency on it. It is also what the engine runs
|
|
# node code with.
|
|
COPY ./worker /app/worker
|
|
|
|
# Sync the project
|
|
# Ref: https://docs.astral.sh/uv/guides/integration/docker/#intermediate-layers
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
--mount=type=bind,source=uv.lock,target=uv.lock \
|
|
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
|
|
uv sync --frozen --package fluksio
|
|
|
|
# Connectors are ordinary installed packages found through the
|
|
# `fluksio.node_types` entry point. `make connectors` builds them into here;
|
|
# an image with none is the normal case.
|
|
COPY ./backend/connector-wheels /tmp/connector-wheels
|
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
|
sh -c 'ls /tmp/connector-wheels/*.whl >/dev/null 2>&1 \
|
|
&& uv pip install --python /app/.venv/bin/python /tmp/connector-wheels/*.whl \
|
|
|| echo "no connectors bundled"'
|
|
|
|
WORKDIR /app/backend/
|
|
|
|
# Single worker on purpose: the process hosts the flow engine, and a second
|
|
# worker would be a second engine — duplicated subscriptions, cron ticks and
|
|
# webhooks. Scaling out is the M5 worker split, not more uvicorn processes.
|
|
CMD ["fastapi", "run", "fluksio/main.py"]
|