Artifacts: bytes a node produced, addressed by their content

A checkpoint is not a message. DType.ARTIFACT carries a reference — digest,
size, media type, name — so everything on the wire stays JSON and thirty
megabytes never sit in Redis, which answers the vision's open binary-payload
question by narrowing it: inline codecs would only serve payloads too small to
be worth a round trip, and nothing asks for that.

The store is content-addressed rather than per-run, for three reasons that all
pay later: a sweep whose fifty configs share one preprocessed input stores it
once, a reference stays valid however it is passed around because it names
content instead of a location, and the digest is what a stage cache will
compare — so building it in now is what keeps that from being a change to the
message contract.

Node code calls fluksio.save_artifact/load_artifact and cannot tell whether it
is writing the engine's own directory or putting bytes over HTTP, which is
what will let the same flow run on a remote worker unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AD8SfVhzXBG2nAfFcVh3iD
This commit is contained in:
2026-08-18 17:09:27 +02:00
co-authored by Claude Fable 5
parent 3f5bb12ed0
commit b1cb8b41bc
14 changed files with 1481 additions and 93 deletions
+2
View File
@@ -2,6 +2,7 @@ from fastapi import APIRouter
from app.api.routes import (
alerts,
artifacts,
dashboards,
flows,
login,
@@ -29,6 +30,7 @@ api_router.include_router(messages.router)
api_router.include_router(modules.router)
api_router.include_router(observability.router)
api_router.include_router(runs.router)
api_router.include_router(artifacts.router)
# Always mounted so the generated SDK stays the same shape; the endpoints
# themselves refuse to work unless MCP is switched on.
api_router.include_router(oauth.router)
+59
View File
@@ -0,0 +1,59 @@
"""Artifacts over HTTP: the one way bytes get in and out of the store.
A node on this host could reach the directory itself, but a node on a remote
worker cannot — and having one path rather than two is what keeps a flow's
code the same wherever it runs.
"""
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
from app.api.deps import get_current_user
from app.flow.artifacts import ArtifactStore
router = APIRouter(
prefix="/artifacts", tags=["artifacts"], dependencies=[Depends(get_current_user)]
)
class ArtifactRef(BaseModel):
digest: str
size: int
media_type: str
name: str = ""
def _store(request: Request) -> ArtifactStore:
store: ArtifactStore | None = getattr(request.app.state, "artifact_store", None)
if store is None:
raise HTTPException(status_code=503, detail="The artifact store is not ready")
return store
@router.put("", response_model=ArtifactRef)
async def put_artifact(
request: Request,
name: str = Query(default=""),
media_type: str = Query(default=""),
) -> Any:
"""Store the request body and answer with the reference to it."""
store = _store(request)
body = await request.body()
return store.put([body], name=name, media_type=media_type)
@router.get("/{digest}")
def get_artifact(digest: str, request: Request) -> Any:
"""Stream one artifact back."""
store = _store(request)
path = store.path(digest)
if path is None:
raise HTTPException(status_code=404, detail="No such artifact")
return StreamingResponse(
store.read(digest),
media_type="application/octet-stream",
headers={"Content-Length": str(path.stat().st_size)},
)