# Runs: pipelines that finish A **live flow** reacts to events. A **batch flow** starts with parameters, runs its graph once and returns a result. Each execution is a **run**, with its own state, parameters, node records, metrics and artifacts. Use runs for experiments, reports and other jobs with a finish line. For a complete first example, start with the [data science guide](../getting-started/data-science.md). ## A batch flow In Python, `Flow(...)` defaults to `mode="batch"`. Declare `inputs` for run parameters and `outputs` for the final result. On the canvas, choose batch mode and define the same inputs and outputs. A batch flow does not need activation and does not open subscriptions, schedules or webhooks. A run executes each node once after its dependencies finish. Missing parameters use their declared initial values. Non-streaming ports cannot use `interval` in a batch flow: there is no timer to deliver a delayed value. Streaming ports can use it to reduce display updates while keeping every metric point. ## Submitting In **Runs → New run**, choose a flow, enter its inputs and press **Run**. **Use draft** runs the working copy without publishing it. From a terminal: ```sh fluksio run train --lr 0.1 --epochs 20 --wait ``` Or submit through the API: ```sh curl -X POST "$FLUKSIO/runs/flows/train" \ -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \ -d '{"params": {"lr": 0.1, "epochs": 20}, "seed": 7}' ``` Here `$FLUKSIO` is the API base URL ending in `/api/v1`, and `$TOKEN` is an access token. The path takes the flow's id, here a flow whose id is its key, as for one made before ids existed. See [The HTTP API](../code/api.md) for authentication. Submission returns immediately with status `queued`. Read `GET /api/v1/runs/{id}` for status and results. Unknown parameters or wrong types return 422 before execution. The API body also accepts `"draft": true`, `"no_cache": true` and an `idempotency_key`. Reusing an idempotency key returns the original submission instead of creating another run. Drafts are read when execution starts, so edits made while a run is queued can affect it. A seed is recorded on the run and fills a declared input named `seed`. Your node code must use that input to seed its random number generator. ### Sweeps A sweep submits every combination of input values as one group: ```sh fluksio sweep train --param lr=0.01,0.1 --param epochs=10,50 --wait ``` These four runs share a `group_id` and execute concurrently within the engine's limits. Each has its own state, so one run cannot overwrite another's messages. Vary seeds with `--param seed=1,2,3` if the flow declares that input. The API equivalent is `POST /api/v1/runs/flows/{flow id}/sweep` with a `runs` list of parameter dictionaries and optional seeds. Filter history with `GET /api/v1/runs?group=`. ## Producing values before you are finished Training curves and progress values leave through output ports. A Python generator publishes them with `yield`; callbacks can use `fluksio.emit`. Declare the port with `stream: true`. See [Writing node code](../code/nodes.md#producing-values-over-time) for examples. Every numeric emission is stored under its flow-qualified message name, such as `train.loss`, with a step counter and timestamp. Retrieve it with: ```text GET /api/v1/runs/{id}/metrics?name=train.loss GET /api/v1/runs/series/compare?ids=,&metric=train.loss ``` A streaming port's `interval` reduces updates sent to the canvas, while the run history keeps every point. In a run, intermediate emissions do not start downstream nodes; those wait for the node's final result. A node's `timeout` measures silence. Each yield or emit resets it, so a node can run for hours while reporting progress. No timeout is imposed by default. ## Artifacts Use `fluksio.save_artifact` for checkpoints, datasets, plots and other bytes. Return its reference through an `artifact` port. Another node calls `fluksio.load_artifact` to obtain a local path. Identical bytes share one stored copy. Returned artifacts are listed on the run and can be downloaded with: ```sh fluksio artifacts fluksio artifacts train.weights -o weights.json ``` Use the exact message name shown by the list command. Emitted media frames are transient; return a frame you need to keep as a run artifact. ## Stage caching Before executing a Python node, the engine checks for a successful previous execution with matching source, settings, port declarations and input values. On a hit, it restores the outputs and artifacts and records the node as `cached`. For code-defined flows, sync also records digests of the project modules reachable from each node. Editing those modules invalidates the corresponding stage. Static import tracking does not follow computed import names or track installed dependency versions; use `--no-cache` after changes it cannot see. Artifact inputs are compared by content digest. Secret settings contribute their references rather than secret values. External state is not part of a cache key, so disable caching on a node that reads changing data or performs side effects: ```python @node(provides=Port("reading", "float"), cache=False) def read_sensor(): return get_current_reading() ``` Use `fluksio run ... --no-cache`, `fluksio sweep ... --no-cache`, or `"no_cache": true` in an API submission to execute every node. Only Python nodes are cached. Built-in and connector nodes are not. The cache uses run history; outputs too large to store there are not cacheable, and missing artifact bytes make an entry a miss. An entry written before the cache key last changed shape can never match again, so the engine drops it from the cache and keeps the node's recorded outputs. A cache hit does not replay emissions. Its metrics are read from the original run, so comparisons still show the recorded curve. Deleting that original run leaves cached results available but removes their curve. The run's page then names the deleted run each lost curve went with. ## Objects that cannot be serialized Live models, data loaders and compiled functions cannot cross a JSON message boundary. Keep stages that share live objects in one node, or save a checkpoint artifact and rebuild the object in the next node. ## Running a node somewhere else Attach a [remote worker](../code/workers.md) to the engine and declare the worker's label as the node's `worker`. `worker_policy="require"` waits for that label; `"prefer"` uses the local engine when none is attached. The selected machine needs your imported packages. Code-defined flows also need your project installed or available there: generated import shims travel with the call, but your package does not. Workers connect outward to the engine; see the worker guide for tokens and resource requests. ## Durability Queued runs are stored in the database and picked up when the engine restarts. Running jobs refresh a lease. If the engine dies and the lease expires, those runs become `abandoned` rather than being executed again automatically. Retry a finished run or the unfinished results of a sweep: ```sh fluksio retry fluksio retry --group ``` A retry creates a new run with the previous inputs, seed and group, linked to its parent. It uses the flow's current definition, and can reuse completed cached stages. Cancel active runs before retrying them. ## Looking at what ran The **Runs** screen filters by flow, status or sweep. Each run shows its parameters, per-node timings and logs, traceback, metrics, artifacts and result. Its inputs are listed as the flow declared them when the run was submitted, and the ones it took from the flow are marked as defaults. Select two or more runs to compare curves and the parameters that differ. Plot against step, elapsed seconds or another metric from the same runs. Comparisons have shareable URLs. Deleting a flow keeps its run history, and the artifacts those runs recorded stay on disk. You can still inspect those runs, but cannot retry them until the flow exists again. Tick **Also delete its runs** in the delete dialog to clear them with the flow, or delete history separately after active runs finish or are cancelled. ### Taking it into a dataframe Export metrics as a long table and results as a row per run: ```sh fluksio export metrics --flow train -o curves.csv fluksio export runs --flow train --status ok -o results.csv ``` Both include run IDs. Result rows also include input values, final numeric outputs, status, duration and code identifiers. Nested values use dotted paths, such as `metric.final_metrics.train_loss`. A row names the flow by its id in `flow` and by its key in `flow_name`. History stays with the id, so a flow keeps its runs whatever its key is later, and a new flow that takes the key of a deleted one starts without them. Metric names keep the key, as in `train.loss`. Python returns the same rows directly. `flow` takes the flow's id or key, as everywhere in the client: ```python from fluksio.sdk.client import Client client = Client() curves = client.export_metrics(flow="train") results = client.export_runs(flow="train", status="ok") ``` Pass either list to `pandas.DataFrame`. The [CLI reference](../code/cli.md#fluksio-export) covers selection filters, JSON Lines and Parquet. ### Bringing in runs made elsewhere Import historical runs without executing them: ```python from fluksio.sdk.client import Client Client().import_runs([{ "flow": "train", "external_id": "previous-study:run-42", "params": {"lr": 0.05, "epochs": 10}, "status": "ok", "created_at": "2026-03-01T10:00:00Z", "result": {"score": 0.94}, }]) ``` The flow must exist, and parameters are checked against its inputs. The Python client takes the flow's id or key for `flow`. Over the HTTP API an entry's `flow` is the flow's id. Repeated imports of the same `external_id` reuse the existing record. Imported runs have no code commit or digest because the engine did not execute them. Entries may also carry metric points and base64-encoded artifacts; see the API schema for their fields. ### A dashboard, read against runs Choose **Open in dashboard** from a run or comparison to view those results with a dashboard's widgets. Charts draw one line per run, scalar widgets use the first run, and controls are disabled. If the flow has no dashboard, the same menu can create one from its streaming ports and outputs. The API equivalent is `POST /api/v1/dashboards/from-flow/{flow id}`. The generated dashboard is published and can be edited like any other dashboard. To keep a chart showing recent experiments, set its source to **Runs** and select the latest runs of a flow, a sweep or specific run IDs. ## See also - [Getting started: data science](../getting-started/data-science.md) - [Writing node code](../code/nodes.md) - [Remote workers](../code/workers.md) - [The HTTP API](../code/api.md)