7efa75e2421c8397c6c79c07c3e11a5f235d2923
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  stroblmeandClaude Opus 5
|
9387755e59 |
Find a study wherever it is, and start on a port that is free
Docs / docs (push) Successful in 33s
Playwright Tests / test-playwright (1, 2) (push) Successful in 2m10s
Playwright Tests / test-playwright (2, 2) (push) Successful in 1m49s
pre-commit / pre-commit (push) Failing after 2m3s
Test Backend / test-backend (push) Successful in 2m32s
Compose Smoke Test / test-compose (push) Successful in 31s
Playwright Tests / merge-reports (push) Successful in 1m9s
Three things the one-folder-per-study layout ran into. **Discovery walks down.** A plain directory is now walked all the way, so `fluksio sync dev` finds `dev/s1_baseline/study.py` and naming each study is no longer the price of the layout. Hidden directories, `__pycache__`, `node_modules` and virtualenvs are left alone, and a package is taken whole. Two files that would import under one module name are refused, naming both: Python keeps one module per name, so the second would silently *be* the first — and a node's generated body imports by that name, so a worker would run the wrong study's code. The message says the fix, which is an `__init__.py` per study directory. A module that raises while importing is now a sentence naming the file rather than an importlib traceback. **`run` and `sweep` sync downwards too**, so the flow is found from the repository root without the sync-then-`--no-sync` two-step. A study that will not import is a warning rather than a stopped run, since a walk meets every study and a half-finished one two directories away is not this run's problem. The upload was already a no-op for a flow nothing changed in, so what the walk costs is import time — `--sync PATH` narrows it, and skipping unchanged subtrees would need a cache keyed on file state that is deliberately not here. **`serve` moves off a busy default port** — 8001, 8002, up to twenty — says which it took, and writes that one into `client.json`. A port given with `--port` still fails when it is taken, because naming one is asking for it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa |
||
|
stroblmeandClaude Opus 5
|
4479eeb726 |
Follow a record into its fields, name the metrics, name the version
Docs / docs (push) Successful in 38s
Playwright Tests / test-playwright (1, 2) (push) Successful in 2m56s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m7s
pre-commit / pre-commit (push) Failing after 2m17s
Test Backend / test-backend (push) Successful in 2m54s
Compose Smoke Test / test-compose (push) Successful in 44s
Playwright Tests / merge-reports (push) Successful in 1m17s
Three things the first export pass got wrong for a real study. **Dotted paths.** A node returns a record, not a scalar — the numbers arrive inside `final_metrics` — so `--metrics final_metrics.train_loss` yielded an empty column and `--metrics final_metrics` yielded the whole record in one cell. Both sides of the wide table now take dotted paths, and the defaults reach the same depth: every number a result carries is a column named by its path, and inputs are compared leaf by leaf, so two configurations differing in one field give that field as the axis rather than two blobs that are merely not equal. Lists stay whole — a curve belongs in the long table. **`--list`.** Metric names are flow-qualified, so `--name train_loss` matched nothing and said only that. `fluksio export metrics --list` prints the names the selection carries, and an empty export made with `--name` points at it. **A version to compare.** The CLI ships ahead of the engine and a stale one answered a flat 404 with nothing anywhere in the API to tell how old it was. The engine reports `version` on `/observability/summary`, `fluksio status` prints it, and a 404 from export now names both versions — or says "older" when the field itself predates the engine. Bumped to 0.1.5, which is what makes the number worth reading. Also formats `flow/metrics.py`, which had been committed unformatted and was the last `ruff format --check` failure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa |
||
|
stroblmeandClaude Opus 5
|
51464941ac |
Export runs and their curves as tables an analysis reads
Docs / docs (push) Successful in 35s
Playwright Tests / test-playwright (1, 2) (push) Successful in 3m11s
Playwright Tests / test-playwright (2, 2) (push) Successful in 2m17s
pre-commit / pre-commit (push) Failing after 2m44s
Test Backend / test-backend (push) Successful in 3m0s
Compose Smoke Test / test-compose (push) Successful in 41s
Playwright Tests / merge-reports (push) Successful in 8m14s
`fluksio export metrics` is the long table — a row per run, metric and step —
and `fluksio export runs` the wide one, a row per run with the inputs that
*vary* across the selection as columns beside its final numbers, status,
duration and the commit and digest of the code it ran. Both carry the run id
on every row, which is the join back to the run page and what makes an
exported file auditable. `Client.export_metrics`/`export_runs` answer the same
rows to a notebook.
The engine streams csv or jsonl from two routes declared above `/{run_id}`;
parquet is a client-side conversion behind the new `fluksio[parquet]` extra,
so nobody pays for pyarrow who does not want dtypes kept. The long export
reads each run through `_series`, so a cached node's curve comes with it, and
`--stride` thins each series rather than the concatenation of all of them.
Two things they needed on the way: `GET /runs` takes `?since=` and `?before=`,
so a long history pages by the last row's own timestamp instead of an offset
that shifts under it; and a read that reaches no engine now says so in half a
second rather than seven, because `runs`, `flavors`, `export` and an unwatched
`status` pass `retries=0`. Everything that submits keeps them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9Hdrmf2cwNABCnE5x9UJa
|
||
|
stroblmeandClaude Opus 5
|
1148e54c9e |
A run says whether the dashboard, the CLI or a script asked for it
`POST /runs/flows/{name}` hardcoded `cause: "api"`, so every row in the
history claimed the same origin. The body now carries an optional `cause`,
closed to the values the column knows — the dashboard sends nothing and stays
"api", `fluksio run` says "cli", and the SDK client says "sdk".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Gf7WaExcJ9bs3kfJXB3nK
|
||
|
stroblmeandClaude Opus 5
|
4941c2787c |
Add fluksio status, and ask for a run's parameters at a terminal
Two halves of the same gap: the CLI could start work but not show you any. `fluksio status` draws the home screen's top half in a terminal — health and what is wrong with it, every flow with its state and node count, and the recent runs and failures under them. `--watch` keeps it there. Rich does the drawing; it was already installed under fastapi's own CLI, and is named now because a command depends on it. `fluksio run` with no parameters at a terminal asks for them, one line per declared input with its declared value in brackets — so Enter through the lot is what running the defaults looks like, and an artifact input takes the `@run:` spelling the engine now resolves. A scripted run is untouched: passing any parameter, or piping the command, skips the questions, as does --defaults. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019V5bsYGNxcgPs4xXmTPx69 |
||
|
stroblmeandClaude Opus 5
|
400d7d9c5c |
Stage caching for batch runs, and an engine that lives in the command
Docs / docs (push) Successful in 19s
Playwright Tests / test-playwright (1, 2) (push) Failing after 1m5s
Playwright Tests / test-playwright (2, 2) (push) Failing after 20s
pre-commit / pre-commit (push) Failing after 2m33s
Test Backend / test-backend (push) Successful in 2m7s
Compose Smoke Test / test-compose (push) Failing after 20s
Playwright Tests / merge-reports (push) Failing after 1m3s
Publish / publish (push) Failing after 12s
A code node in a batch run is now fingerprinted by its source, its raw settings and the values it reads — an artifact input counting as its digest, which is what the content addressing was always for. A run that finds the key restores what the earlier one returned and skips the node, recorded as `cached`. The run history is the cache: `run_node.outputs` beside the `cache_key` the schema already had, no second store. On for code nodes, never for the built-in and connector types that have side effects; off per node with `@node(cache=False)` and per run with `--no-cache`. Emissions are not replayed on a hit, so a cached training node returns its result without redrawing its curve. Recorded in NOTEPAD.md with the two other deliberate limits. `fluksio run --local` boots the real app in the command's own process and drives it through its ASGI interface behind the ordinary client, so a run no longer needs a `serve` terminal beside it — same data directory, same history, and the cache carries between the two. It always waits, because the engine it starts lives exactly as long as the command. Also: `fluksio sweep --param lr=0.1,0.01` for the product of the lists, `run --follow` for a run's numbers as they arrive, Ctrl-C cancelling a waited run rather than abandoning it, coloured statuses on a terminal, and `name` made optional on the metrics endpoint so a follower can ask for every series. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Fable 5
|
7e4f03369b |
Three fewer things to remember
**The portal link puts itself back up.** It already retried a connection that raised, but a session that ended *cleanly* — a portal restarting, a proxy closing an idle socket — returned normally and went straight back round the loop with no wait at all, so an engine could spin against a portal that was merely saying goodbye politely. Every ending now reconnects on a delay, and the delay turns on whether the attempt got as far as attaching: one that stood up and dropped is a network event and retries at once, one that never stood up waits longer each time. Jittered, so a portal coming back is not met by every installation it serves in the same instant. Ping timeouts are named rather than defaulted, since they are what bounds how long a suspended laptop's dead socket looks alive, and the keepalive task is awaited so the reason a link went reaches the log instead of the garbage collector. **`fluksio enroll <code>`** is the whole command now; hub.fluksio.com is the default and `--portal` names another. The one command run before anything works should not need two flags. **`fluksio run` syncs first.** The reason a run exists is usually the edit before it, so remembering to sync was remembering to do something the computer could do — including the worker refresh, which is what makes an edit to your own package take effect at all. `--no-sync` opts out for a tight loop. That last one needed discovery fixed: it only ever looked at top-level `*.py`, so a repository whose code is in a package — the ordinary shape — found nothing from its own root. It now descends into the packages it holds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Fable 5
|
99f6530698 |
One installation per project, and no login to reach it
Two things a local install should not have asked for. `fluksio serve` now signs you in. Logging in to your own machine was a formality — the password was printed by the same process that would have checked it, and the database it authenticates against sits in the directory the token goes into — so `serve` mints the token itself and says where it put it. `fluksio login` is left for an engine somewhere else. And an installation is `.fluksio` beside the code, found the way `.git` is, rather than one `~/.fluksio` for the machine. A repository with its own venv was already getting its own engine; it now gets its own flows, run history and token too, instead of three repositories sharing one database and fighting over one port. `--global` asks for the shared one, `--data-dir` still names any directory, and when both exist the banner says which you are looking at and how to reach the other. The directory ignores itself from within — a `.gitignore` of `*`, the way uv writes one into `.venv` — because it holds a credential and a database, and neither belongs in anybody's history. The token is written mode 600. A login an older version wrote to ~/.config/fluksio is still read, so nothing that worked stops working. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Fable 5
|
a38e2745eb |
Add a Python SDK: flows declared in your own repository
A data scientist keeps their code where it is and decorates it: `@node` declares a function's ports beside the function, `Flow(name, nodes=[...])` says which of them make a flow, and `use(fn, wire=..., **settings)` rebinds one for a single flow. `fluksio sync` uploads the document plus a generated import shim per node, so the store still holds a complete, runnable, git-versioned definition while the code it imports stays theirs. `fluksio login|run|runs` and `flow.submit().wait()` are the client half, over the run endpoints that already existed. Runs record the user repository's commit beside the store's, so "what code produced this number" is answerable on the side that now holds the code. - `fluksio/sdk/`: ports, decorators, the flow builder and its checks, the shim generator, an HTTP client and sync. Standard library only at import, so `from fluksio import node` in a training script pulls in no engine. - `FlowDef.origin` marks a flow code-defined; `Run.origin_commit` carries the repository's commit; `POST /modules/refresh` retires the workers without an install, which every sync calls — a worker holds the imported package in memory, so an edit to it is invisible until the process goes. - The canvas shows a generated body read-only and names the repository to edit instead; a body edited there stops the next sync rather than being discarded. - The worker's reporter carries inert `Port`, `node`, `use` and `Flow`, since the shim imports a module whose first line declares them. - `examples/myresearch` is the worked example, `make sync-example` uploads it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ue1tkFWB1bcGy3aWhCKpU |
||
|
stroblmeandClaude Opus 5
|
0ad8d576ad |
Start without git, and say what that costs
A `pip install` on a locked-down host — the case the CLI exists for — may have no git, and the store shelled out to it while building the flow repository, so the engine refused to start at all. The store is files; git is their history. Missing it is now one warning and no commits rather than a stack trace, which is the difference between a machine that runs your experiments and one that does not. Found by installing the wheels into a bare python:3.12-slim and pairing it with the portal: `fluksio enroll` took the code, `fluksio serve` dialled out, and the hub was proxying requests through the tunnel. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
stroblmeandClaude Opus 5
|
10b0ba9e49 |
Add the fluksio CLI: serve, enroll, worker
`pip install fluksio && fluksio serve` on a machine with no Docker, no database and no configuration — which is the case this is for: a node on a cluster where ports cannot be opened. It makes its data directory, its key and an admin account, prints the password once, and serves. Pairing is `fluksio enroll <code> --portal …`, doing what the Settings screen does through the same function, before the engine starts and without one running — a machine nobody can route to has no browser pointed at it either. The portal serves the dashboard, so nothing is served here. Two things had to give way. `fastapi[standard]` pulls a cloud CLI that wants sentry-sdk 2.x while we pinned below it — no pip resolution existed, so the pin is lifted, which the comment beside it had been waiting for and which also lets the Python cap go. And `uv` is now a dependency rather than something to find on PATH: the Modules screen is how a data scientist installs torch, and it was quietly falling back to the engine's own interpreter. The CLI imports nothing from the engine before it has set DATA_DIR — the settings are built on the first import of core.config, and reaching it early put the database in the working directory. There is a test for that now, because the failure is silent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |