Home: one time range across the health block, and failures that follow it

The health screen was fixed at 24 hours everywhere except its lists, which
were fixed at nothing: `failuresQueryOptions` read the newest 100 rows and
`HealthActivity` filtered them client-side, so on a busy engine the failures
list covered whatever few minutes 100 rows happened to span while the chart
beside it spanned a day — and pinning an older minute showed an empty card.

One `RangePicker` now sits on the Health heading and governs the whole block:
the tiles, the flow table, both charts and both lists. Presets are 1h / 6h /
24h / 7d — the collector prunes at `OBS_RETENTION_DAYS` (30), so a week is
behind the last one. The longer windows ask for coarser buckets, since a week
of minute rollups is ten thousand points nobody can see.

Failures get the escape hatch the runs already had: a pinned minute is asked
for with `since`/`until` rather than filtered out of what is held, and the
list itself is bound to the selected range. `RUN_DEPTH`/`EVENT_DEPTH` become
one `LIST_DEPTH`, which now buys coverage of the window on screen instead of
a fixed newest-N — narrowing the range is what makes the same rows reach the
whole of it. The "Failures (24h)" tile counts errors over the selected window
from the rollups the table is drawn from, so tile, column and chart agree.

The node-panel and edge trend curves get no picker. They are a Redis ring of
the last 120 values per message with no window to ask for, so hovering one
reveals what it actually shows — how many readings, and the span they cover.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XC2jX6Hdj7pxGGKzBTrbqB
This commit is contained in:
2026-08-17 11:51:42 +02:00
co-authored by Claude Opus 5
parent 3d73e42313
commit 683c25b26d
7 changed files with 245 additions and 62 deletions
+4 -3
View File
@@ -13,8 +13,6 @@ should reopen it.
### To be sorted
- BUG: Logs currently only have small time window compared to e.g. graphs (in the home view); can we fix this, especially for the purpose of tracing back events?
- FEAT: we should introduce a unified way to select time ranges in graph views with some convenience buttons like -1h -6h -24h . Make sure this also applies to the smaller "trend" charts like shown in the node panel or on edges; here we could show the time range selector upon hovering the graph (presets are fine)
- INFRA: ensure that all the packages/ dependencies needed to run fluksio are available on arm to make this software runnable on e.g. raspbian
- INFRA: merge the philosophy statement at the beginning of vision.md into the rest of the document. Dissolve the decision dates and fold the decisions into a clean structure
- BUG/UI mobile friendly support is degraded: 1) toolbar in the "Flows" viewport extend mobile viewport width 2) position of nodes should never be static; always adjust such that there are as few as possible overlaps (of nodes and edge labels) and direction is left to right (desktop) or top to bottom (mobile) with a minimal (but clean) overall edge length 3) Dashboard view is not mobile friendly at all; as dashboard design is infeasible on mobile, render all widgets in a vertically stacked order. This allows to inspect each widget and make changes. Layout changes are not a feature on mobile 4) the home view is not responsive; all items shown there should re-order on mobile such that no scrollbars appear. Make sure the mobile support is anchored in the design such that future work does not break it
@@ -63,7 +61,9 @@ is what M4 still waits on, together with porting the flows.
- CHORE/FLOW: `Pipeline.flush` releasing a held value runs its cascade without a run id, so those executions land in the minute rollups but in no run. Threading the scheduling run's id through the queue item would close it.
- CHORE/API: the metrics collector is a bus subscriber, so a storm that overflows the bus queue undercounts. The events dropped are the same ones the websocket drops; exact accounting would need the collector to be fed from the engine rather than the bus.
- CHORE/API: `/observability/summary` reports the work queue's `depth` as the Redis stream length, which is the journal size (capped at `STREAM_MAXLEN`) rather than a backlog. The health screen shows `pending` instead; the field name still invites the wrong reading.
- FEAT/UI: the health screen's window is fixed at 24 hours and the charts fold minute buckets in Python. A range picker (and `date_bin()` behind it) is the next step if anyone wants a week.
- PERF/API: the health block picks its window now, but `/observability/timeseries` and `/observability/flows` still read every `metric_minute` row in it and fold them in Python. `bucket_s` only coarsens what comes back, so the 7d preset pulls a week of rows on each 30 s poll. `date_bin()` is what makes the long windows cheap.
- CHORE/API: `/observability/summary` still returns `failures_24h`, which nothing reads any more — the Home tile counts errors over the selected window from the rollups instead. Drop the field, or let the summary take a window.
- CHORE/UI: the Home block's "Changes" list is the newest 15 audit rows whatever range is selected. Deliberate — an audit trail is worth reading past the window — but it sits under a control that governs everything else on the screen.
- CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but `forget_flow` could offer to clear it.
- CHORE/API: nothing can ask the collector to flush now, so anything needing the tables to be current has to wait out `FLUSH_INTERVAL_S` — which is what the soak harness does before clearing its own rows.
- CHORE/API: `MetricsCollector._start_run`'s `existing is not None` branch is unreachable: a redelivery only arrives after the record it would update has been dropped.
@@ -203,6 +203,7 @@ Open on purpose. Each names what should bring it back.
- CHORE/INFRA: `requires-python` is capped below 3.14 because the MCP SDK wants a newer starlette there than the pinned `sentry-sdk<2` allows. Lift the cap when sentry-sdk moves to 2.x.
- CHORE/INFRA: `bun run --filter frontend build` fails on this workspace with `crypto.hash is not a function` — Vite 7 wants Node 20.12+ and the host has 18. The Docker image builds fine, so it only bites local bundling; `bunx tsc` still type-checks.
- FEAT/UI: an endpoint's edge routes straight across the graph, so it can pass behind a node that sits between the lane and the node it wires to. Readable, but a routed edge would be tidier.
- FEAT/UI: the node-panel and edge trend curves take no range, unlike the health block. They are drawn from a Redis ring of the last 120 values per message, which has no window to ask for — a hover caption names what the curve covers instead of a picker promising a span nothing can serve. Reopen if per-message history ever gains a time window.
- FEAT/UI: an e-ink rendering profile for a dashboard — motion off, hover-only affordances resolved to something visible, high-contrast palette, thick strokes, and a repaint cadence low enough for a display that takes a second to settle. Reopen when a panel with such a display is actually hung.
## Blocked
@@ -0,0 +1,75 @@
import { cn } from "@/lib/utils"
/**
* A window of history, and everything a query needs to ask for it.
*
* `bucketS` keeps the number of points a chart draws roughly constant: a week
* of minute buckets is ten thousand readings nobody can see and a megabyte of
* JSON every refresh, so the longer windows are folded coarser server-side.
*/
export type Range = { label: string; hours: number; bucketS: number }
/**
* The windows on offer.
*
* Bounded by what the collector keeps: it prunes buckets, failures and runs at
* `OBS_RETENTION_DAYS` (30 by default), so a week is behind the last preset
* rather than an empty chart.
*/
export const RANGES: Range[] = [
{ label: "1h", hours: 1, bucketS: 60 },
{ label: "6h", hours: 6, bucketS: 60 },
{ label: "24h", hours: 24, bucketS: 60 },
{ label: "7d", hours: 168, bucketS: 900 },
]
/** A day: long enough to hold a night's worth of trouble, short enough to read. */
export const DEFAULT_RANGE = RANGES[2]
/**
* Where the window starts, as the stamp the history endpoints take.
*
* Read at fetch time rather than when the options are built, so the window
* slides with the clock instead of freezing where the screen opened.
*/
export const rangeStart = (range: Range) =>
new Date(Date.now() - range.hours * 3600_000).toISOString()
/**
* The window a screen is showing, as presets.
*
* The one segmented shape: a single border pill, transparent segments,
* bg-accent on the selected one (root DESIGN-GUIDELINES.md).
*/
export function RangePicker({
value,
onChange,
}: {
value: Range
onChange: (range: Range) => void
}) {
return (
<fieldset
data-testid="range-picker"
className="flex w-fit items-center gap-1 rounded-full border border-border p-1"
>
<legend className="sr-only">Time range</legend>
{RANGES.map((range) => (
<button
key={range.label}
type="button"
aria-pressed={range.hours === value.hours}
onClick={() => onChange(range)}
className={cn(
"rounded-full px-2.5 py-1 text-xs transition-colors",
range.hours === value.hours
? "bg-accent text-accent-foreground"
: "text-muted-foreground hover:bg-accent/50",
)}
>
{range.label}
</button>
))}
</fieldset>
)
}
@@ -3,6 +3,11 @@ import { useEffect, useState } from "react"
import type { HistoryPoint } from "@/client"
import { Sparkline } from "@/components/Common/Sparkline"
import {
Tooltip,
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip"
import { qualify } from "./deriveEdges"
import { useLiveValue } from "./liveStore"
import { messageHistoryQueryOptions } from "./queries"
@@ -16,6 +21,27 @@ function describe(value: unknown): string {
return JSON.stringify(value) ?? String(value)
}
/** A stretch of time in the coarsest unit that still says it. */
function span(seconds: number): string {
if (seconds < 90) return `${Math.round(seconds)}s`
if (seconds < 5400) return `${Math.round(seconds / 60)}m`
return `${Math.round(seconds / 3600)}h`
}
/**
* What the curve is actually showing.
*
* These are a ring of the last `WINDOW` values per message, not a window of
* time — a message that fires twice an hour and one that fires at 10 Hz draw
* the same width for wildly different spans. The readings themselves are what
* says which, so the caption reads it off them rather than claiming a range.
*/
function caption(points: HistoryPoint[]): string {
const readings = `The last ${points.length} reading${points.length === 1 ? "" : "s"}`
const covered = points[points.length - 1].ts - points[0].ts
return covered > 0 ? `${readings}, over ${span(covered)}.` : `${readings}.`
}
/**
* The name only settles once typing stops. Without this, every keystroke in the
* message field would ask the server for a history.
@@ -93,6 +119,17 @@ export function MessageSparkline({
)
}
// The value is live here, so the dot on the newest reading is earned.
return <Sparkline points={points} />
// The value is live here, so the dot on the newest reading is earned. The
// caption is on hover rather than beside the curve: these sit one per port
// in a dense panel, and a line of prose each would crowd it out.
return (
<Tooltip>
<TooltipTrigger asChild>
<div>
<Sparkline points={points} />
</div>
</TooltipTrigger>
<TooltipContent>{caption(points)}</TooltipContent>
</Tooltip>
)
}
@@ -3,6 +3,7 @@ import { ChevronDown, ChevronRight, X } from "lucide-react"
import { useEffect, useState } from "react"
import type { EventRow, HistoryPoint } from "@/client"
import type { Range } from "@/components/Common/RangePicker"
import { UplotChart } from "@/components/Common/UplotChart"
import { useEngineEvents } from "@/components/Flow/liveStore"
import { PANEL_SECTION } from "@/components/Flow/SidePanel"
@@ -16,6 +17,7 @@ import {
deadLetterQueryOptions,
failuresQueryOptions,
healthKeys,
minuteFailuresQueryOptions,
minuteOf,
minuteRunsQueryOptions,
runsQueryOptions,
@@ -175,22 +177,27 @@ function Failure({ event }: { event: EventRow }) {
}
/**
* What the engine has been doing, as charts over a day and the records behind
* them.
* What the engine has been doing over the selected window, as charts and the
* records behind them.
*
* The two charts each drive the list beside them: throughput picks the runs of
* a minute, failures and timing picks that minute's failures.
* a minute, failures and timing picks that minute's failures. The window comes
* from the health block's one range control, so the lists cover the same span
* the charts are drawn from rather than however far their newest rows reach.
*/
export function HealthActivity() {
export function HealthActivity({ range }: { range: Range }) {
const live = useEngineEvents()
const queryClient = useQueryClient()
const runsAt = useMoment()
const failuresAt = useMoment()
const { data: series } = useQuery(timeseriesQueryOptions())
const { data: runs } = useQuery(runsQueryOptions())
const { data: series } = useQuery(timeseriesQueryOptions(range))
const { data: runs } = useQuery(runsQueryOptions(range))
const { data: pinnedRuns } = useQuery(minuteRunsQueryOptions(runsAt.pinned))
const { data: failures } = useQuery(failuresQueryOptions())
const { data: failures } = useQuery(failuresQueryOptions(range))
const { data: pinnedFailures } = useQuery(
minuteFailuresQueryOptions(failuresAt.pinned),
)
const { data: audit } = useQuery(auditQueryOptions())
const { data: dead } = useQuery(deadLetterQueryOptions())
@@ -219,8 +226,8 @@ export function HealthActivity() {
// A pin is read back from the server, so it reaches a minute the recent list
// is nowhere near deep enough to hold. A hover stays the client-side preview
// it is: scrubbing a day's chart would otherwise be a request per minute the
// pointer rests on.
// it is: scrubbing a whole window's chart would otherwise be a request per
// minute the pointer rests on.
const shownRuns =
runsAt.pinned !== null
? (pinnedRuns ?? [])
@@ -228,9 +235,13 @@ export function HealthActivity() {
? (runs ?? []).slice(0, RUNS_SHOWN)
: (runs ?? []).filter((run) => minuteOf(run.started_at) === runsAt.at)
const shownFailures =
failuresAt.at === null
failuresAt.pinned !== null
? (pinnedFailures ?? [])
: failuresAt.at === null
? (failures ?? []).slice(0, FAILURES_SHOWN)
: (failures ?? []).filter((event) => minuteOf(event.ts) === failuresAt.at)
: (failures ?? []).filter(
(event) => minuteOf(event.ts) === failuresAt.at,
)
return (
<>
@@ -329,8 +340,10 @@ export function HealthActivity() {
) : (
<p className="text-sm text-muted-foreground">
{failuresAt.at === null
? "Nothing has failed in the last day."
: "Nothing failed in this minute."}
? `Nothing has failed in the last ${range.label}.`
: failuresAt.pinned !== null
? "Nothing failed in this minute."
: "No failure from this minute is in the recent list."}
</p>
)}
</div>
@@ -2,6 +2,7 @@ import { useQuery } from "@tanstack/react-query"
import { Link } from "@tanstack/react-router"
import type { FlowRollup, HistoryPoint } from "@/client"
import { type Range, RangePicker } from "@/components/Common/RangePicker"
import { Sparkline } from "@/components/Common/Sparkline"
import { PANEL_SECTION } from "@/components/Flow/SidePanel"
import { Badge } from "@/components/ui/badge"
@@ -37,9 +38,11 @@ function Tile({
/**
* A flow's execution trend, drawn from the 60 slices the rollup carries.
*
* The same curve the node panel and the edge popover draw, in the chart ramp
* this page's other graphs use. No live dot: the rollups are polled, so the
* right edge is the last completed slice rather than this instant.
* Sixty slices of whatever window is selected, so the curve stays the same
* width and only its resolution moves. The same curve the node panel and the
* edge popover draw, in the chart ramp this page's other graphs use. No live
* dot: the rollups are polled, so the right edge is the last completed slice
* rather than this instant.
*/
function Spark({ counts }: { counts: number[] }) {
const points: HistoryPoint[] = counts.map((value, index) => ({
@@ -61,28 +64,40 @@ function Spark({ counts }: { counts: number[] }) {
}
/**
* How the engine is doing, and how each flow has been doing for a day.
* How the engine is doing, and how each flow has been doing over the window.
*
* The tiles are the standing state; the table below is the same day the charts
* cover, one row per flow.
* The tiles are the standing state; the table below is the same window the
* charts cover, one row per flow. The range control sits on this heading
* because it governs the whole health block, the activity below included —
* one window, not one per card.
*/
export function HealthOverview() {
export function HealthOverview({
range,
onRangeChange,
}: {
range: Range
onRangeChange: (range: Range) => void
}) {
const { data: summary } = useQuery(summaryQueryOptions())
const { data: flows } = useQuery(flowRollupsQueryOptions())
const { data: flows } = useQuery(flowRollupsQueryOptions(range))
// Shares the list below's cache entry, for the one tile that dates them.
const { data: failures } = useQuery(failuresQueryOptions())
const { data: failures } = useQuery(failuresQueryOptions(range))
const queue = (summary?.queue ?? {}) as Record<string, number>
const degraded = summary?.status === "degraded"
const errors = (flows ?? []).reduce((total, row) => total + row.errors, 0)
return (
<>
<section className="grid gap-3">
<div className="flex items-center gap-3">
<div className="flex flex-wrap items-center gap-3">
<h2 className={PANEL_SECTION}>Health</h2>
<Badge variant={degraded ? "destructive" : "secondary"}>
{degraded ? "Degraded" : "Running normally"}
</Badge>
<div className="ml-auto">
<RangePicker value={range} onChange={onRangeChange} />
</div>
</div>
{summary?.problems.length ? (
<p className="text-sm text-muted-foreground">
@@ -112,8 +127,12 @@ export function HealthOverview() {
}
/>
<Tile
label="Failures (24h)"
value={si(summary?.failures_24h ?? 0)}
label={`Failures (${range.label})`}
// Summed from the rollups the table below is drawn from, so the
// tile, the Errors column and the chart's error line all count the
// same thing over the same window. Narrower than the list beside
// it, which also carries quarantines and crashed tasks.
value={si(errors)}
note={
failures?.length
? `latest ${ago(failures[0].ts)}`
@@ -134,7 +153,7 @@ export function HealthOverview() {
</section>
<section className="grid gap-3">
<h2 className={PANEL_SECTION}>Flow activity (24h)</h2>
<h2 className={PANEL_SECTION}>Flow activity ({range.label})</h2>
<div className={`${CARD} overflow-x-auto`}>
{/* The name anchors the left, the numbers read down their own
centre, and the trend closes the row on the right. */}
@@ -197,7 +216,7 @@ export function HealthOverview() {
colSpan={6}
className="py-6 text-center text-muted-foreground"
>
No flow has run in the last day.
No flow has run in the last {range.label}.
</td>
</tr>
) : null}
+60 -27
View File
@@ -1,7 +1,5 @@
import { ObservabilityService } from "@/client"
/** The live view; anything older is the collector's rollups. */
export const HOURS = 24
import { type Range, rangeStart } from "@/components/Common/RangePicker"
/** Another tab can change things, and the engine fails flows on its own. */
const REFRESH = 30_000
@@ -11,11 +9,14 @@ const REFRESH = 30_000
*
* Deeper than they are shown: hovering a minute on a chart previews these rows
* client-side, and a list holding only the newest handful would have nothing
* to show for any minute but the current one. Pinning asks the server for the
* minute instead, which is what reaches past this depth.
* to show for any minute but the current one.
*
* Both lists are bound to the selected range, so the depth buys coverage of
* whatever window is on screen — narrowing the range is what makes the same
* rows reach the whole of it. Pinning asks the server for the minute, which
* reaches past this depth at any range. 200 is the cap `/runs` enforces.
*/
const RUN_DEPTH = 200
const EVENT_DEPTH = 100
const LIST_DEPTH = 200
export const healthKeys = {
all: ["observability"] as const,
@@ -32,49 +33,81 @@ export const summaryQueryOptions = () => ({
refetchInterval: 10_000,
})
export const timeseriesQueryOptions = () => ({
queryKey: ["observability", "timeseries", HOURS] as const,
queryFn: () => ObservabilityService.readTimeseries({ hours: HOURS }),
export const timeseriesQueryOptions = (range: Range) => ({
queryKey: ["observability", "timeseries", range.hours] as const,
queryFn: () =>
ObservabilityService.readTimeseries({
hours: range.hours,
bucketS: range.bucketS,
}),
refetchInterval: REFRESH,
})
export const flowRollupsQueryOptions = () => ({
queryKey: ["observability", "flows", HOURS] as const,
queryFn: () => ObservabilityService.readFlowRollups({ hours: HOURS }),
export const flowRollupsQueryOptions = (range: Range) => ({
queryKey: ["observability", "flows", range.hours] as const,
queryFn: () => ObservabilityService.readFlowRollups({ hours: range.hours }),
refetchInterval: REFRESH,
})
export const runsQueryOptions = () => ({
queryKey: ["observability", "runs"] as const,
queryFn: () => ObservabilityService.readRuns({ limit: RUN_DEPTH }),
export const runsQueryOptions = (range: Range) => ({
queryKey: ["observability", "runs", "recent", range.hours] as const,
queryFn: () =>
ObservabilityService.readRuns({
since: rangeStart(range),
limit: LIST_DEPTH,
}),
refetchInterval: REFRESH,
})
export const failuresQueryOptions = (range: Range) => ({
queryKey: [...healthKeys.events, "failure", "recent", range.hours] as const,
queryFn: () =>
ObservabilityService.readEvents({
kind: "failure",
since: rangeStart(range),
limit: LIST_DEPTH,
}),
refetchInterval: REFRESH,
})
/** One minute, as the bounds both history endpoints take. */
const minuteWindow = (at: number | null) => ({
since: new Date((at ?? 0) * 1000).toISOString(),
until: new Date(((at ?? 0) + 60) * 1000).toISOString(),
})
/**
* The runs of one minute, wherever it sits in the day.
* The runs of one minute, wherever it sits in the range.
*
* A busy engine writes more runs per minute than the recent list is deep, so a
* pinned moment is asked for rather than filtered out of what is already held.
* `at` is a minute start, and the window is that minute.
*/
export const minuteRunsQueryOptions = (at: number | null) => ({
queryKey: ["observability", "runs", at] as const,
queryKey: ["observability", "runs", "minute", at] as const,
queryFn: () =>
ObservabilityService.readRuns({
since: new Date((at ?? 0) * 1000).toISOString(),
until: new Date(((at ?? 0) + 60) * 1000).toISOString(),
limit: RUN_DEPTH,
}),
ObservabilityService.readRuns({ ...minuteWindow(at), limit: LIST_DEPTH }),
// A minute that has passed does not change, and the current one is refreshed
// by the unpinned list anyway.
enabled: at !== null,
})
export const failuresQueryOptions = () => ({
queryKey: [...healthKeys.events, "failure"] as const,
/**
* The failures of one minute — the same escape hatch the runs have.
*
* Without it the list could only ever answer for the minutes its newest rows
* happen to span, which on a failing engine is a fraction of the range the
* chart beside it draws.
*/
export const minuteFailuresQueryOptions = (at: number | null) => ({
queryKey: [...healthKeys.events, "failure", "minute", at] as const,
queryFn: () =>
ObservabilityService.readEvents({ kind: "failure", limit: EVENT_DEPTH }),
refetchInterval: REFRESH,
ObservabilityService.readEvents({
kind: "failure",
...minuteWindow(at),
limit: LIST_DEPTH,
}),
enabled: at !== null,
})
export const auditQueryOptions = () => ({
+7 -2
View File
@@ -1,8 +1,10 @@
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"
import { createFileRoute, Link } from "@tanstack/react-router"
import { AlertCircle, Workflow } from "lucide-react"
import { useState } from "react"
import { type FlowSummary, FlowsService } from "@/client"
import { DEFAULT_RANGE } from "@/components/Common/RangePicker"
import { BrainView } from "@/components/Flow/BrainView"
import { flowKeys, flowsQueryOptions } from "@/components/Flow/queries"
import { HealthActivity } from "@/components/Health/HealthActivity"
@@ -100,6 +102,9 @@ function FlowRow({ flow }: { flow: FlowSummary }) {
*/
function Dashboard() {
const { user: currentUser } = useAuth()
// The health block's window: one choice, read by the tiles, the flow table,
// the charts and the lists under them.
const [range, setRange] = useState(DEFAULT_RANGE)
const { data, isPending } = useQuery({
...flowsQueryOptions(),
refetchInterval: REFRESH_INTERVAL,
@@ -146,8 +151,8 @@ function Dashboard() {
)}
</Card>
<HealthOverview />
<HealthActivity />
<HealthOverview range={range} onRangeChange={setRange} />
<HealthActivity range={range} />
</div>
)
}