diff --git a/NOTEPAD.md b/NOTEPAD.md index f1a3aca..157b3aa 100644 --- a/NOTEPAD.md +++ b/NOTEPAD.md @@ -15,7 +15,6 @@ should reopen it. - INFRA: merge the philosophy statement at the beginning of vision.md into the rest of the document. Dissolve the decision dates and fold the decisions into a clean structure - BUG/UI mobile friendly support is degraded: 1) toolbar in the "Flows" viewport extend mobile viewport width 2) position of nodes should never be static; always adjust such that there are as few as possible overlaps (of nodes and edge labels) and direction is left to right (desktop) or top to bottom (mobile) with a minimal (but clean) overall edge length 3) Dashboard view is not mobile friendly at all; as dashboard design is infeasible on mobile, render all widgets in a vertically stacked order. This allows to inspect each widget and make changes. Layout changes are not a feature on mobile 4) the home view is not responsive; all items shown there should re-order on mobile such that no scrollbars appear -- BUG/UI remove the up/down counters from integer fields (global) ### Connector write paths @@ -52,6 +51,7 @@ is what M4 still waits on, together with porting the flows. - CHORE/API: the metrics collector is a bus subscriber, so a storm that overflows the bus queue undercounts. The events dropped are the same ones the websocket drops; exact accounting would need the collector to be fed from the engine rather than the bus. - CHORE/API: `/observability/summary` reports the work queue's `depth` as the Redis stream length, which is the journal size (capped at `STREAM_MAXLEN`) rather than a backlog. The health screen shows `pending` instead; the field name still invites the wrong reading. - FEAT/UI: the health screen's window is fixed at 24 hours and the charts fold minute buckets in Python. A range picker (and `date_bin()` behind it) is the next step if anyone wants a week. +- FEAT/API: `/observability/runs` takes no time range and caps at 200 rows, so picking a minute on the throughput chart can only filter the runs the list happens to hold — on a busy engine that is its last minute or so, and any earlier minute reads as empty. A `since`/`until` parameter would let a moment on a chart reach the whole day. Failures are sparse enough that the same pairing works there. - CHORE/FLOW: run records for a deleted flow stay until the retention window passes, so a flow that no longer exists keeps appearing in the history. Deliberate — it is a record of what ran — but `forget_flow` could offer to clear it. - CHORE/API: nothing can ask the collector to flush now, so anything needing the tables to be current has to wait out `FLUSH_INTERVAL_S` — which is what the soak harness does before clearing its own rows. - BUG/UI: `MemoryWorkQueue.stats()` hard-codes `pending: 0`, so the health tile always reads zero on a stack without Redis. @@ -85,6 +85,7 @@ is what M4 still waits on, together with porting the flows. - CHORE/UX: the derived-cron chip also appears on the delay node, where `interval` is a rate limit rather than a schedule. May want it inject-only. - CHORE/UX: free-form params (python nodes) get no suggestions, since there is no schema to key them off. - PERF/UI: `BrainView` runs 300 force-layout ticks synchronously inside a `useMemo`, so the graph is laid out on the render thread. +- FEAT/UI: the brain is a band on a scrolling page now, so it neither pans nor zooms — the fit keeps the whole graph in view instead. An installation with enough flows to make the labels unreadable at that fit needs a way to open the graph larger. - FEAT/UI: the brain graph draws connections as plain lines, so which way a value travels is not shown. An arrowhead needs the path shortened to the target circle's rim, which needs the node radius on the edge. - CHORE/UI: fixing the yellow message pulse gave `edge-pulse` an explicit `to`, so a *selected* edge now decays to `--muted-foreground` and snaps back to its blue at the end instead of decaying into it. Barely visible, and only while an edge is selected. diff --git a/ROADMAP.md b/ROADMAP.md index f0ff954..968dded 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -175,16 +175,20 @@ React + Vite, primarily desktop but usable on mobile. See `docs/architecture/str channels/rules each get a sidebar page, and the OAuth clients an agent registers are listed and revocable under Admin — which needed its management endpoints written first -- [x] Health screen: how the engine is doing now (nodes, flows, queue, loop lag) +- [x] Health sections: how the engine is doing now (nodes, flows, queue, loop lag) over what it has been doing all day — throughput and failure charts, a per-flow table, the recent cascades, failures that expand to their - traceback, dead-lettered work and the audit trail -- [x] Brain graph: every published flow on one canvas, with nodes that talk to the + traceback, dead-lettered work and the audit trail. Hovering a chart + filters the list beside it to that minute and a click pins it +- [x] Brain graph: every published flow at once, with nodes that talk to the same outside thing — a broker topic, a URL, a bucket — drawn as a single neuron, so the wiring that runs between flows through a broker is visible at all. Laid out by a force simulation settled once and then frozen, lit by the same socket the editor listens to, and read-only: a neuron leads back to the flow it came from +- [x] Both of the above sit on Home rather than at routes of their own: the brain + flat across the top, the health sections under the flow switches. One + overview instead of three - [x] Mobile-friendly canvas: touch connect, full-screen node panel - [ ] Installable as a PWA (`vite-plugin-pwa`) diff --git a/frontend/src/components/Common/UplotChart.tsx b/frontend/src/components/Common/UplotChart.tsx index 96564d0..e50fba9 100644 --- a/frontend/src/components/Common/UplotChart.tsx +++ b/frontend/src/components/Common/UplotChart.tsx @@ -4,6 +4,7 @@ import "uplot/dist/uPlot.min.css" import type { HistoryPoint } from "@/client" import { useTheme } from "@/components/theme-provider" +import { compact } from "@/lib/utils" /** * How many lines one chart carries. @@ -16,11 +17,8 @@ export const MAX_SERIES = 5 /** Room for the axis ticks; uPlot measures the rest of the box itself. */ const PADDING: uPlot.Padding = [10, 12, 0, 0] -/** The legend sits under the canvas, so the canvas has to leave it room. */ -const LEGEND_HEIGHT = 26 - const canvasHeight = (element: HTMLElement) => - Math.max(60, (element.clientHeight || 180) - LEGEND_HEIGHT) + Math.max(60, element.clientHeight || 180) /** A token, resolved for the canvas — which cannot read CSS variables. */ function token(name: string): string { @@ -31,6 +29,17 @@ function token(name: string): string { const seriesColor = (index: number) => token(`--chart-${(index % 5) + 1}`) +/** + * An axis tick, kept short. + * + * The gutter the ticks are drawn in has a fixed width, so a grouped "15,000" + * is clipped to something that reads as a different number entirely. + */ +const tick = (value: number) => + Math.abs(value) >= 1000 + ? `${+(value / 1000).toPrecision(3)}k` + : compact(value) + /** The series joined onto one x axis, which is what uPlot draws. */ function table(plots: HistoryPoint[][]): uPlot.AlignedData { return uPlot.join( @@ -52,22 +61,39 @@ function table(plots: HistoryPoint[][]): uPlot.AlignedData { * that arrives. Its own legend doubles as the hover readout, so the cursor * tells you what each line was worth at that moment — and with more than one * line a legend is required anyway. + * + * `onCursor` / `onSelect` report where the pointer is in the data, so a page + * can tie a list to the chart without reaching into the uPlot instance. */ export function UplotChart({ labels, plots, empty = "Nothing has come through yet.", + onCursor, + onSelect, }: { /** One label per series; the set of them is the chart's identity. */ labels: string[] /** The points of each series, in the same order as `labels`. */ plots: HistoryPoint[][] empty?: string + /** The x value under the pointer, and null once it leaves the plot. */ + onCursor?: (ts: number | null) => void + /** The x value clicked, or null for a click that landed on no point. */ + onSelect?: (ts: number | null) => void }) { const host = useRef(null) + const legend = useRef(null) const chart = useRef(null) + // The chart outlives a render, so its handlers are read through a ref + // rather than baked into the config it was built with. + const report = useRef({ onCursor, onSelect }) const { resolvedTheme } = useTheme() + useEffect(() => { + report.current = { onCursor, onSelect } + }) + const points = plots.reduce((total, plot) => total + plot.length, 0) // The identity of the series set: the chart is rebuilt when it changes, // while a new reading only sets its data. @@ -88,6 +114,11 @@ export function UplotChart({ ticks: { stroke: () => token("--border"), width: 1 }, font: `11px ${getComputedStyle(element).fontFamily}`, } + /** The x value the page was last told about, so a move within one bucket + * does not re-render it. */ + let told: number | null = null + const under = (self: uPlot) => + self.cursor.idx == null ? null : Number(self.data[0][self.cursor.idx]) const plot = new uPlot( { @@ -95,11 +126,38 @@ export function UplotChart({ height: canvasHeight(element), padding: PADDING, cursor: { y: false }, - legend: { live: true }, + legend: { + live: true, + // Mounted in its own row under the plot rather than inside it: a + // legend that wraps then takes height from the chart instead of + // spilling past the bottom of the card. + mount: (_self, element) => legend.current?.appendChild(element), + }, + hooks: { + setCursor: [ + (self) => { + const ts = under(self) + if (ts === told) return + told = ts + report.current.onCursor?.(ts) + }, + ], + ready: [ + (self) => { + self.over.addEventListener("click", () => + report.current.onSelect?.(under(self)), + ) + }, + ], + }, scales: { x: { time: true } }, axes: [ { ...axis, size: 28 }, - { ...axis, size: 46 }, + { + ...axis, + size: 46, + values: (_self: uPlot, ticks: number[]) => ticks.map(tick), + }, ], series: [ {}, @@ -109,6 +167,10 @@ export function UplotChart({ // Read at draw time, so a theme toggle is a redraw rather than a // rebuilt chart. stroke: () => seriesColor(index), + // The cursor readout is what decides how wide the legend gets, so + // it is rounded here and the unit named in the card's title. + value: (_self: uPlot, raw: number) => + Number.isFinite(raw) ? compact(raw) : "--", // Series arrive on their own clocks; a joined table is mostly // holes, and a line with a hole per point is not a line. spanGaps: true, @@ -134,6 +196,9 @@ export function UplotChart({ return () => { observer.disconnect() plot.destroy() + // The legend was moved out of the plot's root, so destroying it leaves + // the table behind. + legend.current?.replaceChildren() chart.current = null } }, [key, ready]) @@ -151,13 +216,18 @@ export function UplotChart({ }, [resolvedTheme]) return ( -
-
- {points === 0 ? ( -

- {empty} -

- ) : null} +
+
+
+ {points === 0 ? ( +

+ {empty} +

+ ) : null} +
+ {/* Kept at the legend's resting height, so the plot does not resize + under the pointer the first time a reading arrives. */} +
) } diff --git a/frontend/src/components/Flow/BrainView.tsx b/frontend/src/components/Flow/BrainView.tsx index 8f41ddc..8f3d8c6 100644 --- a/frontend/src/components/Flow/BrainView.tsx +++ b/frontend/src/components/Flow/BrainView.tsx @@ -1,8 +1,6 @@ import { useQuery } from "@tanstack/react-query" import { useNavigate } from "@tanstack/react-router" import { - Background, - BackgroundVariant, type Edge, type Node, ReactFlow, @@ -19,13 +17,11 @@ import { forceY, type SimulationNodeDatum, } from "d3-force" -import { Brain } from "lucide-react" import { useEffect, useMemo } from "react" import type { BrainGraph } from "@/client" import { BrainEdge } from "./BrainEdge" import { BrainNode, type BrainNodeData } from "./BrainNode" -import { CanvasTitle } from "./CanvasTitle" import "./flow.css" import { graphQueryOptions } from "./queries" import { useFlowSocket } from "./useFlowSocket" @@ -155,52 +151,50 @@ function BrainCanvas() { }, [shape, fitView]) return ( - <> - { - const [flow] = (node.data as BrainNodeData).flows - if (flow) - navigate({ to: "/flows/$flowName", params: { flowName: flow } }) - }} - className="h-full w-full" - > - - - -
- - - - Brain - - -
- + { + const [flow] = (node.data as BrainNodeData).flows + if (flow) + navigate({ to: "/flows/$flowName", params: { flowName: flow } }) + }} + className="brain-flat h-full w-full" + /> ) } /** * Every flow at once, merged on what each node talks to. * - * Read-only by design: what a neuron stands for lives in the flow it came - * from, and clicking one goes there. + * Sits flat on whatever surface the page paints — no card, no canvas of its + * own — because it is the overview at the top of Home rather than a screen you + * navigate to. Read-only by design: what a neuron stands for lives in the flow + * it came from, and clicking one goes there. */ export function BrainView() { return ( - - - +
+ + + +
) } diff --git a/frontend/src/components/Flow/flow.css b/frontend/src/components/Flow/flow.css index af18e38..47cc8e4 100644 --- a/frontend/src/components/Flow/flow.css +++ b/frontend/src/components/Flow/flow.css @@ -112,6 +112,14 @@ transform: translate(-50%, -50%); } +/* + * Embedded on Home rather than on a canvas of its own: the graph paints no + * surface, so the page's own background shows through behind the neurons. + */ +.react-flow.brain-flat { + --xy-background-color: transparent; +} + .brain-cell { cursor: pointer; transition: border-color var(--duration-base) var(--ease-standard); diff --git a/frontend/src/components/Health/HealthActivity.tsx b/frontend/src/components/Health/HealthActivity.tsx new file mode 100644 index 0000000..e7fd5b9 --- /dev/null +++ b/frontend/src/components/Health/HealthActivity.tsx @@ -0,0 +1,369 @@ +import { useQuery, useQueryClient } from "@tanstack/react-query" +import { ChevronDown, ChevronRight, X } from "lucide-react" +import { useEffect, useState } from "react" + +import type { EventRow, HistoryPoint } from "@/client" +import { UplotChart } from "@/components/Common/UplotChart" +import { useEngineEvents } from "@/components/Flow/liveStore" +import { PANEL_SECTION } from "@/components/Flow/SidePanel" +import { Button } from "@/components/ui/button" +import { cn, compact } from "@/lib/utils" +import { + ago, + auditQueryOptions, + CARD, + clock, + deadLetterQueryOptions, + failuresQueryOptions, + healthKeys, + minuteOf, + runsQueryOptions, + timeseriesQueryOptions, +} from "./queries" + +/** How much of each list is shown while it is not tied to a moment. */ +const RUNS_SHOWN = 15 +const FAILURES_SHOWN = 25 + +/** + * A moment picked off a chart. + * + * Hovering previews it, so the list under the pointer scrubs along with the + * cursor. A click holds it: reading the list means moving the pointer off the + * chart, and a held moment ignores the cursor until it is released. + */ +function useMoment() { + const [hovered, setHovered] = useState(null) + const [pinned, setPinned] = useState(null) + + useEffect(() => { + if (pinned === null) return + const release = (event: KeyboardEvent) => { + if (event.key === "Escape") setPinned(null) + } + window.addEventListener("keydown", release) + return () => window.removeEventListener("keydown", release) + }, [pinned]) + + return { + /** The minute the paired list is showing, or null for all of it. */ + at: pinned ?? hovered, + pinned, + onCursor: setHovered, + onSelect: (ts: number | null) => + setPinned((held) => (ts === null || ts === held ? null : ts)), + release: () => setPinned(null), + } +} + +type Moment = ReturnType + +/** One chart, and the moment it hands to its list. */ +function Chart({ + title, + labels, + plots, + moment, +}: { + title: string + labels: string[] + plots: HistoryPoint[][] + moment: Moment +}) { + return ( +
+
+

{title}

+ + {moment.pinned !== null ? "Esc clears" : "click to pin"} + +
+ +
+ ) +} + +/** What a list is showing, and the way back to all of it. */ +function ListHeader({ title, moment }: { title: string; moment: Moment }) { + return ( +
+

{title}

+ {moment.at !== null ? ( + + {moment.pinned !== null ? "pinned to" : "showing"} {clock(moment.at)} + + ) : null} + {moment.pinned !== null ? ( + + ) : null} +
+ ) +} + +function Failure({ event }: { event: EventRow }) { + const [open, setOpen] = useState(false) + const [first, ...rest] = event.detail.split("\n") + return ( +
+ + {open && rest.length ? ( + 
+          {rest.join("\n")}
+
+ ) : null} +
+ ) +} + +/** + * What the engine has been doing, as charts over a day and the records behind + * them. + * + * The two charts each drive the list beside them: throughput picks the runs of + * a minute, failures and timing picks that minute's failures. + */ +export function HealthActivity() { + const live = useEngineEvents() + const queryClient = useQueryClient() + const runsAt = useMoment() + const failuresAt = useMoment() + + const { data: series } = useQuery(timeseriesQueryOptions()) + const { data: runs } = useQuery(runsQueryOptions()) + const { data: failures } = useQuery(failuresQueryOptions()) + const { data: audit } = useQuery(auditQueryOptions()) + const { data: dead } = useQuery(deadLetterQueryOptions()) + + // Something just went wrong on the socket. The row for it is written on the + // collector's next flush, so the refetch waits that out rather than asking + // for a failure the database does not have yet. + // + // The newest event's minute, not the count: the count stops changing once the + // ring is full, and a per-event key would let a flapping node restart the + // timer forever without it ever firing. + const seen = live.length ? Math.floor(live[live.length - 1].ts / 60) : 0 + useEffect(() => { + if (!seen) return + const timer = setTimeout(() => { + queryClient.invalidateQueries({ queryKey: healthKeys.events }) + queryClient.invalidateQueries({ queryKey: healthKeys.summary }) + }, 16000) + return () => clearTimeout(timer) + }, [seen, queryClient]) + + const points = series ?? [] + const at = ( + pick: (point: (typeof points)[number]) => number, + ): HistoryPoint[] => + points.map((point) => ({ ts: point.ts, value: pick(point) })) + + const shownRuns = + runsAt.at === null + ? (runs ?? []).slice(0, RUNS_SHOWN) + : (runs ?? []).filter((run) => minuteOf(run.started_at) === runsAt.at) + const shownFailures = + failuresAt.at === null + ? (failures ?? []).slice(0, FAILURES_SHOWN) + : (failures ?? []).filter((event) => minuteOf(event.ts) === failuresAt.at) + + return ( + <> +
+ point.messages), + at((point) => point.executions), + ]} + moment={runsAt} + /> + point.errors), + at((point) => point.avg_ms), + at((point) => point.avg_lag_ms), + ]} + moment={failuresAt} + /> +
+ +
+
+ +
+ {shownRuns.length ? ( + shownRuns.map((run) => ( +
+ + {run.flow} + + + {run.source} + + + {run.status} + + + {compact(run.duration_ms)} ms + + + {ago(run.started_at)} + +
+ )) + ) : ( +

+ {runsAt.at === null + ? "No runs recorded yet." + : "No run from this minute is in the recent list."} +

+ )} +
+
+ +
+ +
+ {shownFailures.length ? ( + shownFailures.map((event) => ( + + )) + ) : ( +

+ {failuresAt.at === null + ? "Nothing has failed in the last day." + : "Nothing failed in this minute."} +

+ )} +
+
+
+ + {dead?.length ? ( +
+

Given up on

+
+ {dead.map((item) => ( +
+ + {item.node} + + {item.reason} + + {ago(item.ts)} + +
+ ))} +
+
+ ) : null} + +
+

Changes

+
+ {audit?.length ? ( + audit.map((event) => ( +
+ + {event.actor} {event.detail} + {event.flow ? ( + + {event.flow} + + ) : null} + + + {ago(event.ts)} + +
+ )) + ) : ( +

+ Nothing has changed yet. +

+ )} +
+
+ + ) +} diff --git a/frontend/src/components/Health/HealthOverview.tsx b/frontend/src/components/Health/HealthOverview.tsx new file mode 100644 index 0000000..ce42f1e --- /dev/null +++ b/frontend/src/components/Health/HealthOverview.tsx @@ -0,0 +1,195 @@ +import { useQuery } from "@tanstack/react-query" +import { Link } from "@tanstack/react-router" + +import type { FlowRollup, HistoryPoint } from "@/client" +import { shape } from "@/components/Flow/MessageSparkline" +import { PANEL_SECTION } from "@/components/Flow/SidePanel" +import { Badge } from "@/components/ui/badge" +import { compact } from "@/lib/utils" +import { + ago, + CARD, + failuresQueryOptions, + flowRollupsQueryOptions, + summaryQueryOptions, +} from "./queries" + +function Tile({ + label, + value, + note, +}: { + label: string + value: string + note?: string +}) { + return ( +
+
{label}
+
{value}
+ {note ? ( +
{note}
+ ) : null} +
+ ) +} + +/** A flow's execution trend, drawn from the 60 slices the rollup carries. */ +function Spark({ counts }: { counts: number[] }) { + const points: HistoryPoint[] = counts.map((value, index) => ({ + ts: index, + value, + })) + if (points.every((point) => point.value === 0)) { + return nothing yet + } + const { line } = shape(points) + return ( + + ) +} + +/** + * How the engine is doing, and how each flow has been doing for a day. + * + * The tiles are the standing state; the table below is the same day the charts + * cover, one row per flow. + */ +export function HealthOverview() { + const { data: summary } = useQuery(summaryQueryOptions()) + const { data: flows } = useQuery(flowRollupsQueryOptions()) + // Shares the list below's cache entry, for the one tile that dates them. + const { data: failures } = useQuery(failuresQueryOptions()) + + const queue = (summary?.queue ?? {}) as Record + const degraded = summary?.status === "degraded" + + return ( + <> +
+
+

Health

+ + {degraded ? "Degraded" : "Running normally"} + +
+ {summary?.problems.length ? ( +

+ {summary.problems.join(" · ")} +

+ ) : null} + +
+ + + + + +
+
+ +
+

Flow activity (24h)

+
+ + + + + + + + + + + + + {(flows ?? []).map((row: FlowRollup) => ( + + + + + + + + + ))} + {flows?.length === 0 ? ( + + + + ) : null} + +
FlowExecutionsErrorsAvgLagTrend
+ + {row.flow || "—"} + + {row.executions} + {row.errors ? ( + {row.errors} failed + ) : ( + none + )} + {compact(row.avg_ms)} ms{compact(row.avg_lag_ms)} ms + +
+ No flow has run in the last day. +
+
+
+ + ) +} diff --git a/frontend/src/components/Health/queries.ts b/frontend/src/components/Health/queries.ts new file mode 100644 index 0000000..97cf5fc --- /dev/null +++ b/frontend/src/components/Health/queries.ts @@ -0,0 +1,99 @@ +import { ObservabilityService } from "@/client" + +/** The live view; anything older is the collector's rollups. */ +export const HOURS = 24 + +/** Another tab can change things, and the engine fails flows on its own. */ +const REFRESH = 30_000 + +/** + * How deep the run and failure lists are read. + * + * Deeper than they are shown: picking a minute off a chart filters these rows + * client-side, and a list holding only the newest handful would have nothing + * to find for any minute but the current one. + * + * ponytail: the runs endpoint caps at 200 and takes no time range, so a busy + * engine still only covers its last minute or so. A `since` parameter is what + * would let a moment on the chart reach the whole day. + */ +const RUN_DEPTH = 200 +const EVENT_DEPTH = 100 + +export const healthKeys = { + all: ["observability"] as const, + summary: ["observability", "summary"] as const, + events: ["observability", "events"] as const, +} + +/** The resting surface these screens are built from. */ +export const CARD = "rounded-lg border border-border bg-card p-4 shadow-e1" + +export const summaryQueryOptions = () => ({ + queryKey: healthKeys.summary, + queryFn: () => ObservabilityService.readSummary(), + refetchInterval: 10_000, +}) + +export const timeseriesQueryOptions = () => ({ + queryKey: ["observability", "timeseries", HOURS] as const, + queryFn: () => ObservabilityService.readTimeseries({ hours: HOURS }), + refetchInterval: REFRESH, +}) + +export const flowRollupsQueryOptions = () => ({ + queryKey: ["observability", "flows", HOURS] as const, + queryFn: () => ObservabilityService.readFlowRollups({ hours: HOURS }), + refetchInterval: REFRESH, +}) + +export const runsQueryOptions = () => ({ + queryKey: ["observability", "runs"] as const, + queryFn: () => ObservabilityService.readRuns({ limit: RUN_DEPTH }), + refetchInterval: REFRESH, +}) + +export const failuresQueryOptions = () => ({ + queryKey: [...healthKeys.events, "failure"] as const, + queryFn: () => + ObservabilityService.readEvents({ kind: "failure", limit: EVENT_DEPTH }), + refetchInterval: REFRESH, +}) + +export const auditQueryOptions = () => ({ + queryKey: [...healthKeys.events, "audit"] as const, + queryFn: () => ObservabilityService.readEvents({ kind: "audit", limit: 15 }), + refetchInterval: REFRESH, +}) + +export const deadLetterQueryOptions = () => ({ + queryKey: ["observability", "dead-letter"] as const, + queryFn: () => ObservabilityService.readDeadLetters({ limit: 20 }), + refetchInterval: REFRESH, +}) + +/** How long ago something happened, in the coarsest unit that still says it. */ +export function ago(ts: string | number | null | undefined): string { + if (!ts) return "—" + const stamp = typeof ts === "number" ? ts * 1000 : Date.parse(ts) + const seconds = Math.max(0, (Date.now() - stamp) / 1000) + if (seconds < 90) return `${Math.round(seconds)}s ago` + if (seconds < 5400) return `${Math.round(seconds / 60)}m ago` + if (seconds < 172800) return `${Math.round(seconds / 3600)}h ago` + return `${Math.round(seconds / 86400)}d ago` +} + +/** + * The minute a timestamp belongs to, as unix seconds. + * + * Minutes are the granularity the engine stores its metrics at, so this is + * also the finest moment a chart can be asked about. + */ +export const minuteOf = (ts: string) => Math.floor(Date.parse(ts) / 60000) * 60 + +/** A moment as a clock reading, which is how the charts label their axis. */ +export const clock = (ts: number) => + new Date(ts * 1000).toLocaleTimeString([], { + hour: "2-digit", + minute: "2-digit", + }) diff --git a/frontend/src/components/Sidebar/AppSidebar.tsx b/frontend/src/components/Sidebar/AppSidebar.tsx index 314166b..841d08a 100644 --- a/frontend/src/components/Sidebar/AppSidebar.tsx +++ b/frontend/src/components/Sidebar/AppSidebar.tsx @@ -1,7 +1,5 @@ import { - Activity, Bell, - Brain, Home, KeyRound, LayoutDashboard, @@ -24,12 +22,11 @@ import useAuth from "@/hooks/useAuth" import { type Item, Main } from "./Main" const baseItems: Item[] = [ - // "Home" rather than "Dashboard": dashboards are their own thing now. + // "Home" rather than "Dashboard": dashboards are their own thing now, and + // the brain and the health screens are sections of Home rather than routes. { icon: Home, title: "Home", path: "/" }, { icon: Workflow, title: "Flows", path: "/flows" }, - { icon: Brain, title: "Brain", path: "/brain" }, { icon: LayoutDashboard, title: "Dashboards", path: "/dashboards" }, - { icon: Activity, title: "Health", path: "/health" }, // Both are engine-wide operator settings rather than personal ones, so they // sit here and not among the per-user tabs under Settings. { icon: KeyRound, title: "Secrets", path: "/secrets" }, diff --git a/frontend/src/index.css b/frontend/src/index.css index b632ebd..cb3daaa 100644 --- a/frontend/src/index.css +++ b/frontend/src/index.css @@ -173,6 +173,18 @@ [role="button"] { cursor: pointer; } + /* Number fields are typed into, not clicked up and down one step at a time, + and the spinner crowds the pill. */ + input[type="number"] { + -moz-appearance: textfield; + appearance: textfield; + } + input[type="number"]::-webkit-outer-spin-button, + input[type="number"]::-webkit-inner-spin-button { + -webkit-appearance: none; + appearance: none; + margin: 0; + } } /* Display face for headings / wordmarks. Plain class so it stays outside the diff --git a/frontend/src/lib/utils.ts b/frontend/src/lib/utils.ts index d084cca..d146e78 100644 --- a/frontend/src/lib/utils.ts +++ b/frontend/src/lib/utils.ts @@ -4,3 +4,13 @@ import { twMerge } from "tailwind-merge" export function cn(...inputs: ClassValue[]) { return twMerge(clsx(inputs)) } + +/** + * A reading at about three significant digits: 203.63 → 204, 2.714 → 2.7. + * + * Shared by the health tables and the chart legends, so a value read off the + * cursor cannot grow wide enough to push a legend out of its card. + */ +export function compact(value: number): string { + return String(Math.abs(value) >= 100 ? Math.round(value) : +value.toFixed(1)) +} diff --git a/frontend/src/routeTree.gen.ts b/frontend/src/routeTree.gen.ts index 2a84ae2..478a5b3 100644 --- a/frontend/src/routeTree.gen.ts +++ b/frontend/src/routeTree.gen.ts @@ -21,10 +21,8 @@ import { Route as OauthAuthorizeRouteImport } from './routes/oauth.authorize' import { Route as LayoutSettingsRouteImport } from './routes/_layout/settings' import { Route as LayoutSecretsRouteImport } from './routes/_layout/secrets' import { Route as LayoutModulesRouteImport } from './routes/_layout/modules' -import { Route as LayoutHealthRouteImport } from './routes/_layout/health' import { Route as LayoutAlertsRouteImport } from './routes/_layout/alerts' import { Route as LayoutAdminRouteImport } from './routes/_layout/admin' -import { Route as CanvasBrainRouteImport } from './routes/_canvas/brain' import { Route as LayoutFlowsIndexRouteImport } from './routes/_layout/flows/index' import { Route as LayoutDashboardsIndexRouteImport } from './routes/_layout/dashboards/index' import { Route as CanvasFlowsFlowNameRouteImport } from './routes/_canvas/flows/$flowName' @@ -88,11 +86,6 @@ const LayoutModulesRoute = LayoutModulesRouteImport.update({ path: '/modules', getParentRoute: () => LayoutRoute, } as any) -const LayoutHealthRoute = LayoutHealthRouteImport.update({ - id: '/health', - path: '/health', - getParentRoute: () => LayoutRoute, -} as any) const LayoutAlertsRoute = LayoutAlertsRouteImport.update({ id: '/alerts', path: '/alerts', @@ -103,11 +96,6 @@ const LayoutAdminRoute = LayoutAdminRouteImport.update({ path: '/admin', getParentRoute: () => LayoutRoute, } as any) -const CanvasBrainRoute = CanvasBrainRouteImport.update({ - id: '/brain', - path: '/brain', - getParentRoute: () => CanvasRoute, -} as any) const LayoutFlowsIndexRoute = LayoutFlowsIndexRouteImport.update({ id: '/flows/', path: '/flows/', @@ -135,10 +123,8 @@ export interface FileRoutesByFullPath { '/recover-password': typeof RecoverPasswordRoute '/reset-password': typeof ResetPasswordRoute '/signup': typeof SignupRoute - '/brain': typeof CanvasBrainRoute '/admin': typeof LayoutAdminRoute '/alerts': typeof LayoutAlertsRoute - '/health': typeof LayoutHealthRoute '/modules': typeof LayoutModulesRoute '/secrets': typeof LayoutSecretsRoute '/settings': typeof LayoutSettingsRoute @@ -155,10 +141,8 @@ export interface FileRoutesByTo { '/recover-password': typeof RecoverPasswordRoute '/reset-password': typeof ResetPasswordRoute '/signup': typeof SignupRoute - '/brain': typeof CanvasBrainRoute '/admin': typeof LayoutAdminRoute '/alerts': typeof LayoutAlertsRoute - '/health': typeof LayoutHealthRoute '/modules': typeof LayoutModulesRoute '/secrets': typeof LayoutSecretsRoute '/settings': typeof LayoutSettingsRoute @@ -177,10 +161,8 @@ export interface FileRoutesById { '/recover-password': typeof RecoverPasswordRoute '/reset-password': typeof ResetPasswordRoute '/signup': typeof SignupRoute - '/_canvas/brain': typeof CanvasBrainRoute '/_layout/admin': typeof LayoutAdminRoute '/_layout/alerts': typeof LayoutAlertsRoute - '/_layout/health': typeof LayoutHealthRoute '/_layout/modules': typeof LayoutModulesRoute '/_layout/secrets': typeof LayoutSecretsRoute '/_layout/settings': typeof LayoutSettingsRoute @@ -200,10 +182,8 @@ export interface FileRouteTypes { | '/recover-password' | '/reset-password' | '/signup' - | '/brain' | '/admin' | '/alerts' - | '/health' | '/modules' | '/secrets' | '/settings' @@ -220,10 +200,8 @@ export interface FileRouteTypes { | '/recover-password' | '/reset-password' | '/signup' - | '/brain' | '/admin' | '/alerts' - | '/health' | '/modules' | '/secrets' | '/settings' @@ -241,10 +219,8 @@ export interface FileRouteTypes { | '/recover-password' | '/reset-password' | '/signup' - | '/_canvas/brain' | '/_layout/admin' | '/_layout/alerts' - | '/_layout/health' | '/_layout/modules' | '/_layout/secrets' | '/_layout/settings' @@ -354,13 +330,6 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof LayoutModulesRouteImport parentRoute: typeof LayoutRoute } - '/_layout/health': { - id: '/_layout/health' - path: '/health' - fullPath: '/health' - preLoaderRoute: typeof LayoutHealthRouteImport - parentRoute: typeof LayoutRoute - } '/_layout/alerts': { id: '/_layout/alerts' path: '/alerts' @@ -375,13 +344,6 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof LayoutAdminRouteImport parentRoute: typeof LayoutRoute } - '/_canvas/brain': { - id: '/_canvas/brain' - path: '/brain' - fullPath: '/brain' - preLoaderRoute: typeof CanvasBrainRouteImport - parentRoute: typeof CanvasRoute - } '/_layout/flows/': { id: '/_layout/flows/' path: '/flows' @@ -414,13 +376,11 @@ declare module '@tanstack/react-router' { } interface CanvasRouteChildren { - CanvasBrainRoute: typeof CanvasBrainRoute CanvasDashboardsNameRoute: typeof CanvasDashboardsNameRoute CanvasFlowsFlowNameRoute: typeof CanvasFlowsFlowNameRoute } const CanvasRouteChildren: CanvasRouteChildren = { - CanvasBrainRoute: CanvasBrainRoute, CanvasDashboardsNameRoute: CanvasDashboardsNameRoute, CanvasFlowsFlowNameRoute: CanvasFlowsFlowNameRoute, } @@ -431,7 +391,6 @@ const CanvasRouteWithChildren = interface LayoutRouteChildren { LayoutAdminRoute: typeof LayoutAdminRoute LayoutAlertsRoute: typeof LayoutAlertsRoute - LayoutHealthRoute: typeof LayoutHealthRoute LayoutModulesRoute: typeof LayoutModulesRoute LayoutSecretsRoute: typeof LayoutSecretsRoute LayoutSettingsRoute: typeof LayoutSettingsRoute @@ -443,7 +402,6 @@ interface LayoutRouteChildren { const LayoutRouteChildren: LayoutRouteChildren = { LayoutAdminRoute: LayoutAdminRoute, LayoutAlertsRoute: LayoutAlertsRoute, - LayoutHealthRoute: LayoutHealthRoute, LayoutModulesRoute: LayoutModulesRoute, LayoutSecretsRoute: LayoutSecretsRoute, LayoutSettingsRoute: LayoutSettingsRoute, diff --git a/frontend/src/routes/_canvas/brain.tsx b/frontend/src/routes/_canvas/brain.tsx deleted file mode 100644 index af735f2..0000000 --- a/frontend/src/routes/_canvas/brain.tsx +++ /dev/null @@ -1,8 +0,0 @@ -import { createFileRoute } from "@tanstack/react-router" - -import { BrainView } from "@/components/Flow/BrainView" - -export const Route = createFileRoute("/_canvas/brain")({ - component: BrainView, - head: () => ({ meta: [{ title: "Brain - Fluksio" }] }), -}) diff --git a/frontend/src/routes/_layout/health.tsx b/frontend/src/routes/_layout/health.tsx deleted file mode 100644 index 48ef983..0000000 --- a/frontend/src/routes/_layout/health.tsx +++ /dev/null @@ -1,461 +0,0 @@ -import { useQuery, useQueryClient } from "@tanstack/react-query" -import { createFileRoute, Link } from "@tanstack/react-router" -import { ChevronDown, ChevronRight } from "lucide-react" -import { useEffect, useState } from "react" - -import { - type EventRow, - type FlowRollup, - type HistoryPoint, - ObservabilityService, -} from "@/client" -import { UplotChart } from "@/components/Common/UplotChart" -import { useEngineEvents } from "@/components/Flow/liveStore" -import { shape } from "@/components/Flow/MessageSparkline" -import { PANEL_SECTION } from "@/components/Flow/SidePanel" -import { useFlowSocket } from "@/components/Flow/useFlowSocket" -import { Badge } from "@/components/ui/badge" - -export const Route = createFileRoute("/_layout/health")({ - component: Health, - head: () => ({ - meta: [ - { - title: "Health - Fluksio", - }, - ], - }), -}) - -const healthKeys = { - all: ["observability"] as const, - summary: ["observability", "summary"] as const, - events: ["observability", "events"] as const, -} - -/** The live view; anything older is the collector's rollups. */ -const HOURS = 24 -const CARD = "rounded-lg border border-border bg-card p-4 shadow-e1" - -function ago(ts: string | number | null | undefined): string { - if (!ts) return "—" - const stamp = typeof ts === "number" ? ts * 1000 : Date.parse(ts) - const seconds = Math.max(0, (Date.now() - stamp) / 1000) - if (seconds < 90) return `${Math.round(seconds)}s ago` - if (seconds < 5400) return `${Math.round(seconds / 60)}m ago` - if (seconds < 172800) return `${Math.round(seconds / 3600)}h ago` - return `${Math.round(seconds / 86400)}d ago` -} - -const round = (value: number) => - value >= 100 ? Math.round(value) : +value.toFixed(1) - -function Tile({ - label, - value, - note, -}: { - label: string - value: string - note?: string -}) { - return ( -
-
{label}
-
{value}
- {note ? ( -
{note}
- ) : null} -
- ) -} - -/** A flow's execution trend, drawn from the 60 slices the rollup carries. */ -function Spark({ counts }: { counts: number[] }) { - const points: HistoryPoint[] = counts.map((value, index) => ({ - ts: index, - value, - })) - if (points.every((point) => point.value === 0)) { - return nothing yet - } - const { line } = shape(points) - return ( - - ) -} - -function Failure({ event }: { event: EventRow }) { - const [open, setOpen] = useState(false) - const [first, ...rest] = event.detail.split("\n") - return ( -
- - {open && rest.length ? ( - 
-          {rest.join("\n")}
-
- ) : null} -
- ) -} - -function Health() { - // The page is a socket subscriber like the editor: a failure should appear - // without waiting for the next poll. - useFlowSocket() - const live = useEngineEvents() - const queryClient = useQueryClient() - - const { data: summary } = useQuery({ - queryKey: healthKeys.summary, - queryFn: () => ObservabilityService.readSummary(), - refetchInterval: 10000, - }) - const { data: series } = useQuery({ - queryKey: ["observability", "timeseries", HOURS], - queryFn: () => ObservabilityService.readTimeseries({ hours: HOURS }), - refetchInterval: 30000, - }) - const { data: flows } = useQuery({ - queryKey: ["observability", "flows", HOURS], - queryFn: () => ObservabilityService.readFlowRollups({ hours: HOURS }), - refetchInterval: 30000, - }) - const { data: runs } = useQuery({ - queryKey: ["observability", "runs"], - queryFn: () => ObservabilityService.readRuns({ limit: 15 }), - refetchInterval: 30000, - }) - const { data: failures } = useQuery({ - queryKey: [...healthKeys.events, "failure"], - queryFn: () => - ObservabilityService.readEvents({ kind: "failure", limit: 25 }), - refetchInterval: 30000, - }) - const { data: audit } = useQuery({ - queryKey: [...healthKeys.events, "audit"], - queryFn: () => - ObservabilityService.readEvents({ kind: "audit", limit: 15 }), - refetchInterval: 30000, - }) - const { data: dead } = useQuery({ - queryKey: ["observability", "dead-letter"], - queryFn: () => ObservabilityService.readDeadLetters({ limit: 20 }), - refetchInterval: 30000, - }) - - // Something just went wrong on the socket. The row for it is written on the - // collector's next flush, so the refetch waits that out rather than asking - // for a failure the database does not have yet. - // - // The newest event's minute, not the count: the count stops changing once the - // ring is full, and a per-event key would let a flapping node restart the - // timer forever without it ever firing. - const seen = live.length ? Math.floor(live[live.length - 1].ts / 60) : 0 - useEffect(() => { - if (!seen) return - const timer = setTimeout(() => { - queryClient.invalidateQueries({ queryKey: healthKeys.events }) - queryClient.invalidateQueries({ queryKey: healthKeys.summary }) - }, 16000) - return () => clearTimeout(timer) - }, [seen, queryClient]) - - const points = series ?? [] - const at = ( - pick: (point: (typeof points)[number]) => number, - ): HistoryPoint[] => - points.map((point) => ({ ts: point.ts, value: pick(point) })) - - const queue = (summary?.queue ?? {}) as Record - const degraded = summary?.status === "degraded" - - return ( -
-
-
-

Health

- - {degraded ? "Degraded" : "Running normally"} - -
-

- {summary?.problems.length - ? summary.problems.join(" · ") - : "What the engine has been doing over the last day, and what it is doing now."} -

-
- -
- - - - - -
- -
-
-

Throughput per minute

- point.messages), - at((point) => point.executions), - ]} - empty="Nothing has run yet." - /> -
-
-

Failures and timing

- point.errors), - at((point) => point.avg_ms), - at((point) => point.avg_lag_ms), - ]} - empty="Nothing has run yet." - /> -
-
- -
-

Flows

-
- - - - - - - - - - - - - {(flows ?? []).map((row: FlowRollup) => ( - - - - - - - - - ))} - {flows?.length === 0 ? ( - - - - ) : null} - -
FlowExecutionsErrorsAvgLagTrend
- - {row.flow || "—"} - - {row.executions} - {row.errors ? ( - {row.errors} failed - ) : ( - none - )} - {round(row.avg_ms)} ms{round(row.avg_lag_ms)} ms - -
- No flow has run in the last day. -
-
-
- -
-
-

Recent runs

-
- {runs?.length ? ( - runs.map((run) => ( -
- - {run.flow} - - - {run.source} - - - {run.status} - - - {round(run.duration_ms)} ms - - - {ago(run.started_at)} - -
- )) - ) : ( -

- No runs recorded yet. -

- )} -
-
- -
-

Failures

-
- {failures?.length ? ( - failures.map((event) => ) - ) : ( -

- Nothing has failed in the last day. -

- )} -
-
-
- - {dead?.length ? ( -
-

Given up on

-
- {dead.map((item) => ( -
- - {item.node} - - {item.reason} - - {ago(item.ts)} - -
- ))} -
-
- ) : null} - -
-

Changes

-
- {audit?.length ? ( - audit.map((event) => ( -
- - {event.actor} {event.detail} - {event.flow ? ( - - {event.flow} - - ) : null} - - - {ago(event.ts)} - -
- )) - ) : ( -

- Nothing has changed yet. -

- )} -
-
-
- ) -} diff --git a/frontend/src/routes/_layout/index.tsx b/frontend/src/routes/_layout/index.tsx index cc87acd..af2a21b 100644 --- a/frontend/src/routes/_layout/index.tsx +++ b/frontend/src/routes/_layout/index.tsx @@ -3,7 +3,10 @@ import { createFileRoute, Link } from "@tanstack/react-router" import { AlertCircle, Workflow } from "lucide-react" import { type FlowSummary, FlowsService } from "@/client" +import { BrainView } from "@/components/Flow/BrainView" import { flowKeys, flowsQueryOptions } from "@/components/Flow/queries" +import { HealthActivity } from "@/components/Health/HealthActivity" +import { HealthOverview } from "@/components/Health/HealthOverview" import { Badge } from "@/components/ui/badge" import { Card } from "@/components/ui/card" import { Skeleton } from "@/components/ui/skeleton" @@ -25,17 +28,6 @@ export const Route = createFileRoute("/_layout/")({ /** Another tab can stop a flow, and the engine can fail one on its own. */ const REFRESH_INTERVAL = 10_000 -function Tile({ label, value }: { label: string; value: number }) { - return ( - -
-

{value}

-

{label}

-
- - ) -} - function FlowRow({ flow }: { flow: FlowSummary }) { const queryClient = useQueryClient() const { showErrorToast } = useCustomToast() @@ -101,6 +93,11 @@ function FlowRow({ flow }: { flow: FlowSummary }) { ) } +/** + * The one overview: what the engine is wired up as, what is running, and how + * it has been doing. The brain and the health screens compose in here rather + * than living at routes of their own. + */ function Dashboard() { const { user: currentUser } = useAuth() const { data, isPending } = useQuery({ @@ -110,7 +107,6 @@ function Dashboard() { const flows = data?.data ?? [] const running = flows.filter((flow) => flow.enabled ?? true).length - const failing = flows.filter((flow) => (flow.error_count ?? 0) > 0).length return (
@@ -125,11 +121,7 @@ function Dashboard() {

-
- - - -
+ {isPending ? ( @@ -153,6 +145,9 @@ function Dashboard() { flows.map((flow) => ) )} + + +
) }