Show what differs between the runs being compared
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m38s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m40s
pre-commit / pre-commit (push) Failing after 2m44s
Test Backend / test-backend (push) Successful in 2m17s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Failing after 1m3s
Docs / docs (push) Successful in 23s
Playwright Tests / test-playwright (1, 2) (push) Failing after 2m38s
Playwright Tests / test-playwright (2, 2) (push) Failing after 1m40s
pre-commit / pre-commit (push) Failing after 2m44s
Test Backend / test-backend (push) Successful in 2m17s
Compose Smoke Test / test-compose (push) Successful in 32s
Playwright Tests / merge-reports (push) Failing after 1m3s
This commit is contained in:
@@ -205,6 +205,13 @@ export function RunsScreen({
|
|||||||
flow={runs.find((run) => run.id === selected[0])?.flow}
|
flow={runs.find((run) => run.id === selected[0])?.flow}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{/* Below the chart, and only once there are two runs to hold apart.
|
||||||
|
The table above keeps its height and the page scrolls instead:
|
||||||
|
what differs is worth reading in full. */}
|
||||||
|
{selected.length > 1 && (
|
||||||
|
<ParamDiff rows={runs.filter((run) => selected.includes(run.id))} />
|
||||||
|
)}
|
||||||
</section>
|
</section>
|
||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
@@ -546,3 +553,93 @@ function Compare({
|
|||||||
</section>
|
</section>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What actually differs between the runs being compared.
|
||||||
|
*
|
||||||
|
* Only the parameters that vary. Everything they share is by definition not
|
||||||
|
* why their curves came out differently, and a sweep shares almost everything
|
||||||
|
* — so listing it in full would bury the two knobs that were turned.
|
||||||
|
*
|
||||||
|
* Runs are the rows, as they are in the table above: a comparison of fifty
|
||||||
|
* grows downwards, where there is somewhere to grow.
|
||||||
|
*/
|
||||||
|
function ParamDiff({ rows }: { rows: RunRow[] }) {
|
||||||
|
const varying = varyingKeys(rows)
|
||||||
|
// Not a parameter, but it is part of what produced the number, and in a
|
||||||
|
// sweep it is often the only thing that moved.
|
||||||
|
const seedVaries = new Set(rows.map((run) => run.seed ?? null)).size > 1
|
||||||
|
const keys = new Set<string>()
|
||||||
|
for (const run of rows)
|
||||||
|
for (const key of Object.keys(run.params)) keys.add(key)
|
||||||
|
const shared = keys.size - varying.length
|
||||||
|
|
||||||
|
if (varying.length === 0 && !seedVaries) {
|
||||||
|
return (
|
||||||
|
<p className="text-muted-foreground text-sm">
|
||||||
|
{keys.size === 0
|
||||||
|
? "These runs carried no parameters of their own; each took what its flow declares."
|
||||||
|
: `All ${keys.size} parameter${keys.size === 1 ? "" : "s"} are the same across these runs, and so is the seed. Whatever separates them is not in what they were asked.`}
|
||||||
|
</p>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<section className="flex flex-col gap-2">
|
||||||
|
<div className="flex flex-wrap items-baseline gap-x-3">
|
||||||
|
<h2 className="font-medium text-sm">What differs</h2>
|
||||||
|
<p className="text-muted-foreground text-xs">
|
||||||
|
{varying.length + (seedVaries ? 1 : 0)} of {keys.size + 1} varied
|
||||||
|
{shared > 0
|
||||||
|
? shared === 1
|
||||||
|
? "; the other one is the same across all of them"
|
||||||
|
: `; the other ${shared} are the same across all of them`
|
||||||
|
: ""}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<Table containerClassName="overflow-x-auto">
|
||||||
|
<TableHeader className="bg-muted">
|
||||||
|
<TableRow className="hover:bg-transparent">
|
||||||
|
<TableHead>Run</TableHead>
|
||||||
|
{seedVaries && <TableHead>seed</TableHead>}
|
||||||
|
{varying.map((key) => (
|
||||||
|
<TableHead key={key}>{key}</TableHead>
|
||||||
|
))}
|
||||||
|
</TableRow>
|
||||||
|
</TableHeader>
|
||||||
|
<TableBody>
|
||||||
|
{rows.map((run) => (
|
||||||
|
<TableRow key={run.id} data-testid="diff-row">
|
||||||
|
<TableCell>
|
||||||
|
<Link
|
||||||
|
to="/runs/$id"
|
||||||
|
params={{ id: run.id }}
|
||||||
|
className="font-mono text-sm hover:underline"
|
||||||
|
>
|
||||||
|
{shortId(run.id)}
|
||||||
|
</Link>
|
||||||
|
</TableCell>
|
||||||
|
{seedVaries && (
|
||||||
|
<TableCell className="text-muted-foreground text-sm tabular-nums">
|
||||||
|
{run.seed ?? "—"}
|
||||||
|
</TableCell>
|
||||||
|
)}
|
||||||
|
{varying.map((key) => (
|
||||||
|
<TableCell key={key}>
|
||||||
|
{key in run.params ? (
|
||||||
|
<ParamValue value={run.params[key]} />
|
||||||
|
) : (
|
||||||
|
// Not "empty": this run was never given the parameter, and
|
||||||
|
// took whatever the flow declares as its default.
|
||||||
|
<span className="text-muted-foreground text-xs">unset</span>
|
||||||
|
)}
|
||||||
|
</TableCell>
|
||||||
|
))}
|
||||||
|
</TableRow>
|
||||||
|
))}
|
||||||
|
</TableBody>
|
||||||
|
</Table>
|
||||||
|
</section>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user