Compare eval runs

Diffs two runs: aggregate deltas, and per-query wins and losses.

admin.dataSourceEvals.compare({ runA, runB })

Puts two runs side by side: the delta on every aggregate, b minus a, and, when both share a set, the queries that improved and regressed with their rank change. It is how a retrieval config change is judged, not guessed.

Parameters

runA

string

RequiredBaseline run id (positional).

runB

string

RequiredRun to compare against it (positional).

CLI

remy-admin datasources eval compare run_before run_after