Back to all work
Assay
Blinded, citation-verified benchmark for how well LLMs reason over a domain corpus
Python, LLM evaluation, Statistics
Overview
Assay anonymises model answers and has a three-family jury score them, so no model judges its own family. Citations are checked deterministically against a pinned corpus. Rankings use Bradley-Terry ratings with bootstrap confidence intervals, and overlapping intervals are reported as ties, never as wins.