evaluation-task · Source-linked discovery
Humanity's Last Exam
Humanity's Last Exam is a QA benchmark containing 2,500 questions across 100+ subjects, including mathematics, sciences, and humanities. About a quarter of the questions are multiple-choice and the rest open-ended, and about 14% include images. The prompt requests reasoning, a final answer, and 0-100% confidence; LLM judges mark correctness to produce accuracy and calibration metrics.
OriginLong Phan, Alice Gatti, Ziwen Han et al.TopicsGeneral capabilityStatusimported
Can support
Not independently assessed by FronteraEval yet.
Cannot support by itself
No inference beyond the upstream source should be made until the protocol is reviewed.