evaluation-task · Source-linked discovery
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
Evaluates reading comprehension where models must resolve references in a question, perhaps to multiple input positions, and perform discrete operations over them (such as addition, counting, or sorting).
OriginDheeru Dua, Yizhong Wang, Pradeep Dasigi et al.TopicsGeneral capabilityStatusimported
Can support
Not independently assessed by FronteraEval yet.
Cannot support by itself
No inference beyond the upstream source should be made until the protocol is reviewed.