evaluation-task · Source-linked discovery
Needle in a Haystack (NIAH): In-Context Retrieval Benchmark for Long Context LLMs
NIAH evaluates in-context retrieval ability of long context LLMs by testing a model's ability to extract factual information from long-context inputs.
OriginElliot Nelson, Georgios Kollias, Payel Das et al.TopicsGeneral capabilityStatusimported
Can support
Not independently assessed by FronteraEval yet.
Cannot support by itself
No inference beyond the upstream source should be made until the protocol is reviewed.