evaluation-package · Source-linked discovery
CharXiv
CharXiv evaluates multimodal LLMs on chart understanding using 2,323 real-world charts from arXiv papers. It contains two question types: descriptive (19 template-based questions per chart testing basic element extraction, with 25% unanswerable) and reasoning (one hand-crafted free-form question per chart requiring numerical/visual synthesis). Answers are graded by GPT-4o using binary correctness scoring. The implementation supports filtering by subset (descriptive/reasoning) and field of study.
OriginZirui Wang, Mengzhou Xia, Luxi He et al.TopicsGeneral capabilityStatuscatalogued
Can support
Not independently assessed by FronteraEval yet.
Cannot support by itself
No inference beyond the upstream source should be made until the protocol is reviewed.