evaluation-package · Source-linked discovery

SPaRC

SPaRC evaluates spatial and rule-based reasoning via 500 test 2D grid pathfinding puzzles (difficulty 1-5) inspired by The Witness video game. Each puzzle requires drawing a continuous, non-self-intersecting path from start to end satisfying combinatorial rules (dot collection, gap avoidance, color separation, star pairing, triangle edge-counting, polyomino fitting, ylop subtraction). Scoring is exact-match: a model's extracted coordinate path must match one of the brute-force-verified stored solutions. Metrics include per-difficulty accuracy and invalid-path rate.