evaluation-package · Source-linked discovery

VoxParity

VoxParity tests whether voice agents change their typed tool call when an audible cue (emotional delivery, a second voice, environmental sound, speaker age, disfluency, a masked word, sarcasm) changes the correct action while the transcript stays fixed. This task runs the public development split: 40 of the paper's 183 scenarios, as 81 audio samples plus 40 transcript-only twins; the other 143 are held out. Scoring is judge-free: the first-turn tool call is matched on tool name and typed arguments, with partial credit. Audio-minus-twin metrics show whether hearing the call changes the action, and a summary script adds the paper's words-only null test against a Whisper + LLM cascade. Install with `uv sync --extra inspect`.