Risk and capability map

Evaluation topics

Topics help with discovery. They do not imply that the included evaluations measure the same construct.

• Autonomy & agents43 recordsTool use, long-horizon task completion, self-directed work and operation in external environments.
• AI R&D10 recordsResearch engineering, model development and capabilities that may accelerate AI progress.
• Cyber29 recordsCybersecurity knowledge, vulnerability discovery, exploitation, defence and autonomous operations.
• Bio / CBRN15 recordsHazardous biological, chemical, radiological or nuclear knowledge and operational assistance.
• Deception & misalignment20 recordsScheming, covert action, strategic deception, sandbagging and misaligned agent behaviour.
• Human influence & agency11 recordsPersuasion, manipulation, social engineering, trust formation and effects on human agency.
• Safeguards25 recordsRefusal, jailbreak robustness, harmful-response prevention, monitoring and defensive controls.
• Evaluation integrity17 recordsValidity, contamination, elicitation, judge reliability, eval awareness and protocol integrity.
• General capability164 recordsReasoning, knowledge, coding, mathematics and broad task performance used as capability context.
• Multimodal8 recordsEvaluations requiring or assessing combinations of text, images, audio or other modalities.