Evaluation topic

Cyber

Cybersecurity knowledge, vulnerability discovery, exploitation, defence and autonomous operations.

33 records are tagged here. Inclusion does not establish construct equivalence, completeness, or comparability.

• CTI Realm 25Arjun Chakraborty, Sandra Ho, Adam Cook et al.
• CTI Realm 25 MinimalArjun Chakraborty, Sandra Ho, Adam Cook et al.
• CTI Realm 25 SeededArjun Chakraborty, Sandra Ho, Adam Cook et al.
• CTI Realm 50Arjun Chakraborty, Sandra Ho, Adam Cook et al.
• CVEBench: Benchmark for AI Agents Ability to Exploit Real-World Web Application VulnerabilitiesYuxuan Zhu, Antony Kellermann, Dylan Bowman et al.
• Cybench: Capture-The-Flag Cybersecurity ChallengesAndy K. Zhang, Neil Perry, Riya Dulepet et al.
• CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at ScaleZhun Wang, Tianneng Shi, Jingxuan He et al.
• Cybermetric 10000Norbert Tihanyi, Mohamed Amine Ferrag, Ridhi Jain et al.
• Cybermetric 2000Norbert Tihanyi, Mohamed Amine Ferrag, Ridhi Jain et al.
• Cybermetric 500Norbert Tihanyi, Mohamed Amine Ferrag, Ridhi Jain et al.
• Cybermetric 80Norbert Tihanyi, Mohamed Amine Ferrag, Ridhi Jain et al.
• CyberSecEvalMeta
• Cyse2 Prompt InjectionManish Bhatt, Sahana Chennabasappa, Yue Li et al.
• Cyse2 Vulnerability ExploitManish Bhatt, Sahana Chennabasappa, Yue Li et al.
• CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language ModelsManish Bhatt, Sahana Chennabasappa, Cyrus Nikolaidis et al.
• Cyse4 Malware AnalysisManish Bhatt, Sahana Chennabasappa, Yue Li et al.
• Cyse4 MitreManish Bhatt, Sahana Chennabasappa, Yue Li et al.
• Cyse4 Mitre FrrManish Bhatt, Sahana Chennabasappa, Yue Li et al.
• Cyse4 Multilingual Prompt InjectionManish Bhatt, Sahana Chennabasappa, Yue Li et al.
• Cyse4 Multiturn PhishingManish Bhatt, Sahana Chennabasappa, Yue Li et al.
• Dangerous Capability EvaluationsGoogle DeepMind
• ExploitBenchSeunghyun Lee, David Brumley
• GDM Dangerous Capabilities: Capture the FlagMary Phuong, Matthew Aitchison, Elliot Catt et al.
• InterCode: Security and Coding Capture-the-Flag ChallengesJohn Yang, Akshara Prabhakar, Karthik Narasimhan et al.
• PRISM Instruction OrderBlessings Mambwe
• CodeIPI: Indirect Prompt Injection for Coding AgentsDebu Sinha
• MonitorBench (Dual Objectives: Summarization subset)Han Wang, Yifan Sun, Brian Ko et al.
• PerspectiveGapYouran Sun, Xingyu Ren, Kejia Zhang et al.
• PRISMBlessings Mambwe
• PRISM GSM8K ExhaustiveBlessings Mambwe
• Terminal-BenchStanford and the Laude Institute
• WMDPCenter for AI Safety and the WMDP paper authors
• WMDP CyberNathaniel Li, Alexander Pan, Anjali Gopal et al.