evaluation-package · Source-linked discovery

WildClawBench

This is an Inspect AI wrapper of the original WildClawBench implementation. It does not contain its own dataset or scoring logic. The original implementation can be found at https://github.com/internlm/WildClawBench. Users provide a pinned checkout of the original repository, tested here at commit 86d71447413d38f38740a021cb776f64eb396ee0, and the wrapper reports native WildClawBench results through Inspect's scoring interface. WildClawBench contains 60 human-authored bilingual and multimodal long-horizon agent tasks across productivity flow, code intelligence, social interaction, search and retrieval, creative synthesis, and safety alignment. The native harness runs OpenClaw-style agents in Docker and grades with deterministic rule checks, environment-state auditing, and LLM/VLM judges; this wrapper parses native summary*.json or per-task score.json outputs and reports the mean native overall_score across scored tasks with Inspect mean/stderr metrics. It requires Docker, an OpenAI-compatible model endpoint, the pinned Docker base image node:22-bookworm@sha256:c601a46abb4d2ab80a9dc3da208d50d1122642d53f17a101926ace71e5a9bf1c, openclaw@2026.6.10, and the exact Python dependencies pinned in the upstream pyproject.toml and Dockerfile.