A benchmark from
HarnessEval-W
Agentifying the Evaluation of Visual Worlds
Evaluation defines the taste of evolution.
July 2026 dataset snapshot
Coverage at a glance
Why HarnessEval-W
More than a scalar score
HarnessEval-W brings the harness paradigm from the LLM ecosystem to world-model benchmarking: every verdict is backed by an inspectable reasoning chain.
01
Agentified evaluation
Flip for detailsRather than applying a fixed rubric, the harness interprets the context of each case, decomposes the question into measurable subproblems, and spawns specialized sub-agents with tailored context and diagnostic tools.
02
Transparent evidence trees
Flip for detailsEvery evaluation becomes an evidence tree that records exactly what was tested, which tool supplied the visual grounding, and the complete logical chain that justifies the final score — auditable end to end.
03
Human-aligned and robust
Flip for detailsJudgments closely track human A/B preferences on the hardest settings — intentional and physical transitions — and stay consistent under repeated evaluation runs.
04
A living benchmark
Flip for detailsOpen-sourced as an executable agentic system: the skill library is extensible, and new skills and evaluation cases grow as world models evolve.
ρ = 0.93
Spearman correlation with human Bradley–Terry ranking on Intentional Transition
71.7%
Pairwise accuracy vs. human choices on Physical — up from 31.9% for the closest WBench protocol
4.9×
Narrower repeat-evaluation envelope than WBench across three independent runs
HarnessEval-W Leaderboard
Snapshot
2026-08-18 V1
Leaderboard release
2026-08-18 V1
2026-08-18 V1
Coming Soon
No matching rows
Change the search term, interface filter, or metric group filter.
Evaluation Gallery
BibTeX
@article{mirros2026harnessevalw,
title = {HarnessEval-W: Agentifying the Evaluation of Visual Worlds},
author = {{MirroS Team}},
journal = {arXiv preprint arXiv:2608.16859},
year = {2026}
}