World action models · Test-time planning

Beyond Visual Quality

A Study of Test-Time Planning
with World Action Models

Jianhao Yuan¹Yu Yuan²Benjamin Ramtoula¹Lukas Vierling¹
Paul Newman¹Lars Kunze³Philip Torr¹Daniele De Martini¹

¹ University of Oxford² Purdue University³ UWE Bristol

The question

Can imagined futures tell us which action to take?

World action models generate actions together with visual predictions of their consequences. Sampling several pairs creates a choice. We study when these predictions provide useful evidence for making that choice.

Producing better action choices and recognising them in generated futures are two different problems.

Success versus number of sampled candidates. Selection using realised outcomes outperforms learned value, visual quality and VLM progress in the controlled same-state experiment. — Selection using realised outcomes reveals room for improvement that the tested scores do not fully recover. The oracle is retrospective, not a deployable selector.

A controlled study

Same model. Different ways to choose.

We hold the pretrained world action model fixed, sample paired actions and predicted futures, then change the score used to select a candidate. The robot executes the corresponding actions and observes the environment again.

Current observation enters a frozen world action model. It samples paired predicted futures and actions. A score selects a future, its actions execute in the environment, and the new observation closes the loop. — Best-of-K selection is the experimental tool, not a new planning method. Each future stays paired with the actions generated alongside it. View full-size diagram ↗

What we learn

Look beyond appearance.

Useful alternatives exist

In the initial same-state study, an outcome-informed oracle increases success from 68.9% to 79.2% compared with uniform selection. This measures opportunity within the sampled candidates.

Ranking remains difficult

Some selectors yield higher observed success, but gains are uneven. Visual quality, physical consistency and task progress scores do not consistently identify the better action.

Consequences matter

Numerically different actions need not produce different task outcomes. We analyse candidate outcomes and score rankings to distinguish limited choice from failures to recognise useful alternatives.

The primary study uses Cosmos Policy in RoboCasa. The initial same-state comparison covers 48 decision states across four tasks with 16 candidates per state; these percentages are not full-benchmark success rates. Further experiments test complete action execution and additional model interfaces.

Minimal research code

Sample. Score. Select.

A small, readable starting point for future-based action selection. Try the CPU smoke test, then connect Cosmos Policy using the included adapter.

SSIM and pretrained learned-value scores
Paired action–future candidates with deterministic tie handling
A Cosmos Policy example and CPU tests

Read the quick start

Python 3.10+ · CPU demo

git clone https://github.com/\
YuanJianhao508/Beyond-Visual-Quality.git
cd Beyond-Visual-Quality
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python examples/toy_demo.py
pytest

The synthetic demo tests the interface. It is not a robot experiment or a reproduction of the paper’s results.

Reference

Citation

The preprint has been submitted to arXiv. The identifier will be added when it is publicly available.

@misc{yuan2026beyondvisualquality,
  title  = {Beyond Visual Quality: A Study of Test-Time
            Planning with World Action Models},
  author = {Yuan, Jianhao and Yuan, Yu and Ramtoula, Benjamin
            and Vierling, Lukas and Newman, Paul and Kunze, Lars
            and Torr, Philip and De Martini, Daniele},
  year   = {2026},
  note   = {Preprint},
  url    = {https://yuanjianhao508.github.io/Beyond-Visual-Quality/}
}