Driving World Models Move the Camera, Not the Scene

Yuqiao Feng^* Xiaoyu Ao^*

Chongqing University

^*Equal contribution

Paper (arXiv) Videos Code (coming soon)

Teaser: released world models move the camera but not the scene; ReTime-Compose follows the real outcome. — **Released driving world models move the camera, not the scene, and the trajectory score cannot tell** (paper Fig. 1). A: the stereo right camera records the real outcome of a plan 0.54 m to the right; released Epona moves the scene content the wrong way, while ReTime-Compose on the same frozen Epona follows. B: at half speed the real scene recedes; released Orbis 2's barely changes, ReTime-Compose's recedes. C: scale-aligned ADE favours a video that ignores a half-speed command; the scene response α does not.

TL;DR. Recorded drives already contain the real outcomes of plans that were never driven. Measured against them, five released plan-conditioned world models move their camera but barely move the scene: under speed plans they reproduce at most .23 of the real change in distance, where real video reaches .64–.73, and under a 0.54 m lateral shift at most .23 of the real image motion, where real video reaches .99 (paper Table 2). ReTime-Compose delivers the plan to a frozen generator as geometry, without training: lateral response .90–1.04 (Table 4), and a higher speed response than re-sampling on every set (Table 5).

1 · What metrics miss

The camera follows, the scene does not

Released models reproduce at most about a third of the real change in distance under speed plans, and scale-aligned ADE cannot tell an executed speed command from an ignored one.

2 · Where the truth comes from

Real counterfactual references

Retiming a recorded drive gives pose-matched images and LiDAR for other speeds; a stereo rig records the view of a camera shifted 0.54 m to the side.

3 · What ReTime-Compose fixes

The plan enters as geometry

Transport one logged-speed rollout to the planned cameras and let the frozen generator complete only the unseen pixels. No training.

Videos

All numbers in the paper are computed over every context of each evaluation set. The videos illustrate them; where a context was selected for visibility, the caption says so.

Key results

Released models move the camera but not the scene (paper Table 2). Scene response α: the share of the real change in distance that the forecast reproduces (1 = changes as the world does, 0 = ignores the plan). Lateral: slope of predicted image motion on real stereo disparity for a 0.54 m right shift. KITTI, 59–256 contexts per model.

Scene response under speed plans (paper Table 2). Every released model stays far below real video at both speeds. Real video is below 1 because the depth reader compresses depth; it is the attainable reference.

Released model	α at 0.5×	α at 1.5×	lateral slope
DrivingWorld	.17	.08	−.30 / −.15
Vista	−.03	−.05	.03 / .02
Epona	.18	.16	−.03 / −.02
Orbis 2	.23	.11	.23
GEM	.20	.10	−.02
real video	.64–.73	.66–.71	.99

Real video reads about .72 rather than 1 under speed plans because the depth reader compresses depth; it is the attainable reference. Lateral pairs are two seeds where available.

ReTime-Compose on the same frozen generators (paper Tables 4 and 5). Lateral: slope against real stereo disparity (KITTI) or LiDAR parallax (nuScenes, Argoverse 2), real = 1.00. Speed: gain in α over re-sampling, the training-free baseline that replays the logged-speed rollout at the planned speed.

Generator	lateral 0.54 m	speed gain at 0.5×	speed gain at 1.5×
Epona (7 KITTI sets)	.93–1.04	+.17 (+.14 on test set 5)	+.16 (+.11 on test set 5)
Epona (nuScenes, Argoverse 2)	.94–.96	–	–
GEM	.96 / 1.01	+.24	–
Orbis 2	.90	+.16	–

Epona speed gains are pooled over the validation set and test sets 1–4; test set 5 (80 contexts from 25 further drives) was read once after the method was frozen. GEM's rollout only covers slow-down plans. Full tables, intervals and controls are in the paper.

Method: ReTime-Compose

ReTime-Compose pipeline: roll out once, re-time by geometry, complete only the unseen. — **The plan acts through transport, and the frozen generator completes only what transport cannot supply** (paper Fig. 3). (a) One frozen rollout at the logged speed. (b) Read depth and poses transport its frames from the rollout cameras to the planned ones. (c) The frozen model completes the pixels no frame covers; covered pixels are pasted back.

Measuring the scene, not only the camera

How the scene response alpha is read from LiDAR points. — **Scene response α** (paper Fig. 4). The least-squares slope of the forecast change in distance against the real change (half against logged speed, KITTI). A forecast that changes as the world does lies on the dashed line.

Response versus noise trade-off. — **Completion takes ReTime-Compose off the response–noise trade-off** (paper Fig. 7). Re-sampling lowers the noise, the composite raises the response, and completion brings the noise below re-sampling's (validation set).

Qualitative example

Slow-down plan: real video, released model, re-sampling and ReTime-Compose. — **Under a slow-down plan, ReTime-Compose shows static content at the planned pose** (paper Fig. 6). Real pose-matched video, the released model, re-sampling and ReTime-Compose at steps 16 and 24 of the 0.5× plan. Dashed line: the bottom of the real lead car.

Abstract

Plan-conditioned driving world models are used to compare candidate plans, so their forecast should change with the plan as the real scene would, yet widely used control metrics check only whether the camera follows the command. Recorded drives already contain real outcomes of plans that were never driven: retiming a log gives pose-matched images and LiDAR for other speeds, and a stereo rig records a laterally shifted view. Against these references, released world models move their camera but barely move the scene: their scene response, the share of the real change in distance that the forecast reproduces, reaches at most about a third of real video's, and the trajectory score cannot tell an executed speed command from an ignored one. Interventions inside the generator trace the failure to the plan's channel. The forecast geometry follows the context frames; once they carry the planned geometry, plan tokens add little. ReTime-Compose therefore delivers the plan as geometry, without training. It transports the frames of one logged-speed rollout to the planned cameras, and the frozen model completes only what no frame covers. Frozen generators then follow a 0.54 m lateral shift nearly as the real scene does on KITTI, nuScenes and Argoverse 2, and respond more than re-sampling to slow-down plans in three generators and to speed-up plans in two.

Code

Evaluation code for the scene response and the ReTime-Compose implementation will be released in this repository. Datasets (KITTI, nuScenes, Argoverse 2) and released world-model checkpoints must be obtained from their original sources under their own licenses.

BibTeX

@article{retimecompose,
  title   = {Driving World Models Move the Camera, Not the Scene},
  author  = {Feng, Yuqiao and Ao, Xiaoyu},
  journal = {arXiv preprint arXiv:2611.15260},
  year    = {2026}
}