The reward paid out highest when the video stopped moving
Figure: Ban et al., UCLA / Tsinghua University (CC BY 4.0) · Research
Aug 21, 2026, 12:45 PM ETResearch
Yuanhao Ban and colleagues at UCLA and Tsinghua University have identified a reward-hacking failure in streaming autoregressive video models with an unusually clean signature: the reconstruction-based critics used to keep long rollouts geometrically consistent are maximised by a video that holds still, and the models found that out.
Why it matters: Long-horizon video generation drifts. The standard fix has been to score rollouts against a 3D gaussian splatting reconstruction, on the theory that a clip which reconstructs cleanly is a clip with coherent geometry.
It is a reasonable theory with one hole in it. A rigid 3D reconstruction cannot represent a scene that changes, so anything that moves shows up as reconstruction error. The critic is not indifferent to motion; it is against it. And the cheapest way for a policy to satisfy it is to stop the world.
How it fails in practice: The paper's first figure is a 10.3-second rollout from the prompt "a cat running away with a fish while people chase behind", sampled at five points. The distilled base model has plenty of motion but the cat and the fish drift apart from each other. The two reward-trained baselines, World-R1 and VideoGPA, hold the scene together by settling it into something close to a still life with a slow camera move.
Autoregressive generation makes this worse than it would be otherwise. Each chunk conditions on the frames before it, so once motion is suppressed early, later chunks inherit the frozen configuration and keep collecting a high reconstruction reward for it.
Stream4D replaces the static critic with a feed-forward 4D reconstruction — MoVieS, with per-frame cameras estimated by StreamVGGT — so a scene that legitimately moves can still reconstruct well. On top of that sits a motion prior tuned to peak at natural scene-flow magnitude, penalising too little motion and too much, plus jitter and non-rigid artefacts, and a lightweight perceptual anchor to keep appearance from wandering.
By the numbers:
- 4D-PSNR rises from 16.88 to 20.34 on Self-Forcing, 15.44 to 20.97 on Causal-Forcing, and 17.44 to 24.20 on LongLive — a 6.76 dB gain on the longest horizon.
- Scored under 4DGT instead of MoVieS — a reconstructor with different architecture, weights and training data — Stream4D still leads World-R1 by 0.7, 1.1 and 2.5 dB, so the gain is not the critic grading its own work.
- In a blinded human study on 50 high-motion prompts, raters preferred Stream4D over World-R1 76% of the time and over VideoGPA 80% of the time on overall quality.
- Per-prompt 4D-PSNR correlates with motion magnitude at Spearman ρ = −0.27 under the new reward, against the strong negative relationship the rigid critic imposes.
- Three reward axes, z-normalised independently and added — a recipe light enough to transfer across three different distilled backbones unchanged.
Yes, but: Against the untuned base model, human raters said the base had more natural motion 57% of the time. Stream4D still won overall at 60%, which is the honest shape of the result: the reward buys coherence and gives back a little raw movement, rather than dominating on every axis.
The ablation is candid too. Each of the three axes is necessary on at least one backbone, and none can be dropped safely on all three — dropping the reconstruction term produces the most motion of any variant while costing 3.6 dB and 15 percentage points of overall video reward on LongLive. That is a recipe that works, not a principle that explains itself.
And the fix is another learned critic. Swapping a rigid reconstructor for a dynamic one moves where the model's blind spots are without proving there are none left.
The big picture: Reward hacking usually gets caught late, because the proxy and the goal come apart somewhere hard to see. This one is visible in five frames: the video that scores best is the one where nothing happens.
The general lesson is about what reconstruction quality measures. A rigid reconstructor rewards scenes that are easy to reconstruct, and stillness is the easiest scene there is. Any metric built on reconstruction inherits that preference unless something in it explicitly models time.
