Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising
VPT — a fine-tuning framework for physically consistent video generation
Guangting Zheng1*, Haojing Chen2*, Hao Li3, Jingtao Zhang4,
Zhen Yang1, Xiaosong Jia3, Xue Yang5, Shaofeng Zhang1, Yanyong Zhang1
1University of Science and Technology of China
2University of Electronic Science and Technology of China
3Fudan University
4Georgia Institute of Technology
5Shanghai Jiao Tong University
*Equal contribution.
Abstract
While modern video diffusion models excel in visual fidelity, maintaining long-range physical consistency remains a formidable challenge. Conventional pixel-reconstruction objectives mainly focus on appearance details and often fail to capture the underlying dynamics of a scene. To mitigate this, recent efforts have integrated auxiliary modalities (e.g., optical flow) to introduce physics priors via joint training with video appearance. However, these methods have three main limitations: (1) they do not distinguish the different motion patterns of different entity types; (2) joint modeling of visual and auxiliary modalities can cause capacity conflicts and weaken the pretrained visual prior; and (3) auxiliary modalities may accumulate errors during inference. To address these issues, we propose VPT, a fine-tuning framework for improving physical consistency in video diffusion models. VPT introduces a role-aware signal that groups entities into agents, controlled objects, passive objects, and background, so that different physical roles can be modeled more clearly. We further propose a modality-decoupled denoising strategy, where the visual and auxiliary channels are assigned independent noise levels. Together with a loss-weight decay strategy, this design makes auxiliary modalities serve as soft constraints rather than strong dependencies, mitigating recursive prediction errors during inference. We also introduce cross-step auto-guidance to further strengthen physical dynamics. Experiments show that VPT improves physical consistency while preserving visual quality, achieving relative gains of 39.4% in VideoPhy SA and 17.9% in VideoPhy PC over Wan2.1-T2V-1.3B, and consistent improvements on VideoPhy-2.
Auxiliary Modality Reconstruction
Original vs. reconstruction for the video, role, and flow modalities.
Video (original vs. reconstruction)
Role (original vs. reconstruction)
Flow (original vs. reconstruction)
Qualitative Results
Wan2.1-1.3B Comparisons
Each row compares the base model, the VideoJAM baseline, and our VPT method on the same prompt.
Prompt: A wine bottle pours a red blend into a glass.
Prompt: An apple falls into a vat of red wine.
Prompt: A coffee stirrer swirling iced coffee in a glass.
Prompt: A drum vibrating from the beating stick.
Prompt: A leaf falls delicately into a slow-moving river.
Prompt: A metal spoon stirs soda in a glass.
Prompt: A paintbrush dipping into a pot of paint.
Prompt: A pebble drops into a clear puddle creating ripples.
Wan2.1-14B Comparisons
Each row compares the base model with our VPT method on the same prompt.
Prompt: A pebble drops into a clear puddle creating ripples.
Prompt: A plastic spoon spinning in a cup of coffee.
Prompt: A sharp razor shaves the facial hair close.
Prompt: A spoon scoops creamy soup from a pot.
Prompt: A teaspoon stirs sugar into a cup of coffee.
Prompt: Hook clinging onto a fish.
Prompt: Toothpaste squirting from a tube onto a brush.
Prompt: A bat swinging and hitting a baseball.
Quantitative Results
VideoPhy & VideoPhy-2
Overall Semantic Adherence (SA) and Physical Commonsense (PC). VPT is applied on both the 1.3B and 14B Wan2.1-T2V backbones.
| Model | VideoPhy SA | VideoPhy PC | VideoPhy-2 SA | VideoPhy-2 PC |
| Wan2.1-T2V-1.3B | 47.7 | 21.2 | 19.3 | 53.7 |
| + Full Fine-tune | 45.1 | 20.9 | 18.9 | 53.6 |
| + VideoJAM | 49.1 | 22.1 | 20.6 | 54.0 |
| + VPT (Ours) | 66.5 | 25.0 | 22.5 | 55.1 |
| Wan2.1-T2V-14B | 56.1 | 23.2 | 21.9 | 52.9 |
| + Full Fine-tune | 62.1 | 21.5 | 20.7 | 54.0 |
| + VPT (Ours) | 67.7 | 30.0 | 23.3 | 59.9 |
VBench
Wan2.1-T2V-1.3B backbone, official raw prompts (no prompt enhancement), 81 frames @ 480×832, 16 FPS.
| Metric | Wan2.1-1.3B | + Full FT | + VideoJAM | + VPT (Ours) |
| Total Score | 76.93 | 78.71 | 78.76 | 79.58 |
| Quality Score | 79.81 | 81.26 | 81.18 | 83.25 |
| Semantic Score | 65.43 | 68.47 | 69.08 | 64.86 |
VPT achieves the best total and quality scores. See the paper for the full per-dimension breakdown and ablations.
BibTeX
@article{zheng2026vpt,
title = {Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising},
author = {Zheng, Guangting and Chen, Haojing and Li, Hao and Zhang, Jingtao and
Yang, Zhen and Jia, Xiaosong and Yang, Xue and Zhang, Shaofeng and Zhang, Yanyong},
journal = {arXiv preprint},
year = {2026}
}