Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising

VPT — a fine-tuning framework for physically consistent video generation
Guangting Zheng1*, Haojing Chen2*, Hao Li3, Jingtao Zhang4, Zhen Yang1, Xiaosong Jia3, Xue Yang5, Shaofeng Zhang1, Yanyong Zhang1
1University of Science and Technology of China   2University of Electronic Science and Technology of China   3Fudan University   4Georgia Institute of Technology   5Shanghai Jiao Tong University
*Equal contribution.

Abstract

While modern video diffusion models excel in visual fidelity, maintaining long-range physical consistency remains a formidable challenge. Conventional pixel-reconstruction objectives mainly focus on appearance details and often fail to capture the underlying dynamics of a scene. To mitigate this, recent efforts have integrated auxiliary modalities (e.g., optical flow) to introduce physics priors via joint training with video appearance. However, these methods have three main limitations: (1) they do not distinguish the different motion patterns of different entity types; (2) joint modeling of visual and auxiliary modalities can cause capacity conflicts and weaken the pretrained visual prior; and (3) auxiliary modalities may accumulate errors during inference. To address these issues, we propose VPT, a fine-tuning framework for improving physical consistency in video diffusion models. VPT introduces a role-aware signal that groups entities into agents, controlled objects, passive objects, and background, so that different physical roles can be modeled more clearly. We further propose a modality-decoupled denoising strategy, where the visual and auxiliary channels are assigned independent noise levels. Together with a loss-weight decay strategy, this design makes auxiliary modalities serve as soft constraints rather than strong dependencies, mitigating recursive prediction errors during inference. We also introduce cross-step auto-guidance to further strengthen physical dynamics. Experiments show that VPT improves physical consistency while preserving visual quality, achieving relative gains of 39.4% in VideoPhy SA and 17.9% in VideoPhy PC over Wan2.1-T2V-1.3B, and consistent improvements on VideoPhy-2.

Method

Method Architecture

Auxiliary Modality Reconstruction

Original vs. reconstruction for the video, role, and flow modalities.

Video (original vs. reconstruction)
Role (original vs. reconstruction)
Flow (original vs. reconstruction)

Qualitative Results

Wan2.1-1.3B Comparisons

Each row compares the base model, the VideoJAM baseline, and our VPT method on the same prompt.

Prompt: A wine bottle pours a red blend into a glass.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)
Prompt: An apple falls into a vat of red wine.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)
Prompt: A coffee stirrer swirling iced coffee in a glass.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)
Prompt: A drum vibrating from the beating stick.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)
Prompt: A leaf falls delicately into a slow-moving river.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)
Prompt: A metal spoon stirs soda in a glass.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)
Prompt: A paintbrush dipping into a pot of paint.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)
Prompt: A pebble drops into a clear puddle creating ripples.
Wan2.1-1.3B
Wan2.1-1.3B + VideoJAM
Wan2.1-1.3B + VPT (Ours)

Wan2.1-14B Comparisons

Each row compares the base model with our VPT method on the same prompt.

Prompt: A pebble drops into a clear puddle creating ripples.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)
Prompt: A plastic spoon spinning in a cup of coffee.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)
Prompt: A sharp razor shaves the facial hair close.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)
Prompt: A spoon scoops creamy soup from a pot.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)
Prompt: A teaspoon stirs sugar into a cup of coffee.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)
Prompt: Hook clinging onto a fish.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)
Prompt: Toothpaste squirting from a tube onto a brush.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)
Prompt: A bat swinging and hitting a baseball.
Wan2.1-14B
Wan2.1-14B + VPT (Ours)

Quantitative Results

VideoPhy & VideoPhy-2

Overall Semantic Adherence (SA) and Physical Commonsense (PC). VPT is applied on both the 1.3B and 14B Wan2.1-T2V backbones.

ModelVideoPhy SAVideoPhy PCVideoPhy-2 SAVideoPhy-2 PC
Wan2.1-T2V-1.3B47.721.219.353.7
  + Full Fine-tune45.120.918.953.6
  + VideoJAM49.122.120.654.0
  + VPT (Ours)66.525.022.555.1
Wan2.1-T2V-14B56.123.221.952.9
  + Full Fine-tune62.121.520.754.0
  + VPT (Ours)67.730.023.359.9

VBench

Wan2.1-T2V-1.3B backbone, official raw prompts (no prompt enhancement), 81 frames @ 480×832, 16 FPS.

MetricWan2.1-1.3B+ Full FT+ VideoJAM+ VPT (Ours)
Total Score76.9378.7178.7679.58
Quality Score79.8181.2681.1883.25
Semantic Score65.4368.4769.0864.86

VPT achieves the best total and quality scores. See the paper for the full per-dimension breakdown and ablations.

BibTeX

@article{zheng2026vpt,
  title   = {Enhancing Video Physical Consistency via Role-aware Joint Training and Modality-decoupled Denoising},
  author  = {Zheng, Guangting and Chen, Haojing and Li, Hao and Zhang, Jingtao and
             Yang, Zhen and Jia, Xiaosong and Yang, Xue and Zhang, Shaofeng and Zhang, Yanyong},
  journal = {arXiv preprint},
  year    = {2026}
}