This paper presents VAMPO, a post-training framework that improves visual dynamics in video action models for robot control. The key contribution is formulating multi-step denoising as a sequential decision process and optimizing the denoising policy with rewards defined over expert visual dynamics in latent space. The approach addresses the objective mismatch in current diffusion-based video predictors by explicitly optimizing precision-critical visual dynamics needed for manipulation tasks.