Motion Control and Camera Movements

The Challenge of Controlled Motion

Left to itself, a video generation model picks its own motion from the prompt and model priors. The resulting motion is often generic — a slow zoom, gentle floating, or random head turns. When you need specific camera behavior (dolly in, pan left, crane up) or controlled subject motion (turn 90 degrees, raise left arm), you need explicit motion control beyond just prompt text.

Three methods are available in ComfyUI depending on which model you are using:

• Motion LoRAs for AnimateDiff: fine-tuned additions that specifically train the motion module for a particular camera movement type
• Wan 2.2 control video: supply a reference video clip as motion guidance for a new generation
• Prompt motion language: effective across all models when used with specific camera vocabulary

Motion LoRAs in AnimateDiff

AnimateDiff Evolved supports Motion LoRAs — small model files trained to produce specific camera movements. Available motion types typically include: zoom in, zoom out, pan left, pan right, tilt up, tilt down, and roll.

Download Motion LoRA files from Hugging Face (search "guozinan/MotionLoRA" or "AnimateDiff MotionLoRA"). Place the files in: ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/models/loras/

In your AnimateDiff workflow, add the Motion LoRA Loader node (from the AnimateDiff Evolved node collection). Connect it between the AnimateDiff Loader and the KSampler. Select the motion LoRA file (e.g., v2_lora_ZoomIn.ckpt) and set a strength value (0.8–1.0 for strong camera movement, 0.4–0.6 for subtle motion). Multiple motion LoRAs can be combined by chaining Motion LoRA Loader nodes.

Wan 2.2 Control Video

Wan 2.2's control video workflow takes an existing video clip as a motion reference and generates new content that follows the same motion pattern. This is one of the five built-in Wan 2.2 workflow templates.

The control video works using depth maps or optical flow extracted from the reference clip:

1. Load your reference video using VHS Load Video
2. Run the frames through a depth or optical flow preprocessor (from comfyui_controlnet_aux) to extract motion information
3. Feed the control video to the Wan 2.2 control video conditioning node alongside your text prompt
4. Generate new video that follows the reference motion with your text prompt defining the visual content

This lets you film a simple reference video with your phone (or use any existing footage) and transfer its motion pattern to a completely different AI-generated scene. The actor's walk becomes a robot's walk; the camera pan becomes a stylized panning shot.

Prompt Language for Camera Control

All video models respond to camera vocabulary in prompts, though the accuracy varies. Use these specific terms for the most consistent results:

• Camera movement: "slow dolly forward," "smooth pan left," "crane shot descending," "tracking shot following the subject," "aerial descent," "handheld slight shake"
• Zoom: "slow zoom in on the face," "wide establishing shot slowly zooming in," "zoom out to reveal the full scene"
• Static shots: "locked camera, static shot," "tripod mounted, no camera movement" — explicitly stating no movement sometimes prevents unwanted drift
• Subject motion: "the figure slowly turns to face the camera," "leaves blowing in the wind," "water rippling outward," "steam rising"

Place camera and motion language at the start of the prompt where it receives more weight. Models weight earlier tokens more heavily, so "slow dolly forward through the forest" at the start produces stronger camera movement than buried "with a slow dolly" at the end.