One of the most common frustrations when generating images with Stable Diffusion is not being able to control the character's pose. No matter how detailed your text prompt is, the pose the model produces is often unpredictable. This is exactly where ControlNet OpenPose comes in: it extracts a human skeleton from a reference image and feeds that skeleton to generation as a control condition, so the character takes exactly the pose you want. In this guide you will learn how pose control works and how to apply it step by step in ComfyUI and Automatic1111.
How ControlNet and Pose Control Work
ControlNet, developed by Lvmin Zhang and colleagues (ICCV 2023 Marr Prize), is an architecture that adds spatial control to a pretrained diffusion model. The OpenPose ControlNet model is conditioned on a pose skeleton (a stick figure of keypoints). The flow is: a pose preprocessor (OpenPose or DWPose) extracts a skeleton from a reference image, that skeleton is fed to ControlNet, and Stable Diffusion generates an image that follows the pose together with the text prompt. This way, clothing, background, and style can change via the prompt while the pose stays fixed.
Step 1: Choose the Preprocessor (OpenPose vs DWPose)
To extract the skeleton you choose a preprocessor. The options include classic openpose (body), openpose_full (body + face + hands), and dw_openpose_full. DWPose is a modern RTMPose-based estimator and is noticeably more accurate than OpenPose, especially on hands; where standard OpenPose misses a hand, DWPose captures both. For the highest accuracy, dw_openpose_full is recommended; for a lighter option, classic openpose can be used.
Step 2: Extract the Skeleton from a Reference Image
Upload a reference image containing the pose you want. In Automatic1111, enable ControlNet, add the image to the single-image area, select OpenPose as the Control Type, and run the preprocessor to preview the detected skeleton. In ComfyUI, use the DWPose Estimator (DWPreprocessor) or OpenPose node, connect the reference image, and set detect_hand/detect_face/detect_body; the node outputs both a skeleton image and POSE_KEYPOINT JSON.
Step 3: Configure the ControlNet Model and Settings
Connect the extracted skeleton to the OpenPose ControlNet model (for example control_v11p_sd15_openpose). Key settings: Control Weight (typical range 0-2, default around 1.0) determines how strictly the pose is applied; Guidance Start/End set the percentage of total steps during which ControlNet is active. In ComfyUI, the Apply ControlNet node's strength (recommended 0.5-1.5) and start_percent/end_percent serve the same function. With Pixel Perfect enabled, resolution is computed automatically.
Step 4: Write the Prompt and Generate
Write your positive and negative prompts, choose your base model (control_v11p_sd15_openpose for SD 1.5, the relevant SDXL OpenPose ControlNet for SDXL), and generate. The result follows both your text description and the skeleton pose you specified. By using the same skeleton with different prompts, you can produce different characters, outfits, and scenes in the same pose.
Step 5: Improve the Results
If the generated image does not follow the skeleton closely, increase the control weight or try the 'ControlNet is more important' mode; however, very high values can produce stiff, mannequin-like results. If hands are broken, switch to the dw_openpose_full preprocessor. For a consistent character identity, combine ControlNet OpenPose with a LoRA or IPAdapter — OpenPose controls the pose while IPAdapter/LoRA fixes the identity. For multiple characters, you can add several figures with a skeleton editor.
Tips and Common Pitfalls
Front/back ambiguity: because the 2D skeleton carries no orientation/depth information, foot direction and front-vs-back posture can sometimes come out unexpectedly; a clear reference image reduces this. Hardware: around 4-8 GB VRAM is enough for SD 1.5 + ControlNet, while SDXL + ControlNet at 1024x1024 is better with 12-16 GB. The SD 1.5 OpenPose ControlNet has a native resolution of 512x512.
Frequently Asked Questions
Is ControlNet OpenPose free? Yes — the ControlNet code is Apache-2.0 and the OpenPose ControlNet models are CreativeML OpenRAIL-M licensed and open source. Should I use DWPose or OpenPose? For the highest accuracy, especially hands, DWPose (dw_openpose_full) is recommended. How do I keep the same character in different poses? Combine OpenPose with an IPAdapter or character LoRA; separate pose control from identity control.