ProPainter

ProPainter is an advanced deep learning model developed for video inpainting and object removal, representing a significant advancement in temporal content generation. Created by researchers at Nanjing University and S-Lab, ProPainter establishes a new performance standard in video inpainting through enhanced propagation mechanisms and efficient Transformer architectures. Introduced in 2023, the model addresses the temporal consistency and processing efficiency limitations that plagued previous approaches to video content filling and restoration, bringing the field substantially closer to production-ready quality.

The model's technical architecture consists of two core components that work in concert: image-based feature propagation and a dual-domain Transformer module. The feature propagation mechanism efficiently transfers information from neighboring frames to inpainting regions, ensuring temporal consistency across the video sequence and preventing flickering or inconsistency in frame transitions. The dual-domain Transformer applies attention mechanisms in both spatial and temporal dimensions simultaneously, generating high-quality content with a wide receptive field that captures both local details and global context. Optical flow estimation, flow completion, and image inpainting steps operate within an integrated pipeline forming an end-to-end learnable system. This integrated approach produces more consistent and higher-quality results compared to previous modular methods that process each step independently.

ProPainter operates in two primary usage modes, each addressing different video editing needs. In video object removal mode, a specific object in the video (such as a walking person, watermark, or unwanted element) is masked and removed, with the background filled consistently and temporally coherently across all affected frames. In video completion mode, damaged or missing video regions are completed while maintaining temporal coherence throughout the entire sequence, ensuring seamless visual continuity. In both modes, the model performs motion-aware inpainting, producing results that are consistent across the video flow rather than processing each frame independently.

Application domains span from professional video production to personal use cases across multiple industries. In film and TV post-production, removal of unwanted equipment, microphones, reflections, and crew members represents a primary professional use case that saves significant manual rotoscoping time. Social media content creators utilize it for watermark and logo cleanup as an essential workflow step. Security camera footage processing employs it for privacy-focused person masking and anonymization applications. Historical video archive restoration and damaged film frame repair serve as valuable cultural heritage preservation applications. Advertising production teams efficiently remove unwanted elements from existing footage and perform background editing operations using ProPainter's temporal-aware capabilities.

In terms of performance metrics, ProPainter significantly outperforms previous methods on standard video inpainting benchmarks including DAVIS and YouTube-VOS, establishing a new state-of-the-art standard in the field. It achieves particularly strong results on temporal consistency metrics while maintaining competitive performance on visual quality measurements across diverse video content. Regarding processing efficiency, the efficient Transformer implementation enables notably faster processing compared to previous Transformer-based methods, reducing computational overhead without sacrificing output quality. Processing times for an 80-frame video at 448x240 resolution range from seconds to minutes depending on GPU capabilities and model configuration.

The model is available as open source on GitHub with a PyTorch implementation and comprehensive documentation. Pre-trained model weights and example usage code are provided for immediate experimentation and integration. An NVIDIA GPU is required for practical use, though basic operation is possible with 8GB VRAM. Recognized as a research reference in the video inpainting domain, ProPainter serves as a foundation for next-generation video editing and restoration tools. Its approach of combining propagation mechanisms with Transformer architectures for motion-based video processing continues to inspire further research, contributing to the advancement of video generative AI technologies.