Segment Anything 2 (SAM 2)

Segment Anything Model 2 (SAM 2) is a universal segmentation model developed by Meta AI. Built on the major success of the original SAM model, SAM 2 offers both image and video segmentation in a single unified architecture. It can precisely isolate objects in any image or video by providing prompts such as points, boxes, or text. This unified approach represents a significant advancement in segmentation technology, bridging the gap between static images and dynamic video content and ushering in a new era in computer vision applications.

SAM 2's most revolutionary innovation is its video segmentation support. When you mark an object in a video frame, the model automatically tracks that object throughout the entire video and produces consistent segmentation masks in every frame. This capability is a game-changer for video editing, object tracking, and augmented reality applications that previously required manual frame-by-frame annotation. The model successfully handles challenging video scenarios such as object occlusion, size changes, rapid motion, and scene transitions with remarkable robustness. Its memory mechanism preserves information from previous frames to ensure consistent tracking even in long videos with complex motion patterns, and can re-identify objects even after they temporarily leave the field of view.

The model's promptable design makes it extremely flexible and intuitive to use. Segmentation can be initiated by clicking a point, drawing a box, or providing a text description. It can segment object types it has never seen in training data through zero-shot generalization, making it applicable to virtually any domain. It is successfully applied in specialized fields such as medical imaging, satellite photography, industrial quality control, and robotics. Additionally, it can track multiple objects simultaneously, enabling analysis of complex multi-object scenes and monitoring of interactions between different elements in dynamic environments.

Architecturally, SAM 2 extends the original SAM's image encoder with a memory module and temporal attention mechanism designed for video understanding. Its streaming architecture processes video frames sequentially while preserving information from previous frames through a memory bank that efficiently stores and retrieves relevant context. This design enables real-time video segmentation while maintaining reasonable memory usage even for extended video sequences. The hierarchical image encoder captures features at different scales, guaranteeing accurate segmentation of both small and large objects across varying resolutions. The attention mechanism models temporal dependencies to maintain consistency throughout the video duration.

Trained on the SA-V (Segment Anything Video) dataset, SAM 2 utilizes over 600,000 masklet annotations across more than 51,000 real-world videos spanning diverse scenarios. This comprehensive training data enables the model to generalize robustly across diverse video scenarios including sports, nature, urban environments, and indoor settings. The training process relies on a unified strategy across both image and video data, allowing the model to achieve high performance in both modalities without compromising quality in either domain. In image segmentation, it produces results comparable to or better than the original SAM, while in video segmentation it clearly surpasses previous specialized methods.

Released as open source under the Apache 2.0 license, SAM 2 is accessible via Hugging Face and GitHub for immediate use. It is being rapidly adapted to various applications by the research community worldwide. SAM 2-based solutions are being developed across a wide range of use cases including video editing software, autonomous driving systems, robotics applications, medical video analysis, and interactive media tools. The model is establishing the next-generation segmentation standard in computer vision, creating a transformative impact from industrial applications to creative workflows across the technology landscape.