huggingface/transformers

Vision models

Image (and increasingly video) models that ingest pixels and produce labels, masks, boxes, depth maps, keypoints, or features.

Image classification & general backbones

Directory	Notes
`vit/`, `vit_mae/`, `vit_msn/`, `vitdet/`, `vitmatte/`, `vitpose/`, `vitpose_backbone/`	Vision Transformer family
`deit/`, `swin/`, `swinv2/`, `swin2sr/`, `beit/`, `cvt/`, `levit/`, `pvt/`, `pvt_v2/`, `poolformer/`, `convnext/`, `convnextv2/`, `efficientnet/`, `regnet/`, `resnet/`, `bit/`, `mobilenet_v1/`, `mobilenet_v2/`, `mobilevit/`, `mobilevitv2/`, `mvit-style hiera`, `swiftformer/`, `focalnet/`, `dinat/`, `groupvit/`, `nat-style models`, `convnext-style nets`	Conv and hybrid backbones
`dinov2/`, `dinov2_with_registers/`, `dinov3_convnext/`, `dinov3_vit/`, `ijepa/`, `mlcd/`, `metaclip_2/`, `vjepa2/`, `aimv2/`, `siglip/`, `siglip2/`	Self-supervised / multimodal-pretrained backbones
`imagegpt/`	Pixel-level autoregressive
`timm_backbone/`, `timm_wrapper/`, `hgnet_v2/`, `pp_lcnet/`, `pp_lcnet_v3/`, `textnet/`, `pp_doclayout_v2/`, `pp_doclayout_v3/`	Bridges to `timm` and PaddlePaddle backbones

Object detection

Directory	Notes
`detr/`, `conditional_detr/`, `deformable_detr/`, `dab_detr/`, `lw_detr/`, `rt_detr/`, `rt_detr_v2/`, `d_fine/`, `deimv2/`, `mm_grounding_dino/`, `grounding_dino/`, `omdet_turbo/`, `owlvit/`, `owlv2/`, `yolos/`, `table_transformer/`	DETR-style and zero-shot detectors

Segmentation

Directory	Notes
`mask2former/`, `maskformer/`, `oneformer/`, `eomt/`, `eomt_dinov3/`, `segformer/`, `seggpt/`, `upernet/`, `slanet/`, `slanext/`, `clipseg/`	Semantic / panoptic / universal segmentation
`sam/`, `sam2/`, `sam2_video/`, `sam3/`, `sam3_lite_text/`, `sam3_tracker/`, `sam3_tracker_video/`, `sam3_video/`, `sam_hq/`, `edgetam/`, `edgetam_video/`	Segment Anything family + video extensions

Depth and 3D

Directory	Notes
`dpt/`, `glpn/`, `zoedepth/`, `depth_anything/`, `depth_pro/`, `prompt_depth_anything/`, `vitmatte/`, `uvdoc/`	Monocular depth & matting

Keypoints & matching

Directory	Notes
`superpoint/`, `superglue/`, `lightglue/`, `efficientloftr/`, `keypoint_matching/`	Keypoint detection and feature matching

Document AI

Directory	Notes
`donut/`, `nougat/`, `udop/`, `pix2struct/`, `markuplm/`, `layoutlm/`, `layoutlmv2/`, `layoutlmv3/`, `layoutxlm/`, `mgp_str/`, `lighton_ocr/`, `pp_chart2table/`, `pp_formulanet/`, `pp_ocrv5_mobile_det/`, `pp_ocrv5_mobile_rec/`, `pp_ocrv5_server_det/`, `pp_ocrv5_server_rec/`, `qianfan_ocr/`, `glm_ocr/`, `got_ocr2/`, `paddleocr_vl/`	OCR, layout, structured docs

Video classification & understanding

Directory	Notes
`videomae/`, `vivit/`, `timesformer/`, `videomt/`, `vjepa2/`, `x_clip/`, `tvp/`, `pe_video/`, `pe_audio_video/`, `video_llava/`, `video_llama_3/`	Video classification + video VLMs

Where the per-model preprocessing lives

Image models declare an image processor in src/transformers/models/<arch>/image_processing_<arch>.py. The library ships both slow (PIL/NumPy) and fast (torchvision.transforms.v2) variants for many of them. Video models add video_processing_<arch>.py.

Backbone abstraction

Many detection / segmentation / depth models compose a frozen backbone. The contract is BackboneMixin (src/transformers/backbone_utils.py) which exposes out_features, out_indices, and forward_with_filtered_kwargs. Bridges to external backbones live in timm_backbone/ and timm_wrapper/.

Pipelines that consume vision models

Pipeline task	Module
`image-classification`	`src/transformers/pipelines/image_classification.py`
`image-segmentation`	`src/transformers/pipelines/image_segmentation.py`
`image-feature-extraction`	`src/transformers/pipelines/image_feature_extraction.py`
`object-detection`	`src/transformers/pipelines/object_detection.py`
`zero-shot-image-classification`	`src/transformers/pipelines/zero_shot_image_classification.py`
`zero-shot-object-detection`	`src/transformers/pipelines/zero_shot_object_detection.py`
`depth-estimation`	`src/transformers/pipelines/depth_estimation.py`
`mask-generation`	`src/transformers/pipelines/mask_generation.py`
`keypoint-matching`	`src/transformers/pipelines/keypoint_matching.py`
`video-classification`	`src/transformers/pipelines/video_classification.py`