Pipelines

Purpose

Pipeline is the high-level inference API. It packages "tokenize → forward → decode" (or the multimodal equivalent) for 25+ task types so a one-liner produces useful output. It also implements batching, streaming, GPU placement, and optional asynchronous serving.

Key abstractions

Class / function	File	Role
pipeline factory	src/transformers/pipelines/__init__.py (66K LOC)	Top-level pipeline(...) constructor and the task registry
Pipeline (base)	src/transformers/pipelines/base.py (1,370 LOC)	Common machinery: device placement, batching, framework guards
PIPELINE_REGISTRY	src/transformers/pipelines/__init__.py	Maps task name → pipeline class + default model
ChunkPipeline	src/transformers/pipelines/base.py	Subclass for tasks whose inputs are chunked (long ASR, long QA)
KeyDataset, KeyPairDataset, PipelineDataset	src/transformers/pipelines/pt_utils.py	Dataset adapters

Available tasks

Task	Class	File
text-generation	TextGenerationPipeline	src/transformers/pipelines/text_generation.py (25K LOC)
text-classification (sentiment-analysis)	TextClassificationPipeline	src/transformers/pipelines/text_classification.py
token-classification (ner)	TokenClassificationPipeline	src/transformers/pipelines/token_classification.py
question-answering	QuestionAnsweringPipeline	(in __init__.py and text_classification.py)
fill-mask	FillMaskPipeline	src/transformers/pipelines/fill_mask.py
summarization, translation, text2text-generation	Text2TextGenerationPipeline	src/transformers/pipelines/text_generation.py
feature-extraction	FeatureExtractionPipeline	src/transformers/pipelines/feature_extraction.py
zero-shot-classification	ZeroShotClassificationPipeline	src/transformers/pipelines/zero_shot_classification.py
automatic-speech-recognition	AutomaticSpeechRecognitionPipeline	src/transformers/pipelines/automatic_speech_recognition.py (35K LOC)
audio-classification	AudioClassificationPipeline	src/transformers/pipelines/audio_classification.py
text-to-audio	TextToAudioPipeline	src/transformers/pipelines/text_to_audio.py
zero-shot-audio-classification	ZeroShotAudioClassificationPipeline	src/transformers/pipelines/zero_shot_audio_classification.py
image-classification	ImageClassificationPipeline	src/transformers/pipelines/image_classification.py
image-segmentation	ImageSegmentationPipeline	src/transformers/pipelines/image_segmentation.py
image-feature-extraction	ImageFeatureExtractionPipeline	src/transformers/pipelines/image_feature_extraction.py
image-text-to-text	ImageTextToTextPipeline	src/transformers/pipelines/image_text_to_text.py (23K LOC)
object-detection	ObjectDetectionPipeline	src/transformers/pipelines/object_detection.py
zero-shot-image-classification	ZeroShotImageClassificationPipeline	src/transformers/pipelines/zero_shot_image_classification.py
zero-shot-object-detection	ZeroShotObjectDetectionPipeline	src/transformers/pipelines/zero_shot_object_detection.py
keypoint-matching	KeypointMatchingPipeline	src/transformers/pipelines/keypoint_matching.py
depth-estimation	DepthEstimationPipeline	src/transformers/pipelines/depth_estimation.py
mask-generation	MaskGenerationPipeline	src/transformers/pipelines/mask_generation.py
video-classification	VideoClassificationPipeline	src/transformers/pipelines/video_classification.py
document-question-answering	DocumentQuestionAnsweringPipeline	src/transformers/pipelines/document_question_answering.py (29K LOC)
table-question-answering	TableQuestionAnsweringPipeline	src/transformers/pipelines/table_question_answering.py
any-to-any	AnyToAnyPipeline	src/transformers/pipelines/any_to_any.py (26K LOC)

The pipeline() factory accepts a task string or auto-detects from the model.

How Pipeline runs

graph LR
    Input --> Pre[preprocess]
    Pre --> Forward[_forward]
    Forward --> Post[postprocess]
    Post --> Output

Every pipeline implements three hooks: preprocess, _forward, postprocess. The base class wraps them with batching (batch_size=), iteration over generators, GPU dispatch, and optional streamer support for text-generation.

Picking models automatically

pipeline(task="text-generation") (no model) loads gpt2 because of the registry entry in __init__.py. Per-task default models are kept conservative; production code should pass model="..." explicitly.

Device placement

pipe = pipeline("text-generation", model="...", device=0)         # cuda:0
pipe = pipeline("text-generation", model="...", device_map="auto") # accelerate-managed
pipe = pipeline("text-generation", model="...", dtype=torch.bfloat16)

The base class dispatches inputs to the right device per batch, including for image and audio tensors.

Streaming

Text-generation pipelines accept a streamer= argument. For asynchronous use cases (e.g., serving) AsyncTextStreamer from src/transformers/generation/streamers.py exposes an async for interface that transformers serve wraps in OpenAI-compatible SSE responses.

Adding a new pipeline

1. Add <task>.py under src/transformers/pipelines/ with a <Task>Pipeline(Pipeline) subclass.
2. Implement preprocess, _forward, postprocess.
3. Register in PIPELINE_REGISTRY in pipelines/__init__.py.
4. Add tests under tests/pipelines/test_pipelines_<task>.py using PipelineTesterMixin.
5. Run make fix-repo.

The reference doc is docs/source/en/add_new_pipeline.md.

Testing

tests/test_pipeline_mixin.py (37,715 LOC) contains the shared PipelineTesterMixin. Per-task tests are in tests/pipelines/. The mixin is also reused inside tests/models/<arch>/test_modeling_<arch>.py to ensure each model that claims to support a task actually does.

Integration points

• The CLI transformers chat is a thin wrapper around the text-generation pipeline plus a chat template.
• Trainer does not use pipelines directly, but examples and notebooks frequently load a fine-tuned model into a pipeline for evaluation.
• transformers serve exposes pipelines (and generate) over HTTP.

Entry points for modification

• New task → see "Adding a new pipeline" above.
• Cross-cutting change to Pipeline (batching, device dispatch) → edit src/transformers/pipelines/base.py.
• Per-task tweaks → edit the task file and corresponding tests/pipelines/test_pipelines_<task>.py.