huggingface/transformers

Audio models

Speech recognition, audio classification, text-to-speech, music, and unified audio LMs.

Automatic speech recognition (ASR)

Directory	Notes
`whisper/`	OpenAI Whisper — the dominant ASR family
`wav2vec2/`, `wav2vec2_bert/`, `wav2vec2_conformer/`, `wav2vec2_phoneme/`, `wav2vec2_with_lm/`	CTC-based ASR
`hubert/`, `unispeech/`, `unispeech_sat/`, `wavlm/`, `data2vec/`	Self-supervised speech encoders
`sew/`, `sew_d/`, `clvp/`, `speech_to_text/`	Older ASR baselines
`parakeet/`, `lasr/`, `glmasr/`, `kyutai_speech_to_text/`, `cohere_asr/`, `vibevoice_asr/`, `voxtral/`, `voxtral_realtime/`, `granite_speech/`, `granite_speech_plus/`, `moonshine/`, `moonshine_streaming/`	Recent (2024-2026) ASR / speech-to-text models
`csm/`, `dia/`, `mimi/`, `xcodec/`, `dac/`, `encodec/`, `vibevoice_acoustic_tokenizer/`, `higgs_audio_v2_tokenizer/`	Audio tokenizers / codecs

Audio classification

Directory	Notes
`audio_spectrogram_transformer/`	AST
`clap/`	Contrastive audio-language pretraining (zero-shot audio classification)
`wav2vec2/`, `hubert/`, `data2vec/` (also reused)	Frame-level / utterance-level classification heads

Text-to-speech and audio generation

Directory	Notes
`bark/`	Suno Bark
`musicgen/`, `musicgen_melody/`, `musicflamingo/`, `pop2piano/`	Music generation
`speecht5/`, `vits/`, `fastspeech2_conformer/`, `univnet/`	TTS
`seamless_m4t/`, `seamless_m4t_v2/`	Speech-to-speech translation
`moshi/`, `csm/`, `dia/`, `higgs_audio_v2/`, `mimi/`, `vibevoice_acoustic_tokenizer/`, `vibevoice_asr/`, `kyutai_speech_to_text/`	Streaming / interleaved speech LMs

Unified audio LMs

Directory	Notes
`audioflamingo3/`, `qwen2_audio/`, `qwen2_5_omni/`, `qwen3_omni_moe/`, `phi4_multimodal/`, `granite_speech/`, `pe_audio/`, `pe_audio_video/`	Models that mix text and audio in their input/output

Audio models declare a feature extractor in src/transformers/models/<arch>/feature_extraction_<arch>.py. The base class SequenceFeatureExtractor (src/transformers/feature_extraction_sequence_utils.py, 19K LOC) handles padding, truncation, and waveform → log-mel conversion. Common helpers are in src/transformers/audio_utils.py (55K LOC): STFT, mel filterbanks, fbank features, voice activity detection.

Pipelines that consume audio models

Pipeline task	Module
`automatic-speech-recognition`	`src/transformers/pipelines/automatic_speech_recognition.py` (35K LOC)
`audio-classification`	`src/transformers/pipelines/audio_classification.py`
`text-to-audio`	`src/transformers/pipelines/text_to_audio.py`
`zero-shot-audio-classification`	`src/transformers/pipelines/zero_shot_audio_classification.py`

Streaming and continuous batching

Whisper, the wav2vec2 family, Moonshine-streaming, Voxtral-realtime, and Qwen2.5-Omni are designed for streaming inference. transformers serve --enable-audio (when present) routes via the streaming-aware paths in src/transformers/cli/serving/.