Whisper

Whisper is OpenAI's general-purpose speech recognition model. The whisper Python package wraps a Transformer encoder-decoder that performs multilingual transcription, speech translation into English, and language identification from audio. The repository ships pretrained model weights, the inference code that runs them, a whisper CLI, and the supporting tokenizer, audio preprocessing, decoding, and word-timing utilities.

What's in this repository

The repo is small for the scope of what it does. About 4,300 lines of Python in whisper/ cover everything from raw audio I/O to beam search to subtitle output. There is one Python package, one CLI entry point, and a handful of tests. There is no training code in this repository — only inference. The published research and training procedure are described in the paper and in model-card.md.

Headline features

• Multilingual ASR in 99 languages, with language auto-detection from the first 30 s of audio.
• Speech translation into English (--task translate) for multilingual models.
• Sliding-window long-form transcription with temperature fallback, no-speech detection, and prompt conditioning across windows.
• Word-level timestamps via cross-attention DTW alignment, optionally accelerated with a Triton CUDA kernel.
• Multiple model sizes — tiny, base, small, medium, large (v1/v2/v3), and turbo (an optimized large-v3) — most with English-only variants.
• Subtitle and transcript writers that emit txt, vtt, srt, tsv, and json.

How to navigate this wiki

• New here? Start with Architecture for a one-page mental model, then Getting started for installing and running.
• Want to know the meaning of n_audio_ctx, sot_sequence, or "fallback"? See the Glossary.
• Hacking on a specific subsystem? Jump to Systems and pick the file you care about (audio.py, model.py, decoding.py, transcribe.py, timing.py, tokenizer.py, normalizers, output writers).
• Want to understand the user-facing capabilities end to end? See Features.
• Reference material (CLI flags, model dimensions, data formats) lives in Reference.
• Numbers about the codebase are in By the numbers; the project's history is in Lore.

Repository links

• Source: https://github.com/openai/whisper
• Paper: https://arxiv.org/abs/2212.04356
• Blog: https://openai.com/blog/whisper
• Model card: model-card.md
• License: MIT, see LICENSE