You've got two hours of recorded interview and you need the text. Or a video that needs subtitles. Or a podcast you want to pull quotes from. The convenient option is uploading the audio to a cloud service that charges per minute and keeps a copy of your recording. The option almost nobody explains well is that you can do all of it on your own computer, for free, without the audio ever leaving it, using open source models that in 2026 already perform at a professional level.
The problem isn't a lack of options, it's the noise. "Whisper" has become an umbrella term hiding half a dozen different implementations, and every week a new model claims to be "the best." Let's sort it out: which model leads, which to use for your case, and how to get it running locally without drama.
Note
This guide is technical but practical. You don't need to know machine learning: if you're comfortable with the terminal and can install a Python package, you can have quality transcriptions in under ten minutes.
What You Want Speech-to-Text For
Before picking a model, it helps to be clear about the job. STT (speech-to-text) covers scenarios that look alike but weigh differently:
- Transcribing meetings, interviews, lectures or podcasts into plain text. Here accuracy matters most, plus handling long audio.
- Subtitles for video: you need the text plus synced timestamps, ideally at word level.
- Diarization: separating who said what when there are multiple voices. Not every model does this out of the box.
- Dictation or real-time transcription: here latency rules, not absolute accuracy.
- Processing volume: thousands of files in a queue, where what matters is throughput (how much audio you process per second).
Each of these cases tips the scales toward a different model. That's why there's no "best" full stop: there's the best for what you're doing.
What Whisper Is and Why It Still Leads
Whisper is the speech recognition model OpenAI released as open source and that changed the game: for the first time there was a free, open-weights model transcribing 99+ languages with an accuracy that until then only paid services delivered. It was trained on 680,000 hours of internet audio, and that diversity gives it brutal robustness against accents, background noise and poor-quality audio.
In 2026 its reference version is Whisper large-v3, with a WER (word error rate) around 7.4% in English and multilingual support no open source competitor matches in breadth. There are smaller versions (tiny, base, small, medium) that trade accuracy for speed and memory, and an optimized large-v3-turbo variant that processes audio at over 200x real time with minimal accuracy loss.
The key to understanding everything: Whisper is the model; the "variants" are ways to run it. faster-whisper, whisper.cpp and WhisperX use the same weights from OpenAI's Whisper. With the same audio and the same settings, they produce practically the same transcription. What changes is the inference engine: the speed, the memory they use and the extras they add.
Tip
Mental rule: when someone says "I use Whisper," ask with which engine. It's the difference between transcribing an hour of audio in five minutes or in forty.
The Whisper Variants: Which for What
This is where 90% of cases get decided. Three engines cover almost everything:
faster-whisper — the workhorse
A reimplementation of Whisper on CTranslate2, a heavily optimized C++ inference engine. It supports quantization (int8), which cuts memory use and speeds up inference up to 4x faster than original Whisper while keeping the same accuracy. If you have an NVIDIA GPU, this is your default: the speed and VRAM winner. It's what I recommend for most people.
whisper.cpp — for Mac and CPU
A pure C/C++ port with no external dependencies. It compiles to a small binary, uses the GGML format with integer quantization, and runs anywhere. On Mac (Apple Silicon) it leverages Metal and Core ML; on machines without a GPU, it's the lightest way to transcribe. If you don't have a dedicated graphics card or work on a laptop, start here.
WhisperX — when you need timestamps and diarization
Instead of chasing raw speed, WhisperX turns Whisper into a complete transcription pipeline: it adds word-level timestamps (phonetic alignment) and diarization (separating speakers). It's the choice for accurate subtitles or interview transcriptions where it matters who said what.
Heads up
There are more variants (insanely-fast-whisper, distil-whisper, etc.), but don't get lost: faster-whisper for GPU, whisper.cpp for Mac/CPU, and WhisperX for timestamps cover almost any real need. The rest are optimizations for specific cases.
Beyond Whisper: Distil-Whisper, Parakeet and Canary
Whisper is no longer alone at the top. Three families deserve your attention if your case is specific:
Distil-Whisper is a distilled version of Whisper: 51% smaller (756M parameters vs 1,550M for large-v3) and about 6x faster, with only ~1% more error on long audio. The catch: it's optimized for English. If you transcribe English and want speed without giving up almost any accuracy, it's a gem.
NVIDIA Parakeet is NVIDIA's English family (with recent multilingual variants). Its strength is brutal throughput: an RTFx (real-time factor) above 2,000, making it unbeatable for processing large volumes of audio in batch. The TDT v3 version adds multilingual support, at the cost of some English accuracy.
NVIDIA Canary-Qwen 2.5B is, as of today, the king of accuracy on Hugging Face's Open ASR Leaderboard: a WER of 5.63%, below Whisper, covering around 25 languages (mainly English and European) with very high inference speed. If what you want is the most accurate transcription possible in those languages, it's the reference.
And for edge devices, models like Moonshine (from 27MB) run on minimal hardware offline, ideal for embedded dictation or IoT.
Comparison Table
| Model / engine | Best for | Accuracy (English WER) | Speed | Languages |
|---|---|---|---|---|
| Whisper large-v3 | Multilingual, max compatibility | ~7.4% | Baseline (reference) | 99+ |
| faster-whisper | NVIDIA GPU, general use | ~7.4% (same) | Up to 4x faster | 99+ |
| whisper.cpp | Mac and CPU without GPU | ~7.4% (same) | Lightweight, optimized | 99+ |
| WhisperX | Subtitles and diarization | ~7.4% + timestamps | Fast + extras | 99+ |
| Distil-Whisper | Fast English | ~9.7% (~1% worse) | ~6x faster | English |
| NVIDIA Parakeet TDT | Volume / batches | ~8.0% | RTFx >2,000 (max) | English / multi (v3) |
| NVIDIA Canary-Qwen | Maximum accuracy | 5.63% (leader) | Very high | ~25 |
The WER numbers come from public benchmarks (Open ASR Leaderboard and independent comparisons) and are indicative: the real error depends on the audio, the language and the configuration. Don't marry a decimal; marry the use case.
Running Whisper Locally, Step by Step
Let's get practical with faster-whisper, the recommended option for most people. You need Python 3.9+ and, ideally, an NVIDIA GPU (it works on CPU too, just slower).
1. Install the package:
pip install faster-whisper2. Transcribe a file with a Python script:
from faster_whisper import WhisperModel
# large-v3 model with int8 quantization (fast, low VRAM)
# Use device="cpu" if you don't have an NVIDIA GPU
model = WhisperModel("large-v3", device="cuda", compute_type="int8")
segments, info = model.transcribe("interview.mp3", beam_size=5)
print(f"Detected language: {info.language} ({info.language_probability:.2f})")
for segment in segments:
print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")3. What each piece does:
WhisperModel("large-v3", ...)loads the large model (it downloads it the first time). Switch to"small"or"medium"if you're tight on memory.compute_type="int8"enables quantization: less VRAM and more speed with barely any quality loss.transcribe()detects the language on its own. If you already know it, passlanguage="en"to speed things up.- Each
segmentcarries text and timestamps (start,end), which is exactly what you need to export to SRT and generate subtitles.
4. For subtitles, loop over the segments and dump the text with its timestamps into SRT or VTT format. If you want word-level precision or to separate speakers, that's the moment to move to WhisperX.
Tip
Always start with a short clip (one minute) to verify language and quality before running the full two hours. It saves time and tells you whether you need to go up or down a model size.
On Mac without a GPU, the equivalent would be compiling whisper.cpp, downloading a GGML model (ggml-large-v3.bin) and running the binary on your file: same quality, native execution optimized for Apple Silicon.
Who Is Each Model For?
There's no absolute winner. There's fit:
faster-whisper if you have an NVIDIA GPU and want the best general option: fast, multilingual, accurate. It's the answer for 80% of people.
whisper.cpp if you work on Mac or a machine with no graphics card. Lightweight, dependency-free, native.
WhisperX if your work is accurate subtitles or transcriptions with multiple identified speakers. Journalists, video editors, podcast people.
Distil-Whisper if you only transcribe English and speed is your priority. Twice as fast for almost no error.
NVIDIA Parakeet if you process industrial volume: thousands of files, automated pipelines, where throughput rules.
NVIDIA Canary-Qwen if you need the maximum accuracy possible in English and European languages, and you don't mind giving up Whisper's 99-language coverage.
The honest question isn't "what's the best speech-to-text model?", because the answer changes with your hardware and your case. The question is "do I transcribe locally, for free and privately, or pay per minute to a service that keeps my audio?". And in 2026, the answer for almost anyone is the first one: open source already transcribes just as well, on your machine, without going through checkout.