For years, text-to-speech had a very specific, very annoying problem: it could read words, but it couldn’t feel them. Ask any TTS model to sound “barely holding it together, forcing a smile”, and you’d get the same cheerful corporate voiceover delivery, regardless of what the brackets said. The emotion tags were there in the API docs, but the models treated them like a suggestion from an annoying manager, acknowledged, then completely ignored.
Fish Audio just fixed that. Their new Drama 3 model, a 35B-parameter Mixture-of-Experts architecture with about 3B active parameters per token, doesn’t just read your script, it takes direction. And early benchmarks suggest it’s not just the most expressive open TTS model ever released, it’s genuinely competitive with the proprietary incumbents that have dominated the space.
Here’s what actually changed, why the architecture matters, and whether you should care enough to free up 48GB of VRAM.
The Same Direction, Delivered With Actual Feeling
The defining feature of Drama 3 is its ability to follow plain-language acting directions. You write a line like this:
[barely holding it together, forcing a smile] I'm fine. Really. Go on without me.
And the model actually delivers a voice that cracks while holding a smile, with a shaky breath landing exactly where you placed it. Compared to Fish’s own S2.1 Pro, which renders the same line with all the emotional range of a voicemail greeting, the difference is stark.
But the directional control goes deeper than just line-level emotion. Drama 3 supports:
- Paired tags like
<whisper>…</whisper>,<emphasis>…</emphasis>, and<soft>…</soft>that modify only the enclosed words - Mid-line direction changes where the read shifts from “breathless, speaking fast” to “slow, deadly serious” within the same sentence
- Sounds and pauses placed with bracket notation like
[shaky breath]exactly where they should occur - Directions in any language, regardless of the script’s language, write
[像是在强忍着眼泪，却努力笑着]for Chinese direction on an English line - Experimental mid-line language switching using tags like
<french>Je voudrais une table pour deux</french>
The tag system for specific words is particularly useful. In the launch post, Fish demonstrates a line where only the final phrase gets wrapped in <emphasis>do not open that door.</emphasis>, the rest of the sentence stays at a terrified whisper, then those four words land like a hammer. Previous models would either flatten the emphasis across the whole line or ignore it entirely.
Every Word Still Lands: The Pronunciation Numbers
Emotion is great, but it’s useless if the model garbles “Chapter XVII” while trying to sound dramatic. Fish claims Drama 3 tops Artificial Analysis’s independent Pronunciation Robustness benchmark, and the numbers back it up.
| Category | Drama 3 | S2.1 Pro |
|---|---|---|
| Brands, places, technical terms | 95.9% | 87.3% |
| Context-dependent readings | 95.3% | 86.1% |
| Shorthand (numbers, dates, units) | 93.6% | 92.7% |
| Exact sequences (codes, paths, emails) | 86.3% | 79.4% |
| Overall | 93.2% | 88.1% |
The biggest wins are on proper nouns and context-dependent readings, precisely where previous TTS models struggled. S2.1 Pro read only 87.3% of brand names and technical terms correctly, Drama 3 pushes that to 95.9%. Exact sequences like file paths and email addresses still lag at 86.3%, but that’s a substantial jump from 79.4%.
Why the Architecture Breaks the Mold
The dramatic jump in expressiveness isn’t an accident of fine-tuning. Drama 3 introduces a genuinely different architecture from its predecessor, detailed in the technical report.
Backbone: A 35B Mixture-of-Experts with 8 of 256 routed experts plus one shared expert active per token (~3B active parameters), initialized from Qwen3.5-35B-A3B’s language model.
Hybrid attention: 40 layers organized in blocks of three Gated DeltaNet layers followed by one gated full-attention layer. The DeltaNet layers maintain a constant-size state, meaning only one layer in four needs a KV cache, a massive memory savings that enables the long context needed for full scripts and multiple reference voice clips.
Codebook-MTP decoder: Replaces S2’s per-frame Fast AR with a component that attends causally over the entire sequence, including the reference audio’s fine codes. This means the decoder “listens back” to everything generated so far, which is essential for maintaining consistent emotional delivery across long passages.
Partial codebook fusion: Only acoustic codebooks 0 and 1 feed back into the backbone, rather than all ten as in S2. This limits exposure bias during training.
Reinforcement learning: Instead of rewarding models for being “easy to transcribe” (which punishes expressive choices like sobbing or whispering), the team trained a single learned human-preference reward model. The RL stage uses GRPO without a KL penalty, with 1,536 prompts × 8 takes per step. The reward model also hears the script and voice reference, so accuracy still matters, a laugh you requested won’t be penalized, but a mispronounced word will be.
This RL shift is quietly revolutionary. Most speech models are tuned with rewards for being “clear and neutral”, which systematically flattens emotional delivery. By optimizing for human preference instead of transcription ease, Fish Audio trained a model that doesn’t treat emotion as a bug. The resulting performance on the Artificial Analysis quality leaderboard shows Drama 3 at Elo 1242, above Fish’s own S2.1 Pro at 1190 but still trailing ElevenLabs’ v4 Turbo at 1328. The officials would say that’s “competitive.” Reality says the gap is closing fast.
The Multi-Speaker Party Trick
Drama 3 handles full scenes with up to 16 distinct speakers in a single API request using speaker tags. Here’s the actual curl command for a three-character scene:
curl --fail --show-error --request POST https://api.fish.audio/v1/tts \
--header "Authorization: Bearer $FISH_API_KEY" \
--header "Content-Type: application/json" \
--header "model: drama-3" \
--data '{
"text": "<|speaker:0|>[calm, measured storyteller voice] The ship had been silent for three days.<|speaker:1|>[low, gravelly, tired but in control] Status report.<|speaker:2|>[fast, nervous, stumbling over words] Uh, sir, so, the thing is, the reactor is, it is not, it is not great.<|speaker:1|>[slow, dangerously quiet] Define \"not great.\"<|speaker:2|>[voice breaking, close to panic] It is going to <emphasis>blow</emphasis>.",
"reference_id": ["c70cfb98867a4bfb9ae20d2401257b92", "bf322df2096a46f18c579d0baa36f41d", "f3a2b90078d54a65af4b95f63bc7798e"],
"format": "mp3"
}' \
--output dialogue.mp3
Each speaker gets a voice ID, direction tags work per-turn, and the whole scene renders in one shot. For audiobook narrators who slip between character voices, or game developers generating NPC dialogue, this collapses what used to be a multi-step pipeline into a single API call. Directions and tags work identically in the web app if you’d rather point-and-click your way through a scene.
The Elephant in the Room: Hardware Requirements
Here’s where reality sets in. Drama 3 is a chunky model. The installation guide lists a minimum of 48GB of NVIDIA GPU memory in FP8, and CPU, Apple MPS, and AMD ROCm are all unsupported. The MoE experts ship in FP8 with remaining weights in bf16. For reference, that means you need a DGX Spark, an H100, or an A6000-class workstation. An RTX 4090’s 24GB won’t cut it for the full-fat version.
But the community has already started working on this. Dramatically trimmed quantized versions are emerging: one developer released an INT4/GGUF variant that runs in under 24GB, and there are reports of it working in as little as 8GB of RAM with heavy quantization. The 4-bit quantized versions reportedly get “very close to the original” quality, per early tests. Drama 3 has also been merged into audio.cpp on the main branch, though it’s not in the v0.9.2 release yet.
The quantization angle is worth watching. Dramatic improvements in quantization techniques enabling efficient local deployment have been trickling through the ecosystem, and the ability to run a model of this quality on consumer hardware could be a genuine inflection point for self-hosted TTS. The idea of running massive MoE models on consumer hardware through optimization breakthroughs isn’t science fiction anymore, it’s landing in HuggingFace repos. When your 8GB GPU can start running models that would have needed a datacenter a year ago, the entire economics of voice synthesis shifts.
Beyond the hardware, the infrastructure story is strong. Drama 3 serves through SGLang-Omni, with fused kernels validated on Hopper GPUs. On a single H200, median time to first audio is 88.9ms with a real-time factor of 0.131. At 16 concurrent streams, throughput hits 927.6 frames per second. The real-time streaming path on an H100 reportedly achieves a 0.32 RTF, generating audio at about 3.1× real time. This is a datacenter-grade model, and while the community quants are promising for local tinkerers, production deployments will want proper GPUs.
Open Weights, With a Catch
Drama 3’s weights are on Hugging Face under the Fish Audio Community License v1.0. Free for research and non-commercial use, and free for commercial use if your annual revenue (including affiliates) is under $500,000. Above that threshold, or if you want to offer Drama 3 as a service to others, you need a commercial license from Fish Audio. The license also strictly forbids cloning a real person’s voice without their consent.
Fish Audio’s own S2 model remains under the more restrictive Research License, so Drama 3 actually represents a loosening of the licensing around their flagship models. The $500K revenue threshold is a smart move, it captures the hobbyist and indie developer market while ensuring that companies building serious TTS products on top of this model will eventually pay.
For the API, list price is $50 per million characters. Until November 12, 2026, it’s 80% off at $10 per million, which is aggressively cheap for this quality tier when ElevenLabs charges $40 for v4. The free tier of their S2.1 Pro API is genuinely useful for testing, and switching an existing integration to Drama 3 is a single header change: just set model: drama-3 on your existing /v1/tts request. Missing or misspelled headers silently fall back to s2.1-pro, which is forgiving (though potentially confusing if you’re not watching your API logs).
What Drama 3 Is Actually For
Fish Audio pitches this as a model for scripted content: audiobooks, audio drama, game dialogue, and short-form video. These are use cases where you render once and keep the output, where lines are carefully scripted rather than generated on the fly. The directional control is designed for people who know exactly what they want and want to iterate on the delivery until it’s right.
The CEO’s framing is succinct: “Voice AI has gotten very good at generating speech. The next step is directing a performance.” That’s the right lens for understanding Drama 3. It’s not trying to be a conversational voice for agents (though it can certainly do that), it’s designed as a tool for creators who want an AI voice actor that takes notes.
The upcoming editing features could be even more disruptive. Interactive editing, where you tell the model “softer and warmer than that take” and it regenerates with the earlier takes in mind, and audio inpainting, where you mark a single word, type a replacement, and Drama 3 regenerates just that word, matching the voice, mood, and pacing of the surrounding audio, are both in invite-only testing. The inpainting demo shows the line “I left his jacket by the door” becoming “I left his scarf by the door” with only the single word regenerated. When that lands in the API, it’ll change the audiobook editing workflow from “regenerate the whole chapter and pray” to “fix the one word that’s wrong.”
The State of the TTS Wars
Drama 3 arrives at an interesting moment. Qwen’s 27B model is rumored to be coming, and the efficiency gains from projects like GLM-4.7 Flash’s KV cache optimizations are making formerly datacenter-only models increasingly practical on local hardware. Fish Audio isn’t just competing with ElevenLabs anymore, they’re competing with the entire open-weight ecosystem that’s racing toward “good enough to be indistinguishable” TTS.
But on expressiveness specifically, Drama 3 is a statement. It says that RLHF-style training can work for speech, that human preference rewards can produce a model that doesn’t flatten emotion, and that open-weight TTS can be taken seriously by people who need actual acting, not just pronunciation.
There are real limitations, the 48GB VRAM requirement, the Elo gap with ElevenLabs on quality, the pronunciation weakness on exact sequences, the license revenue cap that excludes larger companies from free commercial use. But the direction of travel is unmistakable. If Fish Audio keeps shipping at this pace, the proprietary TTS walls are going to get a lot harder to justify. The “revolutionary” part isn’t that Drama 3 exists, it’s that the weights are sitting right there on Hugging Face, waiting for someone with a beefy GPU to make it their own.
