Temperature, top-k, top-p and repetition penalty, tuned by ear

Modern open TTS models such as Qwen3-TTS generate speech the way a language model generates text: one token at a time, each picked from a probability distribution. That means they expose the same sampling settings as a chat model. The names are familiar. What they do to a voice is not.

This is part four of the voice agent series. It explains each setting in audio terms, how I calibrate them automatically, and why the final say still belongs to a person listening.

What the tokens are

Qwen3-TTS works with a neural audio codec running at 12.5 frames a second. Each frame is 80 ms of sound, described by 16 codebook tokens: the first carries the broad shape of speech, and the rest add the detail that makes a voice recognisable. Sampling decides which of the plausible next frames the model actually commits to. Pick too safely and speech goes flat. Pick too adventurously and you get mumbling, slurring, or a voice that wanders into someone else's.

The four knobs, in audio terms

• Temperature reshapes the whole distribution. Low values sharpen it towards the most likely frame, which sounds controlled but monotone. High values flatten it, which adds expressiveness until prosody turns erratic. For a fine-tuned voice the useful band is narrow: roughly 0.7 to 1.0.
• Top-k keeps only the k most likely tokens and discards the rest. On audio it's the knob that separates a stable voice from a varied one. Small values (around 10 to 20) hold the speaker's identity tightly; large values let more of the model's general "average voice" leak in.
• Top-p keeps the smallest set of tokens whose probabilities add up to p. In theory it adapts to how confident the model is. In practice, on this model, it was a much weaker knob by ear than top-k.
• Repetition penalty makes tokens already generated less likely. In text it stops a model looping. Speech repeats sounds constantly, so its effect on audio is subtler and less predictable. It's the setting I'd tune last, and by ear rather than by reasoning.

Starting points

Each way of making a voice ships with slightly different defaults. These are what the project uses before any calibration:

Voice cloning runs coolest because it's working from only a few seconds of reference audio, and anything that loosens sampling also loosens its grip on the speaker. Voice design can run hotter because there's no real person to drift away from.

Calibrating automatically

Every fine-tuned voice in the project goes through a calibration stage. It renders the same neutral sentences at 16 combinations: temperature 0.7, 0.8, 0.9 and 1.0, against top-k 10, 20, 40 and 80, with top-p fixed at 0.95 and repetition penalty at 1.1. Each rendering is scored against reference recordings of the real speaker:

• Speaker similarity, from a speaker-embedding model, is the main score.
• Guards reject combinations that produce babble or dead air: word error rate above 15%, any pause longer than 2 s, or more than 2 s of trailing silence.
• Acoustic distance, comparing pitch, energy, speaking rate and brightness with the reference clips, breaks ties.

Here's part of the grid for Maya's voice:

Two things stand out. First, intelligibility collapses at high top-k: the 80 column is where errors appear, and they get worse as temperature rises. Second, look at the similarity column. Every combination scored between 0.989 and 0.994. The metric is good at catching a broken voice and nearly useless at ranking good ones. The calibration recommended temperature 0.9 with top-k 10, and that's a starting point to audition, not a verdict.

Then you listen

For Maya on this website I adjusted the calibrated settings by ear to temperature 0.85, top-k 13, top-p 0.95, and a much stronger repetition penalty of 2. That's a long way from what any automatic metric would pick, and it suits the job: short, conversational answers where a consistent, steady voice matters more than dramatic range. A storytelling voice would land somewhere else entirely.

The general lesson from the fine-tuning work applies here too: use metrics to find the safe region and to catch regressions, then let a person make the final call.

Reproducible voices

Sampling is random, which is a problem when you design a voice from a text description and want to keep it. The Qwen3-TTS sampler doesn't take a seed, so the project seeds MLX's random number generator directly before each render. With a fixed seed, the same description and text produce bit-identical audio, so a designed voice can be stored, shared, and rebuilt exactly.

The LLM runs hot, the voice runs cool

One last contrast. The language model behind Maya's words runs at a high temperature, well above typical chat defaults, so her replies don't sound scripted, while staying inside the model's recommended top-k and top-p. The TTS model runs cool so her voice stays put. Variety belongs in what she says; consistency belongs in how she sounds. Getting those two the right way round made more difference than any single setting.