Giving AI characters a voice — TTS latency, cost and craft

Text gives a character a mind; voice gives them a body. The retention delta is real — a character who speaks gets opened again. But voice is also the easiest feature to build badly: expensive, slow, and flat. This is the playbook we use.

Voice as a moment, not a modality

The instinct is to voice everything. Resist it. Synthesis costs real money per character of input (3 credits per 100 characters here), and a wall of autoplayed audio is fatiguing anyway.

Voice works as punctuation: the greeting when a conversation opens, an emotional beat, a reply the user explicitly asks to hear. Products that treat voice as a premium moment spend a tenth as much and get more retention from it, because scarcity is what makes it land.

The cost model

POST /v1/audio/speech meters per started 100-character block. Practical numbers:

Line	Characters	Credits	~Cost
Short greeting	80	3	$0.03
Typical reply	240	9	$0.09
Long dramatic beat	600	18	$0.18

Two implications. First, write for voice: a spoken line should be shorter than a written one anyway — trimming to 200 characters is a craft improvement that halves your bill. Second, cache aggressively: greetings, catchphrases and recurring beats are the same audio every time. Hash voice + text, store the MP3 yourself (outputs are not retained on our side and are yours to keep), and a huge share of playbacks become free.

Latency and the perception of it

Synthesis takes a second or two. The trick is that perceived latency is a UI property:

• Show the text reply immediately; let the audio arrive behind it with a subtle "voice loading" affordance. Users read while it renders.
• For scripted moments (onboarding, greetings), pre-generate — those lines are known before the user ever arrives.
• Never block the conversation on audio. Voice that fails should degrade to text silently, not error the turn.

Making it feel performed

The difference between "read aloud" and "performed" is mostly in the input text:

• Punctuation is prosody. Commas, dashes and ellipses are how you direct pacing. "Well… that's new." performs; "Well that is new" reads.
• Keep character voice in the words. The synthesis voice carries timbre; the writing carries personality. A sardonic character needs sardonic sentences, not a sardonic voice setting.
• Match voice to character once, then never change it. Voice identity is character identity — switching voices mid-relationship is the audio equivalent of swapping the avatar.

The roster (aria, orion — see the reference) is curated rather than infinite on purpose: two voices that always sound right beat forty that sometimes do.

The round trip: ears too

The same audio pipeline runs the other way — /v1/audio/transcriptions (5 credits) turns a user's voice message into text your character can answer. Voice in, text reasoning, voice out is the full loop of a "call mode", and each leg is one API call.

Start with greetings: one cached line per character, a few credits each, shipped in an afternoon. Measure what it does to day-two retention, then decide how much further to take it. In our experience, that first afternoon is the one that changes the product.