Speech-to-Text for Deaf and Hard of Hearing Users: Beyond Accuracy

For deaf and hard of hearing users, speech-to-text isn't a convenience—it's access. Live captions in meetings, transcripts of phone calls, and real-time subtitles can mean the difference between participation and exclusion.

But accuracy metrics don't capture everything that matters for this community. Here's what product teams should understand when building STT for deaf and hard of hearing users.

What accuracy metrics miss

Word Error Rate tells you how many words are wrong, but not:

Which words are wrong

Missing a name, a number, or a key term is very different from missing "um" or "the." For deaf users relying on captions, critical information errors cause real harm. See our WER explainer for what the metric actually measures.

How errors impact comprehension

Five errors scattered across a paragraph might be fine. Five errors clustered in one sentence can make it incomprehensible.

Non-speech information

Who's speaking? Is there laughter, applause, music, a phone ringing? This context matters for understanding what's happening.

Timing and readability

Captions that lag behind speech, appear in overwhelming chunks, or flash by too quickly fail users even when the words are right.

Design considerations beyond accuracy

Speaker identification

In multi-person conversations, knowing who said what is essential. Good speaker diarization isn't optional—it's core functionality.

Display approaches:

• Name labels before each speaker turn
• Color coding for different speakers
• Position (left/right) to indicate speaker

Sound descriptions

Important non-speech audio needs indication:

• [laughter]
• [phone ringing]
• [music playing]
• [applause]
• [door closes]

Automated sound detection is improving but often needs human review for important content.

Reading pace

Captions need to be readable, not just accurate. Consider:

• Characters per second: Research suggests ~15 CPS maximum for comfortable reading
• Line breaks: Break at natural phrase boundaries, not mid-thought
• Persistence: Leave text on screen long enough to read
• Chunking: Don't display too much text at once

Visual presentation

Design choices affect readability:

• High contrast (white text on black background is common)
• Sufficient font size
• Sans-serif fonts for clarity
• Positioning that doesn't obscure important content
• Customization options (users have different preferences)

Latency tolerance

Deaf users are generally tolerant of reasonable caption delay (1-3 seconds), but longer delays create disorientation—speakers have moved on while captions catch up.

For live events, the tradeoff between accuracy and latency is real. Communicate what users should expect. See our comparison of real-time vs. batch transcription for the architectural considerations.

Common failures

Assuming captions are "good enough"

Automated captions that would be merely annoying for hearing users can be exclusionary for deaf users. Quality matters more, not less.

Ignoring the feedback loop

Deaf users can't hear when captions fail. Provide other ways to flag issues, and actually act on feedback.

One-size-fits-all design

Deaf users have diverse preferences and needs. Hard of hearing users may use captions differently than profoundly deaf users. Customization helps.

Treating accessibility as an afterthought

Bolting captions onto a product designed without them creates awkward experiences. Design for captions from the start.

When automatic STT isn't enough

For high-stakes situations, human captioning often remains necessary:

• Legal proceedings
• Medical appointments
• Job interviews
• Educational assessments
• Emergency communications

Automatic captions are improving rapidly, but knowing their limitations prevents harmful failures.

What deaf users actually want

Research consistently shows deaf users want:

• Accuracy on essential information (names, numbers, key terms)
• Speaker identification in group settings
• Non-speech sound indication
• Readable pacing over raw speed
• Customization of display preferences
• Reliability they can count on

Build with these priorities, not just WER benchmarks.

Compliance and standards

The W3C Web Content Accessibility Guidelines (WCAG) set requirements for caption quality. Our WCAG compliance guide covers the specific requirements for audio content.

The National Association of the Deaf advocates for quality captioning standards and can provide guidance on community needs.

For broader voice feature accessibility, see our guide on designing voice features that actually help.