Tablet Telephone Adds Captions to Calls

A new phone combines old and new tech to bring call captions to the hearing impaired at home

Loading the video player...

A new phone from Clarity, combines touch-screen tablets and old-fashioned telephony to bring real-time captions of phone calls to the hearing impaired at home. ClearCaptions has long offered this on office phones connected to a network, some smartphones, and on tablet or Web apps. But by combining a traditional landline phone with a Wi-Fi-enabled tablet, Clarity and ClearCaptions can offer it on a home phone.

A service subsidized by the government provides free captioning of phone calls for the hard of hearing. Neither humans nor machines are good enough captioners on their own. Human captioning is too slow, and speech-to-text software makes too many errors if it isn’t “trained” to a new voice. In another combination of old and new, a human agent at the center listens to the conversations and repeats the caller’s words to the software. The software, already trained to the agent’s voice, generates highly accurate text that shows up on the phone’s tablet screen. The actual phone call still comes through the public switched telephone network.

Carsten Trads: We can offer the hearing impaired captioning. You can actually read on the tablet what the person at the other end is saying.

Celia Gorman (voiceover): As touch-screen tablets have gone mainstream, they’ve started appearing as components in other devices. Here, Clarity uses an Android tablet to build a more accessible phone.

Carsten Trads: The phone does all the stuff that we always do in terms of amplification and sound processing, but in addition the tablet technology has allowed us to make a very unique user interface.

If you are vision impaired, you can actually make the keypad bigger, or you can make it smaller. If you have special vision challenges, you can choose to have a different contrast. We use an Android-based tablet to do all that. But in addition we can do something that is very, very unique. We can offer the hearing impaired captioning. You can actually read on the tablet what the person at the other end is saying.

It’s a company called ClearCaption, who have been in this industry for years and they’re one of three FCC [Federal Communications Commission] certified providers of this service. And they’re the one offering this service by kind of establishing a conference call between the two parties in the call and a third party who does the interpretation.

Celia Gorman (voiceover): This is Bob. And this is Alice, who has a closed-captioning phone. It doesn’t matter what kind of phone Bob is using. When Alice dials Bob, she’s also connecting to a captioner at a certified telephone-captioning center. Alice may or may not hear Bob say “Hello,” but the captioner does, and transmits “Hello” to the tablet inside Mary’s phone.

Here’s how it works. A human captioner can’t keep up in real time typing. But automated captioning software would need to be trained to Bob’s voice or else it would make a lot of mistakes. That’s okay for watching a football match on television, but not for a personal phone call. So the captioner repeats Bob’s words to captioning software that has been trained to her voice. And it’s those captions, with very few mistakes, that are transmitted to Alice’s phone.

Carsten Trads: The captioning service is actually free. It is a FCC financed service. So it’s old-fashioned PSTN [Public Switched Telephone Network] telephony combined with the newest Android-based tablet that allows us to do this.

NOTE: Transcripts are created for the convenience of our readers and listeners and may not perfectly match their associated interviews and narratives. The authoritative record of IEEE Spectrum’s video programming is the video.