Wearable live captions (making mask wearing more accessible for those who are hard of hearing)

A talk from EMF 2022 by Jo Franchetti

On Friday June 3, 2022 at in Stage C

During the pandemic we've all been feeling pretty isolated, and we've all been doing our best and wearing masks. But what if everyone wearing masks cuts off your ability to converse? My lil mum relies on lip reading and clear sounds to understand what people are saying. But I, of course want her to stay safe. So the thought occurred. Can I make a live caption display fit into a mask so that she can read what I'm saying?

This talk will cover how to build a wearable LED display, how to build a web app which uses AI to convert speech to text and how to compress that text into a scrolling pixel font to send to the display.

We'll be using Node, JavaScript and some C, Azure Cognitive Services and MQTT.