AMERICAN SOCIETY FOR DEAF CHILDREN | Fingerspelling with Machine Learning | 2020
Machine learning connects a parent and child's conversation / American Society for Deaf Children "Fingerspelling"
In the United States, 2 to 3 out of every 1,000 babies are born deaf. 9 out of 10 of them are born to hearing parents. In many cases, that child becomes the first deaf person their parents have ever known.
The problem
If children aren't exposed to sign language early on, they risk missing out on language itself. And yet 72% of families don't sign with their deaf child. Videos and textbooks exist, but there was no way to check whether you were actually doing it right.
The insight
Hello Monday, a creative studio known for playing with technology, landed on the idea of combining hand tracking with ASL fingerspelling. Then they did something unusual: they finished the prototype on their own and sent it, unprompted, to Cheri Dowling at the American Society for Deaf Children. As Dowling put it, "I get emails every day from people trying to fix our community. But this one was different."
The idea
Fingerspelling.xyz is a site where you can practice fingerspelling using nothing but a webcam. A 3D hand on screen shows the correct shape, and you mimic it with your own hand in front of the camera. MediaPipe Hands reads your finger movements, and if you get it right, your score goes up instantly.
What happened
In the first 10 days, people formed correct hand signs 150,000 times; over 10 months, that number reached 5.5 million. The American Society for Deaf Children still uses it as one of its own teaching tools today. The project has won multiple awards, including a D&AD Gold Pencil.
Afterthought
A studio that just wanted to play around with technology ended up creating someone's very first conversation. That's the kind of thing I want to make too.
Awards
- Cannes Lions Gold 2020
