Google DeepMind's Sign Language Model Ships on a Phone, Not a Research Paper

Real-time sign language translation just became a Pixel feature instead of a research demo. Google DeepMind announced SL2T, a multilingual sign-language-to-text model launching in Gboard and Live Transcribe on Pixel 11, starting with American Sign Language to English. The model was trained on more than 100,000 hours of data spanning over 50 sign languages — roughly a quarter of it ASL — and reaches a zero-shot score of 70 BLEURT, ahead of prior benchmarks, translating directly to text without the intermediate “gloss” annotation step that earlier systems relied on. Skipping that step lets SL2T capture non-manual markers and spatial grammar those systems tended to miss.

The architecture choice is the real story for anyone evaluating on-device AI: SL2T processes signing through on-device pose tracking that extracts geometric coordinates and discards the original video footage immediately, rather than running raw camera feed through a model. That sidesteps the privacy objection that has dogged camera-based translation products since the category existed. Google also formed an AI Sign Language Advisory Committee with Deaf-led organizations to guide the rollout, and the engineering work specifically addressed streaming latency, left-handed signing, and one-handed use while holding a phone — constraints that only show up once a model leaves the lab for a shipping device.

SL2T isn’t an isolated bet. Meta’s DINO and Segment Anything models are doing similar work one layer down the stack, powering the University of Pittsburgh’s ARPA-H-funded RAMMP wheelchair-assistance platform — a program built on 5.5 million US wheelchair users and 100,000+ annual wheelchair-related ER visits — where the same edge-deployment tradeoff applies: recognition accuracy weighed against power draw and battery life on compact hardware. Two frontier labs, two different accessibility problems, the same underlying constraint: foundation vision models only matter here once they’re small and private enough to run on the device in someone’s hand.