Translation Blog

Translation Blog

AI sign language interpreter let thousands of hearing-impaired people feel the passion of the Winter Olympics!

release time:May 12,2022

According to statistics, there are nearly 30 million hearing-impaired people in China, just in time for the Winter Olympics, how to let the majority of hearing-impaired people also feel the joy and excitement of our athletes winning the first time

AI sign language interpreters let hearing-impaired people participate in this national event

"China team, first across the finish line!"

Tencent'S 3D sign language "Leaf" expresses the excitement of witnessing China win the gold medal in sign language after the final of the mixed team short-track speed skating relay on Feb. 5.


Tencent 3D Sign language digital intelligence "Leaf", as CCTV VIDEO AI sign language interpreter, provided sign language interpretation services for the match, witnessing the birth of China's first gold medal in this tournament.

The special people in the silent world can also feel the grand event of ice and snow events held in China, further improving the experience of the hearing-impaired.


Tencent 3D Sign Language Digital Intelligent "Leaf" is jointly built by Tencent Cloud, micro and PCG AI and other technical teams, integrating multi-modal interaction technology, 3D digital human modeling, machine translation, speech recognition and natural language understanding and other technologies, so that the sign language expression ability of "Leaf" is close to real people.

Tencent has independently developed a visual action editing platform, which provides a friendly tool platform for more professional sign language teachers and enables them to refine the full range of sign language movements with high efficiency.

Up to now, Tencent'S 3D sign language "Leaf" covers more than 1.6 million words and sentences, and has made a lot of directional optimization for sports events. The comprehensible level of sign language is over 90%, leading the industry.


In sign language interpretation, based on "national common sign language dictionary", "leaf" first by average hearing people sign language and deaf person ability of machine translation will match commentary content low latency to generate high accurate characterization of sign language, again through the tencent multimodal end-to-end generation model, conduct joint modeling and forecasting to generate high accuracy of movement, expression, lip motion, such as sequence, To achieve a natural professional, easy to understand the effect of sign language.

The appearance of "Leaf" relies on Tencent's leading TECHNOLOGY of 3D illumination scanning restoration, facial muscle drive, facial expression and gesture capture, so that the image of "Leaf" is highly realistic and the movements are more natural and vivid. In terms of information accuracy, "Leaf" can quickly learn current new words and hot words, and can also quickly complete the learning of various industries, business scenarios and relevant knowledge, to improve the accuracy of translation.


In the 14th Five-year Plan for the Protection and Development of Persons with Disabilities, it is proposed to speed up the development of barrier-free information, adapt intelligent services to the needs of persons with disabilities, and facilitate the use of intelligent tools in their daily life.

According to the second national sampling survey of the disabled, there are 27.8 million hearing-impaired people in China, and sign language is the first language for them to communicate with each other and obtain information from the outside world.

However, since the word order of sign language is different from that of Chinese, and facial expressions and mouth movements are needed to assist understanding, if improper omitted words and new hot words cannot be expressed, the intelligability of sign language broadcast will be greatly affected.

AI sign language host sapiens hopes to improve the intelligibility of AI sign language expression by establishing a sound human language system, realistic screen language, coherent natural movements and rapid adaptation of new words and buzzwords.


In the field of AI interactive intelligence, Tencent Cloud Xiaowei constantly promotes the development of technology and the implementation of industrial applications, enabling Shuzhi to become a digital assistant in many industries, assisting humans to provide more efficient and accurate services.

In the field of news media, in October 2021, the state administration of radio in the radio and television and the Internet audio-visual "difference" science and technology development plan is also made clear that, for the first time in sign language is widely used to promote virtual host, animation news broadcast, the weather forecast, variety, science and education programs such as production, innovation program form, improve the system efficiency and intelligent level.

In the future, Tencent's 3D sign language "Leaf" will also provide services in more scenarios to help achieve a better barrier-free information communication environment.

"Leaf" is not the only AI sign language interpreter used at the Wise Winter Olympics. Previously, the first "AI sign language anchor" created by CCTV news and Baidu Intelligent Cloud has also been officially unveiled. "She" looks like a real person, wearing formal clothes and broadcasting news in a kind and natural way.

Such technology, the power of science and technology can be seen how powerful ah!

For our reliable translation & localization service