Robust and Generalizable Sign Language Translation
RoGSiLT aims to advance the translation technologies for German Sign Language (DGS) and French Sign Language (LSF) by overcoming current limitations such as reliance on scarce gloss-annotated data, poor generalization, and unnatural translations. Leveraging a collaboration that integrates expertise in neural machine translation (NMT), speech processing, and computer vision (CV), our approach introduces innovative strategies using self-supervised learning, multimodal neural architectures, and large language models (LLMs). These methods aim to enhance the robustness and generalizability of sign language translation (SLT) systems. We anticipate significant societal impacts, including enhanced accessibility and inclusivity for the Deaf and Hard-of-Hearing (DHH) community, by providing more natural and effective communication tools.
Home Page: https://project.inria.fr/rogsilt/
Announcement on DFKI site: https://www-live.dfki.de/en/web/news/dfki-and-inria-develop-robust-ai-for-sign-language-translation
Funding: BMFTR
Duration: 1.4.2026 – 31.03.2029
Partners and People:
- German Research Center for Artificial Intelligence (DFKI)
- SCAAI group, Cognitive Assistants (COS) department, Saarbrücken & Berlin, Germany – Dr. Eleftherios Avramidis, Dr. Fabrizio Nunnari
- Multilinguality and Language Technology department, Saarbrücken, Germany – Prof. Josef van Genabith, Yasser Hamidullah
- French National Institute for Research in Digital Science and Technology (INRIA)
- Multispeech group – Prof. Slim Ouni (Université de Lorraine – Inria), Dr. Mostafa Sadeghi (Inria), Dr. Sam Bigeard (Inria)