Animating 3D avatars from multi-point videos and its application to sign languages

Thesis proposal for students at the Saarland University, Saarbrücken, Germany.

Updated: 14.01.2026

Proposed by the SCAAI Group (Social, Cognitive and Affective AI, https://scaai.dfki.de) at the German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany

## Introduction/Problem

One of the most effective approaches for translating text into sign language is through the animation of signing 3D avatars.

To train translation models capable of animating 3D avatars, large-scale 3D animation corpora are required.

Capturing accurate motion data of the human body, face, and hands remains a challenging task, requiring specialized hardware and operators. This challenge is even greater for sign language, which involves rapid movements, self-occlusions, and highly expressive facial gestures.

Methods exist for estimating 3D human motion from affordable RGB cameras (Pavlakos et al., 2019), but they typically rely on single viewpoints and require extensive post-processing for dynamic motion. Recently, methods for synthesizing full 3D human motion from multi-view video data have emerged (e.g., Junkawitsch et al., 2025). However, their performance in small-scale studios and with limited computational resources remains unexplored.

## Goals

The goals of this thesis are to:

• Investigate motion capture techniques for the human body, hands, and face using affordable RGB cameras.
• Enhance capture quality through the integration of multiple viewpoints.
• Evaluate the applicability of such methods to sign language data.

In addition to the written manuscript, the thesis must produce a demonstrable working system, released as open-source software in a public repository.

Satisfactory results may lead to a scientific publication.

As initial material, the candidate may use the publicly available DGS-Fabeln-1 corpus: a German Sign Language dataset of fairy tales captured simultaneously from seven viewpoints (Nunnari et al., 2024).

## Requirements

• Passion for 3D graphics and animation.
• Strong Python programming skills.
• Software engineering skills for organizing big code repositories into modules, classes, and scripts (not only plain sequential notebooks).
• Use of the Blender 3D editor and APIs (https://www.blender.org).
• Proactive and propositive attitude.
• Knowledge of any sign language is not required but would be considered as a plus.
• The thesis will follow the guidelines of the UMTL department at the University of Saarland: https://umtl.cs.uni-saarland.de/teaching/thesis.html

## Contacts

When interested, send a CV, transcript of your grades, and possibly links to your selected existing open-source software repositories to Fabrizio Nunnari <fabrizio.nunnari@dfki.de>

## References

Junkawitsch, H. et al. (2025) “EVA: Expressive Virtual Avatars from Multi-view Videos,” in Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers. SIGGRAPH Conference Papers ’25: Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, Vancouver BC Canada: ACM, pp. 1–11. Available at: https://doi.org/10.1145/3721238.3730677.

Nunnari, F. et al. (2024) “DGS-Fabeln-1: A Multi-Angle Parallel Corpus of Fairy Tales between German Sign Language and German Text,” in Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Torino, Italy: ELRA and ICCL, pp. 4847–4857. Available at: https://aclanthology.org/2024.lrec-main.434.

Pavlakos, G. et al. (2019) “Expressive Body Capture: 3D Hands, Face, and Body from a Single Image,” in Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 10975–10985.