Background:

This PhD thesis will be realized in the IETR laboratory, specialized in the domain of electronic and digital technologies, within the VAADER team located at INSA Rennes, a well-known French engineering school (Grande École d'Ingénieurs), The thesis is co-financed by the DIGISPORT project and INSA Rennes. The objective of DIGISPORT (Digital Sport Sciences) is to create a unique international graduate school in the field of training and research in sport and digital sciences, computer sciences, data sciences, electronics, human and social sciences in a transversal approach to meet the needs for new skills generated by the entry of sport into the digital age. DIGISPORT relies on internationally renowned research laboratories from Rennes which are affiliated to the CNRS joint laboratories (IRISA, IETR, IRMAR, CREST), the INRIA Rennes laboratory, the network of Rennes Grandes Ecoles (ENS Rennes, INSA Rennes, CentraleSupelec, ENSAI) and the Universities of Rennes and Rennes 2. During this thesis, close collaboration with internationally renowned researchers in INRIA Rennes is envisioned.

Thesis description:

Human motion rendering through motion transfer [1,2,3] can have several applications for sports training. By generating a learned model of a virtual athlete, with combination of characteristics from different existing or virtual athletes, one can combine the morphology of one athlete with the motion of a second athlete and the motion style of a third athlete, and generate the corresponding video. This combination of characteristics from different athletes in the model can be used to train with un-typical competitors, not yet present in the game. It can also allow to model the style of a specific existing (famous) athlete from existing videos of this athlete, and then generate (“predict”) some unseen motions of this athlete keeping his appearance and his motion style. Alternatively, a motion sequence performed by the famous athlete could be transferred to the training athlete, but keeping the training athlete style, in order to visualize the same action but with its own physiology.

This thesis aims to create video animations of humans/athletes following a source motion or performing a generated novel motion for the purpose of improving sports training. Deep generative neural models now enable the synthesis of photo-realistic images and videos. In the context of sport motion modeling and rendering, the goal of this thesis is to leverage on the ability of recent advances in generative neural models and learning-based image/video rendering to produce virtual motion videos of an athlete with photorealistic quality. The usual approach to render sports motion is to extract an articulated 3D shape and motion model of the athlete, based on specific multi-sensors 3D motion capture systems, and then to generate images and videos through 3D model rendering techniques applied to the extracted textured 3D model. Complex lightning and material physical models make it hard to reach a high level of rendering quality targeting photorealism. This project aims to explore an alternative approach to this task, by exploiting the recent advances in generative neural models. The goal is to produce realistic videos of sport gestures directly in the image domain, without the need to build and render an intermediate complex 3D model. The benefit of such an approach is to avoid the use of a complex acquisition setting, and to take as input sets of more easily available monocular videos.

This thesis will be interesting for you if:
- You want to contribute to the advances in promising domains as generative AI and computer vision.
- You are passionate in sports or/and sports applications in a data-driven age.
- You like academic research environment in an interdisciplinary context.

The plus of this thesis:
- International research mobility in Sweden is envisioned during the thesis.

Expected qualifications:
- Master degree or equivalent in computer science, applied mathematics, signal/image processing, and so on.
- Solid background in computer vision and deep learning. Previous experience on deep generative neural models is a plus.
- Good programming skills.
- Technical English or/and French practice. English writing is a must.
- Having previous scientific publications (conference, journal) is a plus.

Starting date: September-October 2024.

Applications: Please send resume, application letter and one or several recommendation letters to
- Thesis director: Luce Morin, IETR, Vaader Team luce.morin@insa-rennes.fr
- Thesis supervisor: Xiaoran Jiang, IETR, Vaader Team xiaoran.jiang@insa-rennes.fr

References
[1] Caroline Chan, Shiry Ginosar, Tinghui Zhou, Alexei A. Efros , Everybody Dance Now, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
[2] Natalia Neverova, Riza Alp Guler, Iasonas Kokkinos “Dense Pose Transfer “, European Conference on Computer Vision (ECCV), 2018.
[3] Yang-Tian Sun, Hao-Zhi Huang, Xuan Wang, Yu-Kun Lai, Wei Liu, Lin Gao , Robust Pose Transfer with Dynamic Details using Neural Video Rendering, IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE TPAMI), 2022.