Context :

This thesis is part of the INARI project (multimodal vIsion for Robust Navigation and control of Autonomous vehicle) supported by the ANR (National Research Agency). A multi-partner project (LITIS Rouen, CEREMA, HEUDIASYC Compiègne, COSYS Paris, IRIMAS Mulhouse), INARI aims to develop a system capable of both detecting road obstacles in adverse weather conditions and establishing the appropriate control taking into account the road grip according to its condition.

For the analysis of road scenes, the importance of multimodality (exploitation of different sources of information) is well established. Multi-sensor fusion allows for an extended and robust perception, and therefore an interpretation of objects, static or dynamic, composing the nearby environment. However, depending on external conditions, the information provided by the sensors may not be consistent, or even be contradictory. Systematically merging all information from all sensors used in the acquisition system is therefore not optimal (combinatorial complexity, consideration of inappropriate measurements, etc.).

It is therefore necessary to selectively merge the data by quantifying the imperfection of the sensors/measurements according to external conditions in order to integrate them into the fusion process, which is the core of this thesis. The Dempster-Shafer theory [1, 2] proposes mechanisms particularly suited for the fusion of imperfect or conflicting modalities and allows for a quantitative evaluation of uncertainty [3, 4]. In [5, 6, 9], evidence theory merges information from heterogeneous sources or different agents to enable safe autonomous driving of intelligent vehicles. Finally, coupled with deep neural networks, evidence theory can yield interesting results in applications such as detection, classification, or semantic segmentation [7, 8, 10]. This latter approach, particularly promising, seems appropriate for the objectives of this thesis

Objectives:

This thesis focuses on the development of strategies for automatic selection of data to be fused based on meteorological conditions to ensure high-quality local perception.

In the first part, the doctoral candidate will conduct a state-of-the-art review focusing particularly, but not exclusively, on recent approaches to adaptive multimodal fusion (convolutional neural networks, etc.). This review will also aim to explore approaches that facilitate the consideration of varying imperfections (uncertainty, reliability, etc.) of sensors such as belief functions or evidential networks.

Subsequently, the thesis will concentrate on the automatic estimation of a parameter of imprecision, and even reliability, of the sensors. The goal is to establish a link between external perception conditions and the sensors' perception capabilities. This understanding will then enable the development of fusion strategies for modalities guided by the estimation of external conditions for obstacle detection in the scene.

In a third phase, a comprehensive local map of detected objects/obstacles in degraded conditions and their state (static/dynamic) will be created. This map will be essential for establishing the appropriate control of the vehicle.

These developments will undergo an experimental validation phase in real conditions using testing facilities (autonomous vehicles, etc.) available at the project partners' sites.

Key words: Autonomous vehicles, multimodal perception, adaptive data fusion, Belief functions

Required skills :

• ·Bachelor's or Master's degree in automation, computer science.
• ·Strong skills in computer science (deep learning) and/or signal processing, software development (C/C++, Python, Linux), and programming and simulation environments (Matlab, ROS, etc.).
• ·Knowledge in data fusion would be appreciated.
• ·Very good written and oral communication skills in French and English.

Application : Before June 15th, 2024 by sending an e-mail to jean-philippe.lauffenburger@uha.fr, remi.boutteau@univ-rouen.fr and hind.laghmara@insa-rouen.fr :

1.Cover letter

2.CV including two academic references

3.Transcripts of Master’s/engineering training

4.Recommandation Letter

References.

[1] A. P. Dempster. “A Generalization of Bayesian Inference”. In: Journal of the Royal Statistical Society. Series B(Methodological) 30.2 (1968), pp. 205–247. issn: 00359246. url: http://www.jstor.org/stable/2984504.

[2] G. Shafer. A mathematical theory of evidence. Princeton, NJ, USA: Princeton University Press, 1976

58. doi: doi.org/10.1016/j. inffus.2019.11.002.

[3] Dezert, Jean et al. "Measure of Information Content of Basic Belief Assignments", Int. Conference on Belief Functions, 2022.

[4] Dezert, Jean. "An Effective Measure of Uncertainty of Basic Belief Assignments" 25th International Conference on Information Fusion, 2022.

[5] Hind Laghmara, Thomas Laurain, Christophe Cudel, and Jean-Philippe Lauffenburger. “Heterogeneous sensor data fusion for multiple object association using belief functions”. In: Information FUSION 57, pp. 44––58. doi: doi.org/10.1016/j. inffus.2019.11.002.

[6] H Laghmara, T Laurain, C Cudel, JP Lauffenburger, “2.5 D evidential grids for dynamic object detection”, Int. Conference on Information Fusion, 2019

[7] Tong, Zheng, Philippe Xu, and Thierry Denoeux. "An evidential classifier based on Dempster-Shafer theory and deep learning" N