Vous êtes ici : Kiosque » Annonce

17 novembre 2022

Catégorie : Post-doctorant

This contract is about the research activity of the IRIMAS institute in the field of integration between data fusion and neural networks. The pre-fusion (data fusion before feeding the network) is the center of this post-doctoral contact. The tasks will be :

1 – State of the art of architectures of pre-fusion, cross-fusion and post-fusion, particularly those involving evidential theory and belief functions

2 – Study on the propagation of belief functions and particularly how the ignorance is considered in each layer of different kinds of neural networks (MLP, CNN, Vision Transformers, etc.)

3 – Implementation of the identified pre-fusion scheme within a deep learning architecture

4 – Simulation results and comparison between the different proposed architecture (pre/cross/post fusion) – the hypothesis to be verified is that pre-fusion is a) feasible and b) more performant than cross/post-fusion (regarding evaluation metrics such as F1 score and error rate)

5 – Quantitative comparison of several fusion operators (DS, PCR6/6+, etc.) including performance metrics but also computational cost

(6) – Real-time implementation to the lab prototype may be investigated if desired

Deadline for application: 01/01/23

Thomas Josso-Laurain (thomas.josso-laurain@uha.fr)

**Context**: The ANR EviDeep project focuses on autonomous vehicles, particularly the perception step whose goal is to locate the vehicle and identify potential obstacles that are surrounding the vehicle. The environment perception can be divided into two tasks: detecting the areas of interest and analysing the perceived scene. The target application of the project mainly concerns the analysis and the interpretation of the scene.

With the increase of the number of sensors of different nature (RGB camera, light-field camera, 2D and 3D lidar, radar, etc.), a fusion of these heterogeneous information needs to be done in order to take advantage of each source. This fusion can be realized by several approaches: probabilistic fusion [1] or evidential fusion [2].

With the rising of high computational power and artificial intelligence, various data-based approaches have been developed to perform perception tasks [3], often based on a mono source (RGB camera). On one side, some works integrate the evidential theory inside mono-sensor networks [4] or multi-sensors networks. On the other side, other works directly fuse the information inside the neural network according to different architectures [5].

The proposed architectures are respectively pre-fusion, cross-fusion and post-fusion, depending on where the fusion stage is located in the neural network pipeline. If the prototypes-based approach allows evidential post-fusion [6], the pre-fusion architecture is still a strong scientific lock since the propagation of belief functions inside a neural network (MLP, CNN, Transformers and so on) is not mastered. The post-doctoral researcher will work on this topic to tackle the pre-fusion problem.

**Objectives**:

The hired post-doctoral fellow will focus on the pre-fusion case, particularly how to integrate an evidential fusion before feeding a neural network, by following the steps:

1 – State of the art of architectures of pre-fusion, cross-fusion and post-fusion, particularly those involving evidential theory and belief functions

2 – Study on the propagation of belief functions and particularly how the ignorance is considered in each layer of different kinds of neural networks (MLP, CNN, Vision Transformers, etc.)

3 – Implementation of the identified pre-fusion scheme within a deep learning architecture

4 – Simulation results and comparison between the different proposed architecture (pre/cross/post fusion) – the hypothesis to be verified is that pre-fusion is a) feasible and b) more performant than cross/post-fusion (regarding evaluation metrics such as F1 score and error rate)

5 – Quantitative comparison of several fusion operators (DS, PCR6/6+, etc.) including performance metrics but also computational cost

(6) – Real-time implementation to the lab prototype may be investigated if desired

The obtained results by the post-doctoral researcher will be published in international conferences and impacted journals (Information Fusion, Neurocomputing…). In addition, the post-doctoral researcher will be part of the MIAM team in the ASI dept of the IRIMAS lab. He/she may also propose research projects to the engineering students of the ENSISA college of engineering.

**Prerequisites**:

The candidate must have a PhD degree in Computer Science/Automatic Control with application of neural networks or data fusion. The candidate should demonstrate experience in autonomous driving/perception/data fusion/machine learning/deep learning. Good programming skills are expected. Knowledge about evidential theory and its applications will be benefit.

**Work conditions**:

The whole activity will take place at IRIMAS, in the ASI dept, more precisely in the MIAM team at UHA, Mulhouse, France. The access to autonomous vehicle prototype will be given during the contract. This contract is founded by the ANR JCJC EviDeep.

**Contract**: Post-doctoral contract

**Date of start**: 01/03/2023

**Duration**: 12 months

**Team**: Dr. Thomas Josso-Laurain, Dr. Maxime Devanne, Pr. Jean-Philippe Lauffenburger

**Application**: If interested, please send CV + cover letter + publications list (top 5) to thomas.josso-laurain@uha.fr before 01/01/23. The application interviews will be conducted during the month of January to ensure a start for the beginning of March 2023.

**References**:

[1] J. Dezert, A. Tchamova, and D. Han, “Total Belief Theorem and Generalized Bayes’ Theorem,” in *IEEE International Conference on Information Fusion (FUSION)*, 2018, pp. 1040–1047. doi: 10.23919/ICIF.2018.8455351.

[2] H. Laghmara, T. Laurain, C. Cudel, and J.-P. Lauffenburger, “Heterogeneous Sensor Data Fusion for Multiple Object Association using Belief Functions,” *Information Fusion*, vol. 57, pp. 44–58, 2020.

[3] C.-H. Cheng, C.-H. Huang, T. Brunner, and V. Hashemi, “Towards Safety Verification of Direct Perception Neural Networks,” *arXiv:1904.04706 [cs]*, Nov. 2019.

[4] E. Capellier, F. Davoine, V. Cherfaoui, and Y. Li, “Evidential deep learning for arbitrary LIDAR object classification in the context of autonomous driving,” in *2019 IEEE Intelligent Vehicles Symposium (IV)*, Jun. 2019, pp. 1304–1311. doi: 10.1109/IVS.2019.8813846.

[5] L. Caltagirone, M. Bellone, L. Svensson, and M. Wahde, “LIDAR–camera fusion for road detection using fully convolutional neural networks,” *Robotics and Autonomous Systems*, vol. 111, pp. 125–131, Jan. 2019, doi: 10.1016/j.robot.2018.11.002.

[6] Z. Tong, P. Xu, and T. Denœux, “An evidential classifier based on Dempster-Shafer theory and deep learning,” *Neurocomputing*, vol. 450, pp. 275–293, Aug. 2021, doi: 10.1016/j.neucom.2021.03.066.

(c) GdR 720 ISIS - CNRS - 2011-2022.