Autonomous driving is inherently a geometric problem, where the goal is to identify and understand the scene (road agents, context …) and to navigate a vehicle safely and correctly through 3D space. As sensor configurations get more complex, integrating multi-source information from different around view cameras and representing features in a unified view come of vital importance.
The core subject of this PhD is to study and develop innovative concepts, algorithms and methods to enhance situational awareness of autonomous systems in dynamic environments using spatio-temporal Artificial Intelligence. The overall aim of this research will be a vehicle perception system that is capable of detecting, tracking and understanding the entire environment surrounding a car, navigating in complex conditions ranging from dense urban scenarios to country road with varying weather conditions using a Bird’s-eye-view (BEV) end-to-end perception framework. BEV perception inherits several advantages, as representing surrounding scenes in BEV is intuitive and fusion-friendly; and representing objects and/or road elements in BEV is most desirable for subsequent modules as in planning, trajectory forecasting and/or control. The PhD subject will explicitly address the following topics: 3D object detection, semantic segmentation, drivable space, and multi-agent dynamics by predicting multi-hypothesis future instance segmentation and motion in BEV representation.
The main research axes of this thesis can be broken into the following main tasks.
Task 1. Scene understanding: The goal is to develop new camera-based perception algorithms including semantic segmentation and depth maps topics (i.e., 3D objects and lanes detection). Deep learning approaches have shown good results when applied to these topics and will be investigated.
Task 2. Predict long-term situation awareness by considering the future interactions of the dynamic agents in the scene. A probabilistic approach will predict plausible and multi-modal futures of the dynamic environment integrating the others agent’s intentions and the context.
Task 3. Computation complexity: Explore the ways to encode the multi-view image features into a compact latent space. Decoupled it from the input size and output resolution, enabling precise computational budget control.
Academic partner: I3S laboratory at Sophia-Antipolis
A PhD thesis allows you to develop multiple skills to enable you to carry independent research, at the same time we develop the know-how on key technologies. You shall have the opportunity to formulate novel solutions whilst making the most of our prototype platforms. You will need to gain a strong understanding of computer vision and principles of machine learning from the vehicle navigation perspective and gain critical thinking to formulate the problem, propose solutions, and test them.
You should be completing or have Bac+5 level degree, an engineering diploma or a Master 2 in Computer Science, Computer Vision, Robotics or Signal Processing. You are very much interested in the automotive domain and the use of perception systems. It is very important to be curious, willing to learn new techniques. You will have the opportunity to formulate your own ideas, to test them in the Renault prototype vehicles. Experience with image and signal processing, fundamentals of machine learning, programming (C++, Python language), robotics technologies. A working knowledge of the English language is required.
The interested candidate should send a detailed CV, the academic transcripts (Bachelor and Master), a motivation letter, and at least one recommendation letter as a single pdf file, to Guillaume Allibert guillaume.allibert(at)univ-cotedazur.fr
[1] H. Caesar, V. Bankiti, A. Lang, S. Vora, V. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, "nuScenes: A multimodal dataset for autonomous driving," IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11618-11628, 2020.
[2] Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel, "Backpropagation Applied to Handwritten Zip Code Recognition," Neural Computation, pp. 541-551, 1989.
[3] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, "An Image is Worth 16x16 Words: Transformers for Image Recognition," 9th International Conference on Learning Representations, ICLR, 2021.
[4] F. Dellaert and C. E. Thorpe, "Robust car tracking using Kalman filtering and Bayesian templates," Proceedings of SPIE - The International Society for Optical Engineering, vol. 3207, 1997.
[5] H. Boulahbal, A. Voicila and A. Comport, "Instance-Aware Multi-Object Self-Supervision for Monocular Depth Prediction," IEEE Robotics and Automation Letters, pp. 10962-10968, 2022.
[6] H. Boulahbal, A. Voicila and A. Comport, "Forecasting of depth and ego-motion with transformers and self- supervision," IEEE International Conference on Pattern Recognition, pp. 3706-3713, 2022.
[7] H. Boulahbal, A. Voicila and A. Comport, "Are conditional GANs explicitly conditional?," British Machine Vision Conference (BMVC), 2021.
[8] Z. Wu, G. Allibert, F. Meriaudeau, C. Ma and C. Demonceaux, "Hidanet: Rgb-d salient object detection via hierarchical depth awareness," IEEE Transactions on Image Processing, pp. 2160-2173, 2023.
[9] C.O. Artizzu, G. Allibert and C. Demonceaux, "OMNI-CONV: Generalization of the Omnidirectional Distortion-Aware Convolutions," Journal of Imaging, vol. 9, no. 2, 2023.
[10] C.O. Artizzu, G. Allibert and C. Demonceaux, "Deep Reinforcement Learning with Omnidirectional Images: Application to UAV Navigation in Forests," International Conference on Control, Automation, Robotics and Vision (ICARCV), 2022.
[11] W. Schwarting, J. Alonso-Mora and D. Rus, "Planning and decision-making for autonomous vehicles," Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, pp. 187-210, 2018.
[12] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1627-1645, 2010.
[13] P. Lenz, J. Ziegler, A. Geiger, and M. Roser, "Sparse scene flow segmentation for moving object detection in urban environments," IEEE Intelligent Vehicles Symposium (IV), pp. 926-932, 2011.
[14] C. Chen, A. Seff, A. Kornhauser and J.Xiao, "DeepDriving: learning affordance for direct perception in autonomous driving," IEEE International Conference on Computer Vision (ICCV), pp. 2722-2730, 2015.
(c) GdR IASIS - CNRS - 2024.