Context and problematic

In this work, we propose to focus on a well-known image processing estimation problem referred to as image registration. Given a set of 2D or 3D moving images observed by an imaging sensor, the aim of image registration is to determine the geometric transformation allowing us to adjust these images to a reference (2D or 3D) image in order to exploit diverse information of a scene. This problem of registration is classically encountered in a plethory of space applications typically when optical remote sensing systems detect Earth information. We can cite the example of a SAR satellite observing a 3D Earth image under different angles, e.g., for oceanography [1]. In this application, each sensed image must be aligned to the true image. In aerospace defence applications, in the context of target tracking and facial recognition, a video sequence containing the target of interest needs to be adjusted to another sequence of reference [2].

Different state-of-the-art techniques exist to solve the image registration problem [3]. These techniques are classically based on the minimization of a cost function between the reference image and the transformed observed image. This minimization is performed by an optimization algorithm. However, these conventional methods can have some limits. Indeed, it can be difficult to optimize the cost function because it is highly non-convex and can lead to several solutions [4]. Furthermore, in the context of 3D/2D registration, the model linking both the reference image and the observed image is generally ill-posed because the depth of the 3D image is not observable [5]. To overcome these analytical problems, deep learning and machine learning-based techniques have recently appeared, [6] mainly based on support vector machines (SVM) [7] and neural networks (NN) [8] substituting analytical models by parametric regression models. Regression models are learned thanks to a dataset containing both the pixel information of an arbitrary image to adjust and the transformation relative to the reference image (that can also contain scale and depth parameters). In a second step, from a test image, we are able to predict the new transformation from this image to the reference. Starting from this assumption, two fundamental points have to be considered:

1. the geometric transformation between the test image and the reference has to be constrained by its geometrical properties. Indeed, it is basically an affine transformation containing both rotation and translation between the two images with potential scale parameter. Learning this type of transformation requires to make sure that the properties of the transformation space are respected. In a mathematical point of view, this transformation is constrained to lie on the Lie group SE(2) or SE(3). When the scale parameter is also unknown, then the transformation belongs to the Lie group Sim(3) (space of similitude in dimension 3).
2. Parametric models used in machine learning need to adjust hyperparameters, which is classically a hard challenge typically for SVM. Furthermore, one of the main drawback is that this kind of approach (for SVM or NN) do not provide uncertainty on the estimation of the transformation, which is a crucial information in an operational context.

We propose in this work to solve the image registration problem using Lie groups. Mathematically speaking, the groupe of transformation used for image registration is a space equipped with a Riemannian structure but also with a group structure [9]. Its advantage is to leverage mathematical tools (especially statistics and optimization) allowing us to overcome non-linearities induced by the parameters of the transformation (classically Euler angles). Machine learning methods have already been adapted to Lie groups in a supervised framework for estimation and filtering [10-11-12]. However, in the context of supervised classification and prediction, works are almost non-existent and everything remains to be done. In addition, an attention has to be paid to machine learning methods such as SVM, NN and Gaussian process regression [13-14]. Indeed, these powerful non-parametric prediction tools can be used to learn the correlation between measurements and models [15]. They are also able to learn model hyperparameter and provide uncertainty on the prediction. The challenge of this work is to combine the Lie group formalism with machine learning algorithms (SVM, NN or Gaussian processes) in order to design a new accurate, robust and efficient registration method.

Objectives

The main objectives of this post-doctorate is to develop new machine learning methods adapted to data living in Lie groups. More precisely:

1. A first step is to study the regression problem on Lie groups and define a non-parametric regression model on Lie groups. We think that Gaussian processes are tools adapted to this problem.
2. The second step is to compute both the likelihood and the predictive distribution of the regression model in order to predict the model output and estimate its parameters.
3. The proposed strategy will be numerically tested on simulated image data in a 3D/2D image registration context.

Informations

Disciplines: applied mathematics, machine learning, statistical estimation.

Required curriculum: PhD. in computer science, statistics or machine learning.

Start and duration: as soon as possible for 1 year (can be extended).

Salary: 2900 euros brut by month.

Applications (resume, transcript) and informal inquiries are to be e-mailed to Samy Labsir, (IPSA/TéSA), samy.labsir@ipsa.fr, Jean-Yves Tourneret (IRIT/ENSEEIHT/TéSA), jean-yves.tourneret@toulouse-inp.fr, Julien Lesouple (ENAC), julien.lesouple@enac.fr.

Location: TéSA laboratory, Toulouse, France.

References

[1] Sommervold, O. and Gazzea, M. and Arghandeh, R., A Survey on SAR and Optical Satellite Image Registration, Remote Sensing, 15, 2023.

[2] Curiale, A. and Vegas Sánchez-Ferrero, G. and Aja-Fernández, S., Techniques for tracking: image registration, 2017.

[3] Chen, M. and Tustison, N. J and Jena, R. and Gee, Image Registration: Fundamentals and Recent Advances Based on Deep Learning, Springer, 2023.

[4] P. Markelj and D. Tomaževič and B. Likar and F. Pernuš, A review of 3D/2} registration methods for image-guided interventions, Medical Image Analysis, 2012.

[5] Unberath, M. and Gao, C. and Hu, Y. and Judish, M. and Taylor R. and Grupp, R., The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective,Frontiers in Robotics and AI.

[6] Xiaohuan C. and Jingfan F. and Pei D. and Sahar A. and Pew-Thian Y. and Dinggang, Handbook of Medical Image Computing and Computer Assisted Intervention, Chapter 14 - Image registration using machine and deep learning, Academic press, 2020.

[7] Dai Qiang P. and Ding Xue W. and Jin Wen T., in Proc. 8th International Conference on Pattern Recognition (ICPR'06), A New Efficient SVM-based Image Registration Method, 2006.

[8] H. R. Boveiri and R. Khayami and R. Javidan and A. Mehdizadeh, Medical image registration using deep neural networks: A comprehensive review,Computers and Electrical Engineering, 2020.

[9] J. Faraut, Analysis on Lie Groups: An Introduction, Cambridge University Press, 2008.

[10] S. Bonnabel and A. Barrau, An intrinsic Cramér-Rao bound on Lie groups, International Conference on Networked Geometric Science of Information, 2015.

[11] J. Ćesić and I. Marković and M. Bukal and I. Petrović, Extended information filter on matrix Lie groups, Automatica, 2017.

[12] G. Bourmaud, Online Variational Bayesian Motion Averaging, ECCV, 2016.

[13] E. Schulz and M. Speekenbrink and A. Krause, A tutorial on Gaussian process regression: Modelling, exploring, and exploiting functions,Journal of Mathematical Psychology, 2018.

[14] J. Wang, An Intuitive Tutorial to Gaussian Process Regression, Computing in Science and amp. Engineering, 2023.

[15] C. Palmier and A. Giremus and P. Minvielle, P. and C. Vacar and G. Bourmaud, Terrain-Aided SLAM with Limited-Size Reference Maps Using Gaussian Processes, 2023.