Pansharpening is a fundamental and crucial task in remote sensing which generates a high-resolution hyperspectral image by fusing a low-resolution hyperspectral image and a high-resolution panchromatic (or multispectral) image. A range of methods formulate pansharpening as an inverse problem from which iterative algorithms are derived. Those methods depend on hand-crafted regularization functions designed to describe the feasible set of natural hyperspectral images. Built upon such model based iterative algorithms, Unrolled Neural Network are hybrid architecture. In contrast with classical deep learning approaches, they demand significantly less training data and provide an improved interpretability.
After a comprehensive study of the proposed iterative algorithms in pansharpening, the student will design an Unrolled Neural Network based on a specific algorithm derived from a total variation prior.
Position can be started anytime from February 2025 for a duration up to 6 months. This internship will be hosted at the LISTIC laboratory in Annecy, with regular meetings and exchanges with researchers from the project. Interested students should send a detailed CV and short cover letter to Yassine MHIRI (yassine.mhiri@univ-smb.fr), Argheesh Bhanot (argheesh.bhanot@univ-smb.fr) and Ammar Mian (ammar.mian@univ-smb.fr).
More information are available at https://y-mhiri.github.io/stages/2024-2025_sujet_stage.pdf
Design of an Unrolled Neural Network for Hyperspectral Pansharpening.
Hyperspectral imaging, pansharpening, inverse problem, convex optimization, deep learning, unrolled neural network
Annecy, France.
Pansharpening is a fundamental and crucial task in remote sensing which generates a high-resolution hyperspectral image by fusing a low-resolution hyperspectral image and a high-resolution panchromatic (or multispectral) image. A range of methods formulate pansharpening as a convex optimization problem derived from physical observation model [1,2]. To do so, hand-crafted regularization functions are designed to describe the feasible set of natural hyperspectral images. Subsequently, iterative algorithms are implemented to restore a high-resolution hyperspectral image.
The Total Variation (TV) [3, Chap1, Chap2], one of the most popular regularizers in image processing, assumes that natural images are made of a few objects, resulting in sparse spatial gradients. This approach have been applied successfully to hyperspectral images but without including the spectral correlation between the observed bands.
More recently, deep learning approaches have been proposed for the pansharpening task [4]. Nevertheless, they demand a large amount of training data and resort to high computational cost for the training. Moreover, they often lack interpretability, crucial to scale to scientific applications. In contrast, Unrolled neural network are hybrid architecture derived from model based iterative algorithms. They provide a powerful architecture that demands significantly less training data and provides an improved interpretability [5].
The proposed work aims at designing an unrolled neural network architecture based on a specific pansharpening algorithm based on a regularization derived from the total variation. After a comprehensive study of the proposed iterative algorithm, the student will design an unrolled neural network based on the aforementioned methodology. Finally, a benchmark of various pansharpening methods will be conducted.
He/she should be enrolled in a M2 or engineer diploma in one or more of the following fields: applied mathematics, signal and image processing, computer science. The candidate should have good writing and oral communication skills.
Position can be started anytime from February, 2025 and duration is up to 6 months. The candidate will be based in Annecy. This internship will be hosted in the LISTIC laboratory, with regular meetings and exchanges with researchers from the project.
Send a detailed CV and motivation letter to yassine.mhiri@univ-smb.fr, argheesh.bhanot@univ-smb.fr, ammar.mian@univ-smb.fr.
[1] Ballester, C., Caselles, V., Igual, L., Verdera, J., & Rougé, B. (2006). A variational model for P+ XS image fusion. International Journal of Computer Vision, 69, 43-58.
[2] Loncan. L et al. "Hyperspectral Pansharpening : a Review". In: IEEE Geoscience and Remote Sensing Magazine.
[3] Abergel, R. (2016). Quelques modèles mathématiques et algorithmes rapides pour le traitement d’images (Doctoral dissertation, Université Sorbonne Paris Cité).
[4] Ciotola, M., Guarino, G., Vivone, G., Poggi, G., Chanussot, J., Plaza, A., & Scarpa, G. (2024). Hyperspectral Pansharpening: Critical Review, Tools and Future Perspectives. arXiv preprint arXiv:2407.01355.
[5] Monga, V., Li, Y., & Eldar, Y. C. (2021). Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine, 38(2), 18-44.
(c) GdR IASIS - CNRS - 2024.