Vous êtes ici : Kiosque » Annonce


Mot de passe : 

Mot de passe oublié ?
Détails d'identification oubliés ?


16 mai 2022

PhD dissertation proposal (CIFRE): Learning-based Object Detection and Registration for Augmented Reality Integrated Manufacturing.

Catégorie : Doctorant

We are proposing a PhD with the MOM packaging company (CIFRE) starting in September 2022

Title: Learning-based Object Detection and Registration for Augmented Reality Integrated Manufacturing.

Keywords: Deep Learning, Object Registration, Augmented Reality, Assembly line, Digital Twin, Human-Robot Collaboration.


PhD Dissertation Proposal


Title: Learning-based Object Detection and Registration for Augmented Reality Integrated Manufacturing.

Keywords: Deep Learning, Object Registration, Augmented Reality, Assembly line, Digital Twin, Human-Robot Collaboration.


In the context of the industry 4.0 and digital factory, digital twins, virtual reality and augmented reality are key technologies to design, simulate and optimize cyber-physical production systems (CPPS). They also allow to interact with the CPPS, whether directly, remotely or collaboratively (Uhlemann, Lehmann and Steinhilper, 2017; Vincent Havard et al., 2019; Ke et al., 2019; Sjarov et al., 2020). These technologies open up new possibilities in the design, study or implementation of industrial systems and workstations integrating Industry 4.0 components such as cobots and requiring interaction with the physical system and the virtual system. Indeed, with the evolution of robotic systems and artificial intelligence, humans and robots can collaborate in the same workspace (Rückert, Wohlfromm and Tracht, 2018; Vincent Havard et al., 2019; Pérez et al., 2019).

Human-machine interactions between the operator and the industrial system can be based on augmented reality devices. They are interfaced with the digital twin and allows to visualize information or instructions on the operations to perform, such as assembly or maintenance tasks (Rabah et al., 2018; Gattullo et al., 2019; V Havard et al., 2019; de Souza Cardoso, Mariano and Zorzal, 2020; Lai et al., 2020; Ojer et al., 2020; Havard et al., 2021). These AR devices can also be used to inform the operator about the robots' intentions. Finally, these technologies are also useful for training operators in these professional situations (Matsas and Vosniakos, 2017; Fratczak et al., 2019; Vincent Havard et al., 2019; Badets et al., 2020).

To support its research and training activities, LINEACT CESI has Factory of the Future demonstrators including a metallic additive manufacturing unit, a flexible production system, manual and cobots workstations, mobile robots and robotic arms and prototyping or machining equipment. A first digital twin associated with human machine interfaces based on virtual or augmented reality has been initiated and allows, for example, to perform simulations, in an immersive environment, of assembly operations on manual workstations integrating a cobot station. These physical and digital twins will be exploited within the framework of the development of the proofs of concept of this thesis.

MOM packaging is a company which designs, builds and produces filling machines for many types of products (liquid, powder, …) and containers. As each machine is a unique, MOM needs to keep their know-how. As augmented Reality (AR) is a key technology to train people to assemble complex systems in a reduced amount of time (Havard et al., 2021), MOM packaging is aiming at using this technology to train its employees to assemble its machine as well as its consumers for remote support.

Still, spare parts for industrial machine introduced many challenges. Parts can be symmetric or confused due their size or appearance. Variants of techniques used for AR in domestic environments must be proposed.

Research Topic

Augmented reality relies on adding virtual data to real object in order to give contextualized information to the user. The association of real and virtual data can be performed at different levels. Still, feature matching is the most popular approaches. Features from the real objects in the scene and a reference one are matched in order to register the virtual model in the real scene. Such method is prone to many challenges (Zeng et al., 2017; Xiang et al., 2018):

Most of the literature focuses on standalone objects. Assembly lines and Industry 4.0 brings extra challenges defined as follows (Li et al., 2021):

Expert-engineered registration methods have shown to be outperformed by learning-based approaches. Learning-based approaches requires sufficient data to perform well. Digital twin provides a powerful tool to generate training data to deploy learning-based solution. Moreover, the variety of parts of assembly lines may requires an even larger dataset. Tackling registration and the different viewpoints may even burden the data collection task.

This thesis aims at tackling the challenges specific to the industries 4.0.

Contributions on the following topics are expected:

The PhD thesis major steps can be described as follows:

1.State of the Art of AR for assembly line.

2.Usage of Industry 4.0 platforms and AR tools available at CESI and MOM packaging

3.Contribution in Sim2sim 6 DoF spare part detection and assembly registration in the context of MOM packaging assembly lines.

4.Contribution in Sim2real 6 DoF spare part detection and assembly registration with the help of digital twins in the context of MOM packaging assembly lines.

The expected deliverables are:

This work is in line with the research themes of the LINEACT CESI laboratory and MOM Packaging R&D roadmap. LINEACT CESI is organized around two interdisciplinary scientific themes and two application areas. The themes "Learning and Innovation" and "Engineering and Digital Tools" develop and cross their research in the application areas of the Industry of the Future and the City of the Future.

Within the framework of this research project, the PhD student will be able to rely on the laboratory's expertise in the architecture conception, modeling, development and evaluation of augmented environments or dynamic and collaborative virtual environments applied in the field of Industry 5.0. He will also be able to rely on the laboratory's skills and work on industrial cyber-physical systems and more specifically on digital twins, their architectures and Artificial Intelligence integrated decision system.


Laboratory: LINEACT (Digital Innovation Laboratory for Companies and Apprenticeships for the Competitiveness of Territories), https://lineact.cesi.fr/en/

Company: MOM packaging, https://www.mom-packaging.com/

Thesis Director: Yohan DUPUIS (ydupuis@cesi.fr), Research Director CESI – LINEACT Lab.

Thesis Advisor: Vincent HAVARD (vhavard@cesi.fr), Researcher-Lecturer CESI –LINEACT Lab.

Company Advisor: Louis DERANGÈRE, MOM Packaging CEO.

Funding: CIFRE PhD grant.

Workplace: MOM packaging, Villepinte and CESI Nanterre Campus, Nanterre.

Start date: December 2021.

Duration: 3 years.

Hiring procedure: Application files and interview.

Your application must be sent to Yohan DUPUIS (ydupuis@cesi.fr) and Vincent HAVARD (vhavard@cesi.fr) :

« [Application] PhD Thesis IA for AR Integrated Manufacturing »

Your application files must include:


Please send the document as a zip file entitled « LASTNAME_Firstname.zip »


Applicant skills:

Hard Skills:


Soft skills:


Badets, A. et al. (2020) ‘Using collaborative VR technology for Lean Manufacturing Training: a case study’, in Proc. VRIC. VRIC ConVRgence 2020: 22nd Virtual Reality International Conference - Laval Virtual, Laval, France, pp. 118–127.

Fratczak, P. et al. (2019) ‘Understanding Human Behaviour in Industrial Human-Robot Interaction by Means of Virtual Reality’, in Proceedings of the Halfway to the Future Symposium 2019. HTTF 2019: Halfway to the Future, Nottingham United Kingdom: ACM, pp. 1–7. doi: 10.1145/3363384.3363403.

Gattullo, M. et al. (2019) ‘Towards augmented reality manuals for industry 4.0: A methodology’, Robotics and Computer-Integrated Manufacturing, 56, pp. 276–286. doi: 10.1016/j.rcim.2018.10.001.

Havard, V et al. (2019) ‘An architecture for data management, visualisation and supervision of Cyber- Physical Production Systems’, in Proceedings of the 17th International Conference on Manufacturing Research. ICMR 2019, Belfast, p. 6. doi: 10.3233/ATDE190015.

Havard, Vincent et al. (2019) ‘Digital twin and virtual reality: a co-simulation environment for design and assessment of industrial workstations’, Production & Manufacturing Research, 7(1), pp. 472–489. doi: 10.1080/21693277.2019.1660283.

Havard, V. et al. (2021) ‘A use case study comparing augmented reality (AR) and electronic document-based maintenance instructions considering tasks complexity and operator competency level’, Virtual Reality. doi: 10.1007/s10055-020-00493-z.

Ke, S. et al. (2019) ‘A enhanced interaction framework based on VR, AR and MR in digital twin’, Procedia CIRP, 83, pp. 753–758. doi: 10.1016/j.procir.2019.04.103.

Lai, Z.-H. et al. (2020) ‘Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing’, Journal of Manufacturing Systems, 55, pp. 69–81. doi: 10.1016/j.jmsy.2020.02.010.

Li, W. et al. (2021) ‘Fully convolutional network-based registration for augmented assembly systems’, Journal of Manufacturing Systems, p. S0278612520302314. doi: 10.1016/j.jmsy.2020.12.017.

Matsas, E. and Vosniakos, G.-C. (2017) ‘Design of a virtual reality training system for human–robot collaboration in manufacturing tasks’, International Journal on Interactive Design and Manufacturing (IJIDeM), 11(2), pp. 139–153. doi: 10.1007/s12008-015-0259-2.

Ojer, M. et al. (2020) ‘Projection-Based Augmented Reality Assistance for Manual Electronic Component Assembly Processes’, Applied Sciences, 10(3), p. 796. doi: 10.3390/app10030796.

Pérez, L. et al. (2019) ‘Industrial robot control and operator training using virtual reality interfaces’, Computers in Industry, 109, pp. 114–120. doi: 10.1016/j.compind.2019.05.001.

Rabah, S. et al. (2018) ‘Towards improving the future of manufacturing through digital twin and augmented reality technologies’, Procedia Manufacturing, 17, pp. 460–467. doi: 10.1016/j.promfg.2018.10.070.

Rückert, P., Wohlfromm, L. and Tracht, K. (2018) ‘Implementation of virtual reality systems for simulation of human-robot collaboration’, Procedia Manufacturing, 19, pp. 164–170. doi: 10.1016/j.promfg.2018.01.023.

Sjarov, M. et al. (2020) ‘The Digital Twin Concept in Industry – A Review and Systematization’, p. 8.

de Souza Cardoso, L. F., Mariano, F. C. M. Q. and Zorzal, E. R. (2020) ‘A survey of industrial augmented reality’, Computers & Industrial Engineering, 139, p. 106159. doi: 10.1016/j.cie.2019.106159.

Uhlemann, T. H.-J., Lehmann, C. and Steinhilper, R. (2017) ‘The digital twin: Realizing the cyber-physical production system for industry 4.0’, Procedia Cirp, 61, pp. 335–340.

Xiang, Y. et al. (2018) ‘PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes’, arXiv:1711.00199 [cs]. Available at: http://arxiv.org/abs/1711.00199 (Accessed: 21 April 2021).

Zeng, A. et al. (2017) ‘Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge’, in 2017 IEEE International Conference on Robotics and Automation (ICRA). 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore, Singapore: IEEE, pp. 1386–1383. doi: 10.1109/ICRA.2017.7989165.


Dans cette rubrique

(c) GdR 720 ISIS - CNRS - 2011-2022.