A M2 internship position is available at CESI campus Rouen : "Evaluation and Application of Foundation Models for 6D Pose Estimation in Industrial Environments Using Digital Twins"
Title: M2 Internship –Evaluation and Application of Foundation Models for 6D Pose Estimation in Industrial Environments Using Digital Twins
Scientific Fields: Computer Vision, Industry of the Future
Keywords: Foundation Models, 6D pose estimation, Digital Twin, Synthetic and Hybrid Dataset, Manufacturing environments
|
Name |
Position, Title |
@ |
|
Nicolas Ragot |
Associate professor |
|
|
Vincent Havard |
Associate professor, HDR
|
This M2 internship is part of the FUSION project and its Work Package 3 (WP3), whose goal is to update the digital twin of an industrial environment based on the robot vision-based perception.
Foundation models are increasingly used in the literature across a wide range of applications. Also, this is the case in 6D pose estimation, with Wen et al.’s proposal, titled FoundationPose, which achieves excellent results compared to state-of-the-art methods. The approach has been tested on various datasets.
We aim to evaluate its performance in the context of manufacturing industrial environments through training performed using the digital twin of the production workshop.
The tasks assigned to the intern are as follows:
Project Context
This recruitment is part of the FUSION project (Framework for Universal Software Integration in Open Robotics), which was selected under the "I-Démo – France 2030 Regionalized Normandie" call for projects. The project's partners are Conscience Robotics (lead), OREKA Ingénierie, and CESI LINEACT.
The main objective of the FUSION project is to democratize the use of robotics by introducing a paradigm shift that places the user at the center of the system through:
The targeted use case focuses on dismantling operations within a nuclear site cell, specifically the cutting of contaminated pipelines. Currently, these operations are carried out by operators remotely controlling the robotic arm using only cameras installed on the intervention site and mounted on the robotic arm. This significantly complicates teleoperation due to the lack of depth perception.
Our proposal aims, first, to reduce the complexity of robot teleoperation by replacing environment perception through cameras with immersion in a real-time-generated digital twin of the work area. Secondly, the project seeks to "teach" robots naturally to perform repetitive tasks that require only occasional supervision.
|
|
Work Period |
|||||||
|
Literature review on recent approach on 6D pose estimation |
|
|
|
|
|
|
||
|
Implement FoundationPose |
|
|
|
|
|
|
|
|
|
Generate a DT synthetic dataset, train the model and evaluate the performances |
|
|
|
|
|
|
|
|
|
Evaluate the performances on the real data from the Physical Twin |
|
|
|
|
|
|
||
|
Adjust the initial dataset by incorporating labeled real-world data and evaluate the peformances |
|
|
|
|
|
|
||
Expected Scientific/Technical Production
Laboratory Presentation
CESI LINEACT (UR 7527), the Digital Innovation Laboratory for Businesses and Learning in support of Territorial Competitiveness, anticipates and supports technological transformations in sectors and services related to industry and construction. CESI's historical ties with businesses are a determining factor in its research activities, leading to a focus on applied research in partnership with industry. A human-centered approach coupled with the use of technologies, as well as regional networking and links with education, have enabled cross-disciplinary research that centers on human needs and uses, addressing technological challenges through these contributions.
Its research is organized into two interdisciplinary scientific teams and two application domains:
· Team 1, "Learning and Innovating," is primarily focused on Cognitive Sciences, Social Sciences, Management Sciences, Education Science, and Innovation Sciences. The main scientific objectives are understanding the effects of the environment, particularly instrumented situations with technical objects (platforms, prototyping workshops, immersive systems), on learning, creativity, and innovation processes.
· Team 2, "Engineering and Digital Tools," is mainly focused on Digital Sciences and Engineering. Its main scientific objectives include modeling, simulation, optimization, and data analysis of cyber-physical systems. Research also covers decision-support tools and studies of human-system interactions, especially through digital twins coupled with virtual or augmented environments.
These two teams cross and develop their research in the two application domains of Industry of the Future and City of the Future, supported by research platforms, primarily the Rouen platform dedicated to the Factory of the Future and the Nanterre platform dedicated to the Factory and Building of the Future.
Funding: CESI Nord-Ouest (FUSION project, i-démo régionalisé co-funded by Région Normandie and the European Union)
Workplace : Rouen Campus (in Saint Etienne du Rouvray)
Profile Sought: Master's in Computer Science with a focus on artificial intelligence, computer vision.
Scientific and technical skills:
|
Skills |
Technical stack |
Operating Systems |
|
|
Artificial Intelligence and Computer Vision |
Python & C++ C# (optional) |
PyTORCH, DOCKER UNITY (optional) |
LINUX & WINDOWS |
Interpersonal Skills:
· Autonomy, initiative, curiosity
· Teamwork ability and good interpersonal skills
· Rigorousness
Application Process: by dossier and interview.
Send your application to Nicolas Ragot (nragot@cesi.fr), Vincent Vauchey (vhavard@cesi.fr), with the subject line: "[Application] Title on page 1".
Your application should include:
Please send all documents in a single zip file named LASTNAME_firstname.zip
(c) GdR IASIS - CNRS - 2024.