PhD thesis : Assembly Human Action Recognition and Modeling

29 Juillet 2023

---

Catégorie : Doctorant

---

Thesis proposal title: Assembly human action recognition and modeling

PhD grant: Industrial collaboration with Kickmaker https://www.kickmaker.fr/en/

Research laboratory: BMBI-UTC (France) https://bmbi.utc.fr/

Salary: 1800-2000 euro net/month + health insurance + participation in public transport

Starting date: December 2023/January 2024

Subject :

Assembly action recognition is of great significance for the manual assembly monitoring, human-robot cooperation and ergonomic analysis of assembly operations. Considering the increasing demands for the multiformity and diversification, mass customization has become a trend for many companies. In the assembly process of mass customization, there is a wide range of personalised products with different assembly models. In fact, missing an assembly step or even an irregular operation of a worker adversely affects the product quality. Therefore, numerous operations, including slipping, smoking, calling and walking operations should be monitored or recognized in an assembly line. These operations are classified as macro-actions in the assembly line. The main purpose of monitoring the macro-actions is to determine the action type of operators. In contrast with macro-actions monitoring, the main purpose of micro-action monitoring is to monitor the pose and joint information of an operator. The proposed thesis project will explore the following items:

• A large, complete and detailed study of the existing state of the art works for the related topic in terms of existing public databases, utilised modalities, machine learning technique and evaluation metrics while pointing out the facing challenges and the drawbacks of the existing techniques. Recording a novel large micro/macro assembly actions database by designing a complete experimental setup exploiting multimodality sensors (RGB, RGB-D, EMG, IMU, Radar, vocal command) in order to detect and recognize the various action types of the worker for completion of a movement.
• Learning simple but yet efficient robust high-level representations for each movement from multi-modal data while taking into consideration the various problems of large intra-class variations and the uncertainty of the recorded data. A final objective should be the design of evolutive assembly action dictionaries allowing the recognition and projection of complex recorded tasks.
• Exploiting a priori knowledge during the learning and the evaluation process (morphology, object of interest, technical experience, etc)
• Performing a comparative study of the best computation / accuracy performances using the different pairwise data-modality (sensors) / feature representation for a future real-time assembly action recognition system at low cost.

Profile:

• Engineer or MSc degree in computer science, computer vision, electrical engineering, biomedical engineering or related fields
• Strong programming skills in Python and Matlab
• Knowledges in instrumentation and various sensors
• Knowledges in machine learning computer vision and data fusion
• Good communication skills

Application:

CV, Cover Letter, Recommendation Letters to :

imad.rida@utc.fr
sofiane.boudaoud@utc.fr
dan.istrate@utc.fr