Annonce

Les commentaires sont clos.

Stage M2: Domain Adaptation for Sensor-Robust Panoptic Segmentation of 3D Scans

18 Novembre 2021


Catégorie : Stagiaire


This internship takes place between the 3D digitization startup SAMP and the machine learning department of IGN. The goal is to develop tools for leveraging annotated 3D data from multiple sources into one domain-robust model.

 

Domain Adaptation for Sensor-Robust Panoptic Segmentation of 3D Scans

  • Laboratory: STRUDEL team, LaSTIG laboratory (IGN/Univ. Gustave Eiffel)
  • Company: SAMP SAS
  • Location: IGN: Saint Mandé, France (3 min outside Paris), SAMP: Station F, Paris
  • Advisors: Loic Landrieu, PhD and Shivani Shah, PhD
  • Remuneration: 1200 euros gross
  • Starting Date: May 2022, 5 months duration
  • Key Words: Domain Adaptation, 3D Data, Panotic Segmentation, Deep Learning Development


Context

IGN: STRUDEL Team is a machine-learning research team with IGN, the French Mapping Agency. It focuses on solving large-scale computer vision and remote sensing challenges by developing state of-the-art methods. In particular, it focuses on scalable 3D deep learning [5, 4] and open-source frameworks [1].

SAMP: A deep-tech French startup focused on providing as-built 3D Digital Twin Solutions for the large industrial facilities. Main addressable markets include Energy, Chemical and Manufacturing industries. They are building a collaborative SaaS platform for remote access of the 3D Digital Twins. Also focused on working with stat-of-art 3D Deep learning research to bring Scale and Automation to the solution.

Objective

Bolstered by the rapid progress of 3D sensor technology, private and public actors have seen a stark increase in both the quantity and quality of available 3D data. Alongside this accessibility, recent methodological advancement in terms of deep learning applied to 3D data have considerably improved the capacity for automated analysis of 3D scans. However, training high performance deep learning methods requires large quantity of annotated data. Furthermore, these approach tends to be very sensitive to the data distribution used in the training phase.

Companies such as SAMP have access to a large amount of data to train their models. However, the scans comes from many different sites that may differ in their nature (Nuclear Plants, Oil and Gas plants, Chemical Plants, Manufacturing Plants) as well as the characteristic of the sensors. This makes it hard to leverage the quantity of available data to train models across several datasets. The objective of this internship is to implement an approach allowing to train a network to analyse 3D data of different industrial sites and acquired with a variety of sensors. To this end, an existing panoptic segmentation [3, 6] network will be modified in order to handle the difference in data distribution. The intern will investigate adversarial domain adaptation techniques such as DAN [2]: by making the learned features indistinguishable across all sites, a single model can leverage the entirety of SAMP database.

The tasks of this internship are as follows:

(i) Understand and familiarize with the models developed within SAMP for panoptic semantic segmentation, as well as the characteristics of the available datasets;

(ii) Implement a domain adaptation training routine across different dataset in order to make features site-independent;

(iii) Train a model across all available datasets;

(iv) Validate the approach on available public datasets;

(v) Validate approach on SAMP’s datasets.

Provided satisfying results, this work will lead to the writing of a conference paper with the student.

Requested Profile

  • Student in Master 2 in computer science, applied mathematics or other relevant courses;
  • Familiarity with machine learning and computer vision concepts;
  • Experienced with Python and familiar with PyTorch;
  • Curiosity, rigor;
  • (Optional) Experienced with 3D neural networks and versioning interfaces (github);
  • (Optional) Good level of written English.


Contact
Send a CV and a short statement of purpose (∼10 lines) explaining your interest for this internship to the following adresses: loic.landrieu@ign.fr and ShivaniShah[shivani@samp.ai].


References
[1] Thomas Chaton, Nicolas Chaulet, Sofiane Horache, and Loic Landrieu. Torch-points3d: A modular multi-task frameworkfor reproducible deep learning on 3d point clouds. 3DV, 2020.
[2] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016.
[3] Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár. Panoptic segmentation. In CVPR, 2019.
[4] Loic Landrieu and Mohamed Boussaha. Point cloud oversegmentation with graph-structured deep metric learning. In CVPR, 2019.
[5] Loic Landrieu and Martin Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. In CVPR, 2018.
[6] Quang-Hieu Pham, Thanh Nguyen, Binh-Son Hua, Gemma Roig, and Sai-Kit Yeung. JSIS3d: Joint semantic-instance segmentation of 3D point clouds with multi-task pointwise networks and multi-value conditional random fields. CVPR 2019