Multiple-view reconstruction of deformable objects
26 Juillet 2022
Catégorie : Doctorant
The Imagine team at LIRIS laboratory, Lyon- France is looking for a PhD student to work on deformable 3D reconstruction from multiple views (Link to offer). The goal of this project is to develop fast reconstruction algorithms.
- Masters in computer vision, robotics, machine learning, mathematics or any field related to the topic
- Strong programming skills in C++ and python
- Fluency in English
Project duration: 36 months
Tentative start date: October 2022
How to apply:
Please send your CV, transcripts and 2 reference letters to Liming Chen and Shaifali Parashar with the subject "Multiple-view reconstruction of deformable objects ".
Reconstructing the 3D shape of objects from multiple images is an important goal in computer vision and has been extensively studied for both rigid and non-rigid (or deformable) objects. Structure-from-Motion (SfM) is an algorithm that performs the 3D reconstruction of rigid objects using the inter-image visual motion from multiple images obtained from a moving camera. SfM is a very accurate and stable solution. Deformable 3D reconstruction, however, has been widely studied for monocular images (obtained from a single camera) and still remains an open research problem. It involves an additional modelling of deformations which makes it rather complicated. Numerous strategies have been developed in the past 30 years, ranging from statistical approximations to physical ones (constraining lengths and other metric quantities), all missing a concrete, mathematical way of formulating deformations.
In our recent works[1,2,3], we have shown that breaking a deformable scene into infinitesimal planes allows the deformations to be considered as linear functions which leads to simple, fast solutions to their reconstruction through images. This brought a substantial improvement to the state of the art in terms of complexity, accuracy and robustness. Furthermore, it provides a mathematical framework that combines various kinds of deformation modelling, ranging from isometry (paper-like objects) [1,3] to elasticity [2,4], using simple local structures that preserve length, area, angles and smoothness.
In this project, we will extend the local methods to bring them at par with their rigid counterparts. This involves reducing the computational complexity of image registration, depth computation and including multiple view constraints to simplify the deformation constraints . We will explore the geometric formulations that can allow additional constraints to simplify the problem in order to achieve generic, robust and real-time solutions.
 Parashar et al, TPAMI 2017.Isometric Non-Rigid Shape-from-Motion with Riemannian Geometry in Linear Time.
 Parashar et al, TPAMI 2018. Local Deformable Reconstruction Using Cartan’s connections.
 Parashar et al, TPAMI 2021. Robust Isometric Non-Rigid Shape-from-Motion.
 Parashar et al, CVPR 2020. Local Non-Rigid Shape-from-Motion from Diffeomorphic Mappings.