A PhD position is available in the Inria team MOSAIC, in the context of the European project ROMI (details below). The PhD student will be hosted by the Computer Graphics and Geometry team of the University of Strasbourg, where the main advisor is based. Several stays are planned in Lyon, to work with colleagues of the team and other partners of the project.
3D segmentation and growth tracking of plant point clouds in field conditions
Franck Hétroy-Wheeler, Univ. Strasbourg/Inria
Christophe Godin, Inria
Strasbourg, France, with several stays in Lyon, France
Master or Engineering degree in computer science or applied maths. Excellent programming skills (Python and/or C++), background in computer vision and/or computer graphics, as well as linear algebra and geometry.
The European project ROMI aims at developing an open lightweight robotics platform for crop monitoring and weed reduction in small farming land areas. This platform will be equipped with imaging sensors and software to reconstruct and analyze plants in 3D. Within this project, the proposed PhD position will be at the core of the data processing pipeline. More specifically, the data acquisition and conversion to 3D point clouds will be done by a team at CNRS (Lyon, France) and a team at Sony CSL (Paris, France), while plant architecture models and parameter extraction methods for these models will be developed by teams at CNRS and Inria (Lyon, France). In coordination with them, the hired PhD student will be in charge of developing new mathematical and algorithmic tools to segment a plant (represented as a 3D point cloud) into its organs and track these organs through time.
The overall goal of the PhD is to develop new tools to segment a 3D model of a plant into its organs and to track their growth. Arabidopsis thaliana and Chenopodium album will be the two species taken as examples. The input 3D model is a noisy point cloud with non-uniform density and missing data, due to occlusions.
The first stage of the PhD will be to automatically and independently segment each point cloud into the plant's organs (especially stems and leaves), without any prior knowledge on the species. Existing methods often assume a clean 3D point cloud (e.g., [1,2]). Others are either destructive , plant-specific  or not fully automatic [5,6]. Similar to [2,6], a spectral clustering approach will be considered, but local geometric information around each point of the cloud should first be better estimated despite the noise and varying density. Hence, a first work will be to develop a method for robust local surface estimation and compare to the related work .
The second stage will deal with the tracking of each organ. Compared to the more usual case of human characters, this is challenging since the geometry of the organs drastically changes during the growth, which makes usual rigidity or isometry assumptions impossible. Some organs may even appear or disappear during the growth process. Previous work on this topic has shown that impressive results can be obtained for plants acquired in a controlled environment . As noted in  the problem is more complex in the case of noisy point clouds of plants with large leaves, though a recent work proposes a solution for blooming flowers with a fixed number of petals . Both these works demonstrate that a double forward-backward matching process is necessary to efficiently track growing leaves even if they collide or in case of occlusions, and that it could refine the segmentation. Nevertheless, using a simple template model as in  is impossible in our more complex case because the number of organs can vary through time. As for humans in wide clothing , we therefore plan to start with sparse one-to-one point correspondences and then define a local deformation model.
In the third and last stage of the PhD, prior knowledge about the plant architecture will be integrated to the process in order to make both the segmentation and the tracking more robust. This will be done in collaboration with other partners of the ROMI project and is expected to generate fully consistent 3D+t architectures that faithfully interpret the collected data. The performance of this pipeline will be evaluated against the results provided by machine learning approaches directly applied to the 3D point clouds developed by another partner.
Deadline: 20th of May, 2018.
(c) GdR 720 ISIS - CNRS - 2011-2018.