For a given scene, the problem of assigning a ‘class’ to various spatial regions is an active area of research and is important for a wide range of applications. Classification can be performed with the help of hyperspectral images , where each pixel of the scene has a spectrum associated with it, resulting in a 3-dimensional dataset or datacube, with two spatial and one spectral dimension, illustrated in Figure 1a. This approach usually requires two stages, firstly; the acquisition of the 3D hyperspectral datacube, and secondly, the classification algorithm. Obtaining the hyperspectral datacube of a scene can take a long time, typically requiring multiple acquisitions with a 2D sensor, making the classification of scenes with fast temporal changes difficult. Additionally, the datacube is very large, which may be problematic for limited bandwidth embedded applications, and results in computationally heavy classification algorithms.
2 Snapshot Classification
This project is focused on developing an alternative method of classification, combining the acquisition and classification stages, and avoiding the necessity to obtain the entire 3D datacube. Instead we intend to directly access the 2D classified image by optimized measurements of the hyperspectral datacube, ideally requiring only a single acquisition or ‘snapshot’ with a 2D sensor. By avoiding reliance on the intermediate 3D datacube, we can close the loop between classification and acquisition and increase frame rate of the imaging system.
For this purpose, we resort to a particular dual-disperser hyperspectral imaging system [2,3] that provides access to specific 2D projections of the datacube, specified by the configuration of a digital micro-mirror device - a particular projection being the panchromatic scene image, which provides key spatial information about the scene.
The aim of this internship is to develop and implement a near-snapshot approach to classification for a dual-disperser hyperspectral imager, relying on certain priors on the spectral-spatial correlations on the hyperspectral scene. For example, we can divide the scene into regions of homogeneous spectra using a segmentation method on the panchromatic image, and then infer the class of each segment by way of regularization on randomly measured data, or by solving a constraint satisfaction problem to inform us which voxels to measure within the hyperspectral datacube, giving us the necessary spectral information of each region with a small number of acquisitions. For a complex scene or a large spectral library, other techniques such as principal component analysis could be utilized. Depending on the particular application, there are a number of potential routes by which the internship could achieve the goal of classification, and so there is some flexibility and freedom in the exact direction of project.
For all approaches modelling of the system is vital, to assess the performance of the method compared to standard classification, and to determine optimum operating conditions and applications. If time allows, implementation of the scheme using a prototype system in the lab is possible, taking into account real experimental conditions such as noise and optical aberrations.
This is a multi-disciplinary project, requiring a broad understanding and application of optics, computer science, algorithms and mathematics. If necessary, remote working is possible for the majority of this project.
The candidate should have a background in physics, engineering or mathematics, with some experience programming in python or matlab. Knowledge of optics, signal processing, image analysis, computer vision, remote sensing or instrumentation is a bonus. A good level of English is mandatory.
The candidate should contact Elizabeth Hemsley (email@example.com) with a CV, letter of motivation and any references. The 6 month internship will take place within the PHOTO group at LAAS-CNRS, in close collaboration with robotics research group RIS and signal processing group SISU. The intern will receive a gratification of about 568 € per month.
(c) GdR 720 ISIS - CNRS - 2011-2020.