Imputation of cloudy pixels in Satellite Image Time Series (Stage de fin d'études/Master thesis)
Imputation of cloudy pixels in Satellite Image Time Series
Keywords: Satellite Image Time Series, missing data, Generative Adversarial Networks
On March 7 2017, the European Space Agency (ESA) successfully put its latest high-resolution satellite Sentinel-2B into orbit. The two Sentinel-2 satellites are now capturing pictures of all emerged surfaces every 5 days at high spatial and spectral resolution, which makes it possible to monitor the evolution of the vegetation . Such land surface monitoring is a key input for the prediction of climate trends and the management of territories.
However, satellite image time series (SITS) are contaminated by poor atmospheric conditions, especially the presence of clouds. These affect SITS usage in many application including image classication, object detection, or image interpretation. Although different reconstruction techniques, known as gapfilling methods, have been proposed , most of the methods focuses on single image or on medium spatial resolution images. As few takes benefit from the high spatial and temporal resolutions of Sentinel-2 data, new scalable gapfilling techniques are needed to accurately unveil pixel values below the clouds.
The internship will focus on the proposition of a deep generative model to reconstruct missing values in series of satellite images. Recently, adaptations of the popular Generative Adversarial Network (GAN)  have been proposed to impute missing values in images and time series with same distributions than the valid original data. In particular, Generative Adversarial Imputation Network (GAIN)  is composed of two networks trained concurrently:
1. a generator, that is in charge to generate the missing values,
2. a discriminator, that should discriminate imputed values from original values.
Hence, the goal of the internship would be to propose and evaluate generator and discriminator networks, that takes into account the spatial and temporal dimensions of SITS, e.g. 3D-CNN or ConvLSTM. Experiments would be carried out on recent Sentinel-2 image
Required or appreciated skills: strong programming skills, familiar with deep learning techniques, good communication skills, and keen to learn about time series and satellite data analysis.
 Inglada, J., Vincent, A., Arias, M., Tardy, B., Morin, D., & Rodes, I. (2017). Operational high resolution land cover map production at the country scale using satellite image time series. Remote Sensing, 9(1), 95.
 Shen, H., Li, X., Cheng, Q., Zeng, C., Yang, G., Li, H., & Zhang, L. (2015). Missing information reconstruction of remote sensing data: A technical review. IEEE Geoscience and Remote Sensing Magazine, 3(3), 61-85.
 Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., & Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems (pp. 2672-2680).
 Yoon, J., Jordon, J., & Van Der Schaar, M. (2018). GAIN: Missing data imputation using generative adversarial nets. International Conference on Machine Learning (pp. 5675-5684).
Supervisors : Charlotte Pelletier (Univ. Bretagne Sud/IRISA) & Romain Tavenard (Univ. Rennes 2/LETG)
Contact: Charlotte Pelletier: email@example.com
Universite Bretagne Sud, UMR IRISA,
Campus Tohannic, 56000 Vannes, France
(c) GdR 720 ISIS - CNRS - 2011-2020.