Les commentaires sont clos.

Scaling Remote Sensing Analysis by Domain Incremental Learning

25 Mars 2022

Catégorie : Doctorant

The thesis addresses the scaling of remote sensing image analysis as a domain incremental problem exploiting unsupervised learning techniques.

For more Information, contact :

A more complete description with references can be found at:

Modern earth observation satellites produce nowadays very large volumes of data: the Copernicus Sentinel-2 system from ESA, for instance, visits the entire globe every 5 days and provides several petabytes of data each year. New algorithms are required to process this huge quantity. Scaling, the capacity of reaching the same quality of analysis for data acquired at any location and any date, is a real issue and the long term objective of the thesis.

The main application domain of the thesis is the analysis of remote sensing images for land-cover or land-use classification, object detection, 3D estimation, etc. Finding the good features, with the efficient balance between expressiveness and invariance, is the key for reliable image based prediction. In the last decade, deep learning techniques have transformed this task, previously achieved using a mix of prior knowledge and parameter estimation, to a purely data driven process.
The first, and successful, proposals of deep learning approaches relied on the availability of a large volume of annotated data (several millions of images in ImageNet). Deep networks are able to extract image features in an efficient way for classification using supervised learning. Many other computer vision tasks, such as object detection or semantic segmentation, have given rise to neural network pipelines relying on these deep features, at least as a pre-trained initial step.
One big limitation of pre-trained deep features is their dependence on the quality and scope of the annotated dataset used to learn them, and on the task: supervised learning for classification. Although large, current annotated datasets are biased and cannot encompass all the variability of possible inputs. It is therefore difficult to guarantee their universal relevance to every context or type of image.
Satellite data come with little annotation (mostly the acquisition date, viewing conditions and location), making pure supervised approaches not suitable when more and more data are available. Transfer learning techniques [55] such as domain adaptation or semi-supervision have been proposed to address this issue, but often with lower performance, especially when compared to full supervised learning.

More recently, there has been a strong endeavor to exploit unsupervised constraints as a complement or even as a substitute to supervised learning:
- Disentangled representations: construction of latent spaces of generative models able to build and identify meaningful features ;
- Self-supervision: introducing what is called a pretext task, i.e. a task that is easy to specify from unannotated data but that requires ""good"" features to be solved ;
- Contrastive learning: empirical description of invariance by generating or selecting data that should have similar predictions.
Unsupervised learning techniques to learn features are now believed to almost surpass supervised approaches although questions remain about their universal suitability.

Several of these techniques are beginning to be adapted to remote sensing data but not with a scaling objective in mind.

A second issue related to scaling, besides the lack of annotation, is the computing and storage capacity needed to update prediction models when new data are available. Retraining globally with an ever-increasing dataset is impossible in practice and more incremental schemes are needed to progressively control performance and guarantee backward compatibility, especially when increasing the geographical scope from city/region to country/continent/earth. This problem is referred to continual or incremental learning in the literature where the main phenomenon to control is what is called “catastrophic forgetting"", i.e. the fact that a system loses its predictive capacity on previously learned domains or tasks when new data are. Incremental learning can also be aided when initialized by good representations or by exploiting semi-supervision . The question of updating good representation for an increasing domain, however, is open.

One possible direction of investigation will be to study the capacity of a deep network architecture to accommodate new informative features, either dynamically or by using self-attention mechanisms. Techniques originating from network architecture optimization could also be adapted to the continual learning setting.

As a summary, in this thesis, it is proposed to address the scaling of remote sensing image analysis as a domain incremental problem exploiting unsupervised learning techniques,
The thesis will take place at ONERA. The work will be associated with several research projects about remote sensing data interpretation and will benefit from interactions with other researchers and PhD students of the team.

For more Information, contact :

A more complete description with references can be found at: