Keywords : satellite imagery, time series, multi-scale, multi-label classification, land-cover, land-use,
deep learning, regularization, graphs
Context: MAESTRIA project
The MAESTRIA project (Multi-modAl Earth obServaTion Image Analysis) aims to solve the methodological challenges related to the fully automatic analysis of the massive amount of images acquired by Earth Observation (EO) platforms. MAESTRIA targets to generate land-cover and land-use descriptions (LCDB) at country scale at many spatial resolutions and for different sets of classes. Both public policies at local or national levels and scientific models would benefit from such kinds of products for climate modelling, urban planning, crop monitoring or impact assessment of surface changes.
Many global LCDB have been established during the last two decades. However, they still do not meet the current requirements in terms of semantic and spatial accuracy, automation and updateness. In parallel, a large body of literature has tackled automatic EO data exploitation. However, most existing approaches are limited to a specific environment, site or sensor, and a specific need. They are not flexible enough and not adapted to the new paradigm in EO with the advent of satellite missions with short revisit time and increased spectral and spatial resolutions (e.g., Sentinel satellites).
The MAESTRIA project will first generate various land-cover maps at large-scales using novel approaches in multi-modal data fusion and semi-supervised learning. These maps will be generated with reduced delays and enriched semantics compared to existing solutions. However, they will still exhibit rigid sets of classes and spatial resolution.
This PhD thesis work aims to alleviate this issue, bringing more flexibility by automatically deriving new products to answer various case studies.
The goal here is to develop methods to derive automatically new land-cover products with different spatial and semantic resolutions out of those produced before. As a consequence, we target to obtain a continuum of adapted land-cover layers, both in terms of spatial scales (2 →50-100m) and semantics.
The problem of automatically sliding spatial and semantic scales is still challenging and several significant methodological locks still have to be alleviated: modifying the scale of analysis requires to change the spatial resolution and the set of classes, while keeping coherent labels across scales. One has to deal with the fact that some labels at a coarse spatial resolution can contain several labels, semantically distinct at a finer resolution, while, on the opposite, a same class at a finer spatial resolution corresponds to several distinct classes at a coarser resolution (for instance a "building" at a fine resolution may belong to either "continuous" or "discontinuous urban areas" classes in a coarser resolution). This is a high-order semantic segmentation problem aiming at defining meaningful patterns in terms of semantics at different scales. The novelty relies on the fact that most of the time, classification and segmentation at multiple scales are not jointly addressed (e.g., in cartography with extensive reasearches in generalization processes). Besides, the correspondence between classes across categories or families provided in most computer vision problems is rigid, and therefore not adapted to land-cover mapping. Polysemy is even higher in our context.
A challenge is to automatically obtain spatially and semantically coherent structures containing initial elements that are various individual structures. These algorithms will merge objects/segments from monoscale land-cover maps, so as to retrieve new objects, semantically and spatially coherent with other scales. Most effort has focused on urban areas so far, in particular related to local climate zones. The approaches are thus quite specific, and do not ensure a smooth transition between spatial resolutions. Most of them benefit from a predefined skeleton with predefined units
(grid cells or blocks derived from the road network). The labelling task is processed per unit, taking into account features calculated at such a level. Current approaches require a good initialization of the land-use boundaries, in practice not known beforehand. It is now widely assumed that segmentation and classification are interleaved issues, which will be addressed here. Eventually, the spatial scales of interest are known and limited to 2 layers while we target to gain in flexibility and ensure a smooth transition between all plausible scales (2-5-10-20-50-100m).
Two solutions are conceivable :
The PhD student will be jointly supervised by the LaSTIG lab. (Clément Mallet – Arnaud Le Bris) and UMR CESBIO (Jordi Inglada). The student will be mainly located in IGN (close to Paris, France) with frequent stays in Toulouse (France).
How to apply ?
A single PDF file should be sent to firstname.lastname@example.org. It must include :
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu,D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio. Generative adversarial nets. In : Advances in Neural InformationProcessing Systems, 2014.
J. Inglada, A. Vincent, M. Arias, B. Tardy, D. Morin, I. Rodes. Operational high resolution land cover map production at the country scale using satellite image time series. Remote Sensing, 9(1), 2017
W. Kuhn. Semantic reference systems. International Journal of Geographical Information Science, 17(5):405–409, 2003.
C. Kurtz, N. Passat, P. Gançarski, A. Puissant. Extraction of complex patterns from multiresolution remote sensing images: A hierarchical top-down methodology. Pattern Recognition, 45(2):685 – 706, 2012.
L. Landrieu G. Obozinksi. Cut pursuit: fast algorithms to learn piecewise constant functions on general weighted graphs. SIAM Journal on Imaging Science, 01306779(4):1724–1766, 2017.
J. H. Lowry M. B. Lowry. Comparing spatial metrics that quantify urban form. Computers, Environment and Urban Systems, 44:59 – 67, 2014.
M. Maire, S. Yu, P. Perona. Object detection and segmentation from joint embedding of parts and pixels. In IEEE International Conference on Computer Vision, pages 2142–2149, 2011.
M. Ristin, J. Gall, M. Guillaumin, L. Van Gool. From Categories to Subcategories: Large-Scale Image Classification With Partial Class Label Refinement. In: CVPR, 2015.
B. Russell, A. Torralba, K. Murphy, W. Freeman. Labelme: A database and web-based tool for image annotation. International Journal of Computer Vision, 77(1-3):157–173, 2008.
N. Souly, C. Spampinato, M. Shah. Semi Supervised Semantic Segmentation Using Generative Adversarial Network. In : ICCV, 2017.
M. Wigness, B. Draper J.R. Beveridge. Efficient Label Collection for Image Datasets via Hierarchical Clustering. International Journal of Computer Vision, 126(1):59-85.
(c) GdR 720 ISIS - CNRS - 2011-2019.