Master's internship (M2) / IGN – LaSTIG Lab / FRANCE, Paris area
Title: « Pose-driven re-ranking for heterogeneous visual data geo-localization in urban environment »
PDF version: http://recherche.ign.fr/labos/matis/pdf/stages/2019/sujet_stage19_reranking.pdf
Computer vision, geo-localization, position uncertainty estimation, CBIR.
The proposed internship is part of a French research project (ANR pLaTINUM) that brings together several research laboratories, including LaSTIG from IGN (the French Mapping Agency), LITIS from Rouen Normandie University, INRIA Sophia Antipolis and Le2i from the University of Burgundy. The aim of the project is to develop a mapping and navigation system for long-term localization that can be automatically updated thanks to different kinds of user feedbacks. The localization is treated as an image-retrieval problem  where the image query to locate is compared to a geo-referenced dataset. In the project, this dataset is a graph of RGB-D-L (photometric + depth + semantic label) spheres covering an urban area. A deep-learned global descriptor  is used to evaluate the similarity between the query and the reference data. A pool of potential geo-referenced candidates for the localization is obtained by ranking the score obtained for each reference data according to their similitude to the query. Then, the localization of the query can be approximated by efficiently propagating the ones attached to the top-scored candidates.
Within this context, the internship will deal with the re-ranking of the geo-referenced candidates obtained by the first step of the localization process. Re-ranking is a well-known process used in Content-Based Image Retrieval (CBIR) to increase the relevance of the top ranked candidates retrieved after the similarity comparison . When the reference data are augmented with position information, specific re-ranking method can be applied [4-5]. The aim of the internship is to develop a new re-ranking method that benefits from the geo-localization information and the graph structure of the reference dataset.
The methodology considered will as be as follows: an input query is passed to the visual localization framework from , producing a ranked set of geo-localized candidates. The set is then re-ranked in order to improve relevance of the retrieved candidate according to the ground-truth position of the query.
This internship will in principle focus on how to exploit the position information of the reference data in order to improve the final localization result. In case of structured reference data, a re-ranking exploiting graph property will be also considered. The work is organized into 4 parts:
1. The study of the state-of-the-art re-ranking methods for multi-modal visual geo-localization,
2. The investigation of an innovative re-ranking method exploiting the positional information of the reference dataset as well as the graph structure of the map,
3. Evaluation of the results against a ground truth data and comparison against state-of-the-art methods,
4. Evaluation of the method within the global localization and navigation pipeline during end-project experiments (September 2019), with the project partners.
Duration: 6 months, starting from march/april.
Workplace: IGN, LaSTIG lab, Paris area, Saint-Mandé (73 avenue de Paris, metro Saint-Mandé, Line 1), France. IGN (French Mapping Agency) is a Public Administrative Institution part of the French Ministry for Ecology and Sustainable Development. IGN is the national reference operator for the mapping of the territory. The LaSTIG is one of the research laboratories of IGN, it gathers more than 100 researches centered on geographical information, 35 of them focusing in image analysis, computer vision, AI, photogrammetry and remote sensing.
Bac+5 in computer science, applied math or computer science (master or engineering school); good knowledge in image processing or computer vision, as well as good skills in C/C++ programming or Python.
Submitting your candidature
By sending by e-mail to the contacts, in a single PDF file:
o Motivation letter
o 2 recommendation letters
o Transcript of grades from the last two years of study
o A list of courses followed and passed in the last two years
o Nathan Piasco, PhD student, IGN/LaSTIG – email@example.com, http://recherche.ign.fr/labos/matis/~nathan.piasco
o Valérie Gouet-Brunet, researcher director IGN/LaSTIG – firstname.lastname@example.org, http://recherche.ign.fr/labos/matis/~valerie.gouet-brunet
 N. Piasco, D. Sidibé, C. Demonceaux, V. Gouet-Brunet. A Survey on Visual-Based Localization: On the Benefit of Heterogeneous Data. Pattern Recognition, Volume 74, pp.90-109, February 2018.
 N. Piasco, D. Sidibé, C. Demonceaux, V. Gouet-Brunet. Apprentissage de modalités auxiliaires pour la localisation basée vision. Reconnaissance des Formes, Image, Apprentissage et Perception, June 2018.
 F. Perronnin , Y. Liu , Large-scale image retrieval with compressed fisher vec- tors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2010.
 T. Sattler, M. Havlena, K. Schindler and M. Pollefeys. Large-Scale Location Recognition and the Geometric Burstiness Problem. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
 A. R. Zamir, M. Shah. Image geo-localization based on multiple nearest neighbour feature matching using generalized graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014.