PhD in the IPI team (LS2N lab) in video coding domain.
15 Avril 2022
Catégorie : Doctorant
PhD title: Representation and analysis of immersive dynamic data based on vector quantization and reinforcement learning.
Laboratory: LS2N (UMR 6004, Nantes)
Team: IPI (Image Perception Interaction)
The thesis addresses the issue of representation and analysis of immersive dynamic data by using vector quantization and reinforcement learning. In the LS2N/IPI team we have already an experience in Geometric Point Cloud Compression domain by using Tree-Structured Point- Lattice Vector Quantization. We plan to extend the method for the analysis and representation of the dynamic 3D contents.
Representation and analysis of immersive dynamic data based on vector quantization and reinforcement learning.
Augmented, virtual and mixed reality experiences (AR/VR/MR) enable users to navigate in multisensory 3D media experiences. The technologies used in order to capture the real world in multiple dimensions and to present it to the users, require enormous amounts of data so it is necessary to improve compression quality and signal processing. The context of the thesis is then the compression of volumetric visual data which are commonly represented by Point cloud format. A point cloud (PC) is a set of individual 3D points, where each point has a 3D position and contains some other attributes such as color, surface normal, etc. Point clouds are more flexible than polygonal mesh and could be processed in real-time.
The Point Cloud Compression (PCC) has been addressed by MPEG which has recently published a standard consisting in two classes of solutions : Video-based (V-PCC) appropriates for relatively uniform distribution of dynamic points, and Geometry-based (G- PCC) appropriates for sparse still distributions. So, there is still room for exploratory research on dynamic PCC by using Geometric approach. Beyond compression purpose, another goal of the research is targeted towards the analysis of the dynamic 3D content based on machine learning approaches .
Objectives of the thesis:
In the IPI team, we have already addressed the issue of static G-PCC by using Tree-Structured Point-Lattice Vector Quantization (TSPLVQ) . This representation enables hierarchically structured 3D content that improves the compression performance for static point clouds. The novelty of the proposed scheme lies in adaptive selection of the optimal quantization scheme of the geometric information, that better leverage the intrinsic correlations in point cloud. Based on its adaptive and multiscale structure, two quantization schemes are dedicated to project recursively the 3D point clouds into a series of embedded truncated cubic lattices. At each step of the process, the optimal quantization scheme is selected according to a rate-distortion cost in order to achieve the best trade-off between coding rate and geometry distortion, such that the compression flexibility and performance can be greatly improved. Experimental results show the interest of the proposed multiscale method for lossy compression of geometry.
The objectives of the thesis will be to develop some topics that we just have started to dig, with:
• The compression of the points color attributes after graph transform  where the adjacency matrix is computed directly from the geometric tree structure.
• The compression of dynamic PC , because our model enables to exploit the temporal dependencies of the 3D content. Exactly, TSPLVQ is a top-down method and permits to represent the PC geometry by using a scalable tree, so when quantizing the dynamic geometry of a given PCs sequence, the successive trees are represented as a trunk common for the 3D sequence, and branches added for each frame. Next, the trunk is first coded, followed by the branches that are differentially coded.
• The analysis of the of 3D dynamic content based on reinforcement learning methods.
 The website about MPEG Point Cloud Compression: https://mpeg-pcc.org
 A. Filali, V. Ricordel, N. Normand and W. Hamidouche : Rate-Distortion Optimized Tree-Structured Point-Lattice Vector Quantization for Compression of 3D Point Clouds Geometry, International Conference on Image Processing (ICIP), IEEE, 2019.
 Cohen, R., Tian, D., and Vetro, V., : Attribute compression for sparse point clouds using graph transforms : International Conference Image Processing (ICIP), IEEE, 2016.
 A. Filali, V. Ricordel, and N. Normand: Static and Dynamic 3D Point Cloud Compression by TSPLVQ , Proceedings of the International Conference on Systems Signals and Image Processing (IWSSIP), 2-4 June 2021, Bratislava, Slovakia
 M. Quach, G. Valenzise, and F. Dufaux: Learning convolutional transforms for lossy point cloud geometry compression, International Conference on Image Processing (ICIP), IEEE, 2019.
The funding are from the French research ministry (MESR) and is called CDE for "Contrat doctoral avec activités complémentaires" and this complementing activity is usually teaching. In this case, the PhD student gets a 3 years long contract to achieve his PhD, where he shares is time between research and teaching (typically 64h of teaching per year).
Send CV & motivation letter to firstname.lastname@example.org