Les commentaires sont clos.

Free EURASIP JIVP Webinar on June 2nd with F. Pereira at 12:30pm CET

25 Mai 2021

Catégorie : Autres événements


Point Cloud Coding: Time for Learning-based Alternatives

Fernando Pereira

Instituto Superior Técnico, Universidade de Lisboa - Instituto de Telecomunicações


Multimedia experiences play a growing role in our daily lives as its overwhelming presence vastly impacts many important application areas such as entertainment, sports, education, communication, art and medicine. The recent emergence of powerful imaging sensors is allowing to acquire much richer, faithful, realistic and immersive representations of the visual scenes, notably 3D visual representations, thus offering the users high quality experiences with navigation and immersion functionalities.

The light information associated to these richer visual representations may be modelled by the so-called plenoptic function, which has a limited number of practical acquisition models; the light field and point cloud representation models deserve special reference. Both these representation models have the potential to offer the so-called 6 Degrees of Freedom (DoF) immersion, where different and appropriate visual information is offered to the user while exploiting the three rotational degrees of freedom and the three translational degrees of freedom, very much in the same way as in the real world.

Since point clouds are essentially a set of points in the 3D space with associated features, they are naturally suitable to facilitate user interaction and offer a high level of immersion. However, since providing realistic, interactive and immersive experiences typically requires point clouds with a rather large number of points, efficient coding is critical as recognized by standardization groups such as MPEG and JPEG, which have been developing PC coding standards in the context of the JPEG Pleno and MPEG-I projects.

The popularity of learning-based solutions for multimedia processing tasks has largely increased in recent years due to its impressive performance. In terms of coding, recent learning-based image coding solutions offer very promising results, even outperforming state-of-the-art image codecs. In this context, it is natural to extend the learning-based coding approach to point clouds. This type of coding approach offers three major benefits: i) a single efficient point cloud representation for both humans and machines, i.e., not only for visualization but also for computer vision tasks such as classification, detection and recognition; ii) reduced complexity resources to perform image processing and computer vision tasks (as already starting from compressed domain features); and iii) using features extracted from the original instead of lossy decoded images to perform computer vision tasks with higher accuracy.

In this context, this talk will address the emerging developments in learning-based point cloud coding, including the recent decision by JPEG to develop a learning-based coding standard for point clouds and associated attributes, offering a single-stream, compact compressed domain representation, supporting advanced flexible data access functionalities.



Fernando Pereira is currently with the Department of Electrical and Computers Engineering of Instituto Superior Técnico and with Instituto de Telecomunicações, Lisbon, Portugal.

He is Area Editor of the Signal Processing: Image Communication Journal and Associate Editor of the EURASIP Journal on Image and Video Processing, and is or has been member of the Editorial Board of the Signal Processing Magazine, Associate Editor of IEEE Transactions of Circuits and Systems for Video Technology, IEEE Transactions on Image Processing, IEEE Transactions on Multimedia, and IEEE Signal Processing Magazine. In 2013-2015, he was the Editor-in-Chief of the IEEE Journal of Selected Topics in Signal Processing.

He was an IEEE Distinguished Lecturer in 2005 and elected as an IEEE Fellow in 2008 for “contributions to object-based digital video representation technologies and standards”. He has been elected to serve on the Signal Processing Society Board of Governors in the capacity of Member-at-Large for a 2012 and a 2014-2016 term. He was the SPS Vice-President for Conferences in 2018-2020.

Since 2013, he is also a EURASIP Fellow for “contributions to digital video representation technologies and standards”. He has been elected to serve on the European Signal Processing Society Board of Directors for a 2015-2018 term. Since 2015, he is also an IET Fellow.

He has been the MPEG Requirements Subgroup Chair from 2002 to 2007 and since February 2016 he is the JPEG Requirements Subgroup Chair.

He is/has been a member of the Scientific and Program Committees of many international conferences and workshops. He has been the General Chair of the Picture Coding Symposium (PCS) in 2007, the Technical Program Co-Chair of the Int. Conference on Image Processing (ICIP) in 2010 and 2016, the Technical Program Chair of the International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) in 2008 and 2012, and the General Chair of the International Conference on Quality of Multimedia Experience (QoMEX) in 2016.

He has contributed more than 300 papers in international journals, conferences and workshops, and made several tens of invited talks at conferences and workshops. His areas of interest are visual data analysis, coding, description, adaptation, quality assessment and advanced multimedia services.

Fernando Pereira

Instituto Superior Técnico, Universidade de Lisboa - Instituto de Telecomunicações

Av. Rovisco Pais, 1049-001 Lisboa, PORTUGAL