Ph.D. Thesis Proposal: CausalSpatio-temporal Mathematical Morphology: Theory and Algorithms
Scientific fields: Applied mathematics and Computer Science – Interaction between nonlinear video sequence processing and human-vision inspired information processing.
Supervision: This Ph.D. Thesis will be carried out at the Center for Mathematical Morphology (CMM) at MINES ParisTech, in France. The Ph.D. Thesis will be supervised by Jesus Angulo, email@example.com.
Application: Candidates should send a CV, a cover letter and the grades obtained during the last two years.
Knowledge and skill requirements: The applicant must have a sound knowledge of applied mathematics (linear analysis, metric and Riemannian geometry, partial differential equations, probabilities and statistics, physical modeling, etc), together with a background in image processing and machine learning, validated by a Research Master in this subject. He/she will also have to have a skill in computer science, more particularly in Python and eventually in C/C++ programming.
A sound command of spoken and written English is an absolute requirement. In addition, an experience in a foreign laboratory would be an asset for this position.
Video sequences and similar time-indexed datasets are mainly processed either as 3D (homogenous product of space x time) images or as a set of independent 2D images, without taking into account the mathematical nature of the spatio-temporal space, going from the causality to the existence of a underlying motion vector field which modifies (locally) the metric. Even recent papers on convolutional neural networks (CNN) approaches for spatio-temporal video deep learning still focus on addressing both approaches [Xie et al., 2017]: “Is it important that we 3D convolve jointly over time and space? Or would it suffice to convolve over dimensions independently?”.
The fundamental reasons behind this limitation to genuinely tackle (space x time) are both theoretical and algorithmic.Indeed, especially in the case of real-time processing of video streams, the algorithms should be fast enough to follow the video rate and this was in the past generally incompatible with processing a buffer of previous frames. In the case of off-line processing of video sequences, the speed is not a major constraint; however it is preferred to see video sequences as (non-casual) 3D images since the theory and algorithms available in classic 3D image processing are widely available and can be applied without any change.
Classical scale-space theory is concerned with the mathematical modeling of front-end visual system behavior. In this context, inspired by measurement on the spatio-temporal responses of visual receptive fields and axiomatic formulation of scale-scale theory (i.e., invariance, semi-group structure, etc.), the scale-time kernels has been introduced to deal with real-time causal modeling [ter Haar Romeny et al., 2001]. This kind of theory has been extended to the case of spatio-temporal receptive fields and image data, in particular by considering relative motions between objects in the world and the observer, where a constant velocity translational motion can be modeled by a Galilean transformation [Lindeberg, 2013;2016]. Corresponding time-causal spatio-temporal kernels are related to a system of diffusion equations [Lindeberg, 2011]. Theoretical works on wavelets for physics have also considered the formulation of wavelets on affine (Galilean) groups space-time [Ali et al., 2014], which can be potentially used on video processing.
In the literature of mathematical morphology, there are a few works which have considered properly the formulation of spatio-temporal operators, in particular the idea of structuring elements following the optical flow [Laveau and Bernard, 2005] or sophisticated spatio-temporal structuring elements which decouple time/space connectivity [Luengo-Oroz et al., 2012]. However, despite those past contributions, the lack of a sound theoretical background and corresponding efficient algorithms involves that most of morphological methods nowadays considered for processing video sequences disregard the spatio-temporal phenomenology.
The goal of this thesis is just fill this gap by revisiting the main operators and algorithms of modern mathematical morphology for the case of functions and sets defined on the casual spatio-temporal domain.
Developing efficient time-casual and time-recursive morphological algorithms with requirements of speed, controlled complexity, etc. will need also to study during the thesis the best image representations (i.e., data structures such as tensors or graphs are the natural alternatives) which typically should incorporate also motion estimation.
[Ali et al., 2014] S.T. Ali, J.P. Antoine, J.P. Gazeau. Wavelets Related to Affine Groups. In: Coherent States, Wavelets, and Their Generalizations. Theoretical and Mathematical Physics. Springer, New York, 2014.
[Angulo and Velasco-Forero, 2014] J. Angulo and S. Velasco-Forero. Riemannian mathematical morphology. Pattern Recognition Letters, Vol. 47, 93-101, 2014.
[Angulo 2016] J. Angulo. Generalised morphological image diffusion. Nonlinear Analysis: Theory, Methods & Applications, Vol. 134, 1-30, 2016.
[Hilgert, 1991] J. Hilger. The hyperboloid as ordered symmetric space. Seminar Sophus Lie, Vol.1, 135-141, 1991
[Knuth and Bahrenyi, 2010] K.H. Knurh, N. Bahrenyi. The Order-Theoretic Origin of Special Relativity. AIP Conference Proceedings 1305, 115-121 2011.
[Laveau and Bernard, 2005] N. Laveau, C. Bernard.Structuring Elements Following the Optical Flow. In: Ronse C., Najman L., Decencière E. (eds) Mathematical Morphology: 40 Years On. Computational Imaging and Vision, vol 30. Springer, 2005.
[Lindeberg, 2011] T. Lindeberg. Generalized Gaussian scale-space axiomatic comprising linear scale-space, affine scale-space and spatio-temporal scale-space. Journal of Mathematical Imaging and Vision 40(1): 36-81, 2011.
[Lindeberg, 2013] T. Lindeberg. Invariance of visual operations at the level of receptive fields. PLOS ONE, 8(7): e66990:1-33, 2013.
[Lindeberg, 2016] T. Lindeberg. Time-causal and time-recursive receptive fields. Journal of Mathematical Imaging and Vision 55(1): 50-88, 2016.
[Luengo-Oroz et al., 2012] M.A. Luengo-Oroz, D. Pastor, C. Castro, E. Faure, T. Savy, B. Lombardot, J. Rubio-Guivernau, L. Duloquin, M. Ledesma-Carbayo, P. Bourgine, N. Peyriéras, A. Santos. 3D+t Morphological Processing: Applications to Embryogenesis Image Analysis. IEEE Transactions on Image Processing, vol. 21, no. 8, pp. 3518-3530, 2012.
[Schmidt and Weickert, 2015] M. Schmidt, J. Weickert. The Morphological Equivalents of Relativistic and Alpha-Scale-Spaces. In: Aujol JF., Nikolova M., Papadakis N. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2015. Lecture Notes in Computer Science, vol 9087. Springer, 2015.
[Santalo, 1968] L. A.Santalo. Horospheres and convex bodies in hyperbolic space. Proc.Amer.Math. Soc., Vol. 19, 390–395, 1968.
[Stott, 2016] N. Stott. Maximal Lower Bounds in the Löwner order. arXiv: 1612.05664, 2016.
[ter Haar Romeny, 2001] B.M. ter Haar Romeny, L.M.J. Florack, M.Nielsen (2001) Scale-Time Kernels and Models. In: Kerckhove M. (eds) Scale-Space and Morphology in Computer Vision. Scale-Space 2001. LNSCS vol 2106. Springer, 2001.
[Xie et al., 2017] S. Xie, C. Sun, J. Huang, Z. Tu, K. Murphy. Rethinking Spatiotemporal Feature Learning For Video Understanding. arXiv:1712.04851, 2017.
(c) GdR 720 ISIS - CNRS - 2011-2018.