Detection of "novelty" in images and videos
Directors: Donatello CONTE, Romain RAVEAUX
Supervisor: Julien OLIVIER
Laboratoire d’Informatique Fondamentale et Appliquées de Tours (LIFAT https://lifat.univ-tours.fr/). Computer science laboratory of Tours in France.
Keywords: Object detection, deep learning, generative antagonistic networks, graph-based representation.
This thesis topic is currently the subject of a request for CIFRE funding from the ANRT. Its validity is dependent on the subject's acceptance.
The selected research fellow will share his / her research time between the laboratory and a local company (50% / 50%) in Tours, France. It is a joint proposal from LIFAT laboratory's Pattern Recognition and Image Analysis team (http://www.rfai.lifat.univ-tours.fr/) and the OTODO company (https://www.otodo.com/).
Proven scientific interest on computer vision is a criteria for the selection as well as experience on programming skills. Experience on Tensoflow or PyTorch frameworks programming is considered as a plus for this position. Links on computer vision or machine learning projects (github, ...) made by the candidate is a plus too.
At the time of the application, eligible candidates should have a Master. degree in Computer Science, Engineering or related fields.
The call will remain open until the position is filled but a first deadline for evaluation of candidates will be December 15th, 2019. The position will be approved pending signature of the private partner.
The candidate will have the opportunity to work on realistic problems involving a private partner with the aim to create a novel platform able to understand the semantic scene structure of the environment using an RGB video stream from a smartphone.
This subject of thesis is placed in the field of object recognition, and more particularly the detection of novelty. The ability of an algorithm to recognize unknown inputs is important for many classification-based systems. Ideally a classifier, in addition to its generalization ability, should be able to say "This object is new, it belongs to an unknown class".
The aim of this thesis is to explore new learning technologies, in particular Generative Adversarial Network (GAN) networks, to solve both the problem of classification of an object and the detection of novelty. The classification algorithm must therefore be able to determine if an input can be associated to a known class, or if it corresponds to an unknown category (making its association to a specific class impossible). The inputs can be as well images as videos.
Although the novelty detection problem is already covered in the literature ([1-3]), it is still very open from a scientific point of view.
The main contributions of the thesis will focus on:
GAN exploration to define a new GAN architecture for the novelty detection problem (inspired by ). The research on GAN architectures for videos [5-7, 12], quite recent could be of great interest.
The definition of an original framework for incrementality and for increasing data from novelty detection. For example: once an entry is recognized as new, it can be added to known classes. The classifier will then be able to be trained with this new class by benefiting from similar data generated by the GAN (see [10-11, 13]).
Learning a structural representation of the scene (image or video) to detect a new object in relation to a usual scene (see ). This contribution will study the GANs for the generation of discrete structures (graphs) (inspired by ). Novelty is detected thanks to the neighborhood objects within the scene and a knowledge graph.
In case of application, put the three contact addresses in copy. Please send the application including a CV.
 M. A. F. Pimentel, D. A. Clifton, L. Clifton, and L. Tarassenko. Review: A review of novelty detection. Signal Process., 99, June 2014.
 Blanchard, G., Lee, G., & Scott, C. (2010). Semi-supervised novelty detection. Journal of Machine Learning Research, 11(Nov), 2973-3009.
 Aghazadeh, O., Sullivan, J., & Carlsson, S. (2011, June). Novelty detection from an ego-centric perspective. In CVPR 2011 (pp. 3297-3304). IEEE.
 Kliger, M., & Fleishman, S. (2018). Novelty detection with gan. arXiv preprint arXiv:1802.10560.
 Ravanbakhsh, M., Nabi, M., Sangineto, E., Marcenaro, L., Regazzoni, C., & Sebe, N. (2017, September). Abnormal event detection in videos using generative adversarial nets. In 2017 IEEE International Conference on Image Processing (ICIP) (pp. 1577-1581). IEEE.
 C. Vondrick, H. Pirsiavash, and A. Torralba. Generating videos with scene dynamics. In Advances in Neural Information Processing Systems (NIPS), 2016.
 Tulyakov, S., Liu, M. Y., Yang, X., & Kautz, J. (2018). Mocogan: Decomposing motion and content for video generation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1526-1535).
 Dogra, D. P., Reddy, R. D., Subramanyam, K. S., Ahmed, A., & Bhaskar, H. (2015, March). Scene Representation and Anomalous Activity Detection using Weighted Region Association Graph. In VISAPP (2) (pp. 104-112).
 Wang, H., Wang, J., Wang, J., Zhao, M., Zhang, W., Zhang, F., ... & Guo, M. (2018, April). Graphgan: Graph representation learning with generative adversarial nets. In Thirty-Second AAAI Conference on Artificial Intelligence.
 Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., …, & Fu, Y. (2018). Incremental classifier learning with generative adversarial networks. arXiv preprint arXiv:1802.00853.
 Antoniou, A., Storkey, A., & Edwards, H. (2017). Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04340.
 Aidan Clark, Jeff Donahue, Karen Simonyan. (2019). Efficient Video Generation on Complex Datasets. https://arxiv.org/abs/1907.06571
 Mohammad Sabokrou, Mohammad Khalooei, Mahmood Fathy, Ehsan Adeli. (2018). Adversarially Learned One-Class Classifier for Novelty Detection. https://arxiv.org/abs/1802.09088
(c) GdR 720 ISIS - CNRS - 2011-2020.