Despite the success of deep learning architectures in many fields and particularly in computer vision, deep learning is far from being deployed in many real-world applications as there still exist many concerns related to robustness, reliability, computational time and explainability. A great boost in the performance of neural networks can be obtained through the exploitation of domain knowledge, e.g. problem specific information that can be used to improve the deep learning model and/or simplify the training process and improve model explainability. This research direction in deep learning is emerging and has been typically addressed in relative isolation and via ad’hoc approaches. The research in the thesis aims at firstly formalizing and analyzing the different branches of knowledge integration in deep learning models in the image field, possibly in presence of heterogeneous data. In a second time it aims at developing / improving novel deep learning architectures that can integrate a priori expert knowledge. Applications will be done in popular image benchmarks and databases coming from the environmental fields. As a final and ideal objective, this work intends to delineate the preliminary definition of a framework for knowledge integration unifying the different branches and devoted to the environmental field.
Knowledge integration for improving deep learning in computer vision
PRISME/LIFO (Orléans University – France)
Supervisors : F. Ros, C. Vrain, Y. Lucas, T.B.H. Dao
The internship will be supervised by researchers from two laboratories of the University of Orléans : PRISME (Laboratoire Pluridisciplinaire de Recherche en Ingénierie des Systèmes, Mécanique, Énergétique) and LIFO (Laboratoire d'Informatique Fondamentale d'Orléans). These laboratories collaborate via their complementarities in computer vision, machine learning and knowledge representation.
A PhD grant on a similar topic will be published in Spring 2021, supported by ANR (National Research Agency) and University of Orléans in the context of the project AI.iO (Artificial Intelligence in Orléans: Learning from heterogeneous data and expert knowledge. Applications in geological and environmental sciences).
This master internship aims specifically at:
- Updating the state of the art of deep learning architectures in computer vision.
- Exploring and analyzing knowledge integration in deep learning architectures devoted to images.
- Testing existing methods integrating expert knowledge in computer vision and comparing them to a baseline method integrating no expert knowledge (except the labelling of data). Use case either on emotion detection, segmentation or pattern recognition.
- If possible, an improved knowledge integration mechanism
We are looking for a candidate who is currently in Master. A particular strength or experience in computer vision, machine learning, data mining, computer programming or applied mathematics is highly appreciated. French and/or English are the working languages.
The candidates are encouraged to contact us as soon as possible. Start is expected between January and March 2021. The complete application consists of the documents below, which should be sent as a single PDF file to Frédéric Ros (firstname.lastname@example.org) and Christel Vrain (email@example.com)
• One-page cover letter (clearly indicating available start date as well as relevant qualifications, experience and motivation)
• University certificates and transcripts (both B.Sc and M.Sc degrees)
• Contact details of up to three referees
• Possibly an English language certificate and a list of publications
• All documents should be in English or French.
1. S. Aditya , Y. Yang , C. Baral. Integrating knowledge and reasoning in image understanding IJCAI International Joint Conference on Artificial Intelligence, 2019, pp. 6252-6259.
2. T.-B.-H. Dao, K.-C. Duong, C. Vrain. Constrained Clustering by Constraint Programming. Artificial Intelligence, Elsevier. Volume 244, pp. 70–94, 2017.
3. G. Hinton, O. Vinyals, J. Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
4. T. A. Lampert, B. Lafabregue, T.-B.-H. Dao, N. Serrette, C. Vrain, P. Ganç arski. Constrained Distance-Based Clustering for Satellite Image Time-Series. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(11) : 4606- 4621. 2019.
5. J. Song, S. Gao, Y. Zhu & C. Ma (2019) A survey of remote sensing image classification based on CNNs, Big Earth Data, 3:3, 232-254, DOI: 10.1080/20964471.2019.1657720
6. M. Kerkech, A. Hafiane, R. Canals, F. Ros. Vine Disease Detection by Deep Learning Method Combined with 3D Depth Information Image and Signal Processing. ICISP 2020. LNCS 12119. Springer, Cham, (2020)
7. L. Von Rueden, S. Mayer, K. Beckh et al. Informed Machine Learning--A Taxonomy and Survey of Integrating Knowledge into Learning Systems. arXiv preprint arXiv:1903.12394, 2019.
8. F. Ros, S. Guillaume, M. El-hajji, R.Riad. Kdmutual: a novel clustering algorithm combining mutual neighboring and hierarchical approaches using a new selection criterion. Knowledge Based Systems, 2020.
9. X. Xie,J. Niu, X.Liu etal. ASurveyon Domain Knowledge Powered Deep Learning for Medical ImageAnalysis.arXivpreprint arXiv:2004.12150, 2020.
For further information about the studentship, please, contact Christel Vrain (christel.vrain at univ- orleans.fr) or Frédéric Ros (Frederic.Ros@univ-orleans.fr)
(c) GdR 720 ISIS - CNRS - 2011-2020.