Annonce
Thesis offer:Uncertainty quantification for machine and deep learning techniques
27 Avril 2023
Catégorie : Doctorant
PhD title: Uncertainty quantification for machine and deep learning techniques.
Host laboratory: FEMTO-ST
Speciality of PhD: Engineering science
Keywords: Uncertainty quantification, Neural networks, Diagnosis, Breast Cancer.
Context and motivation: Most of the real physical system and everyday situations include uncertainty. This is the case for medical diagnosis, weather forecasting, evolution of the stock market and so on. In the literature two types of uncertainty are distinguished: aleatoric uncertainty denotes the one that is inherent to the data, e.g., noise in measurements or natural variability of the inputs, and epistemic uncertainty related to the model and due to lack of knowledge. Measuring the uncertainty is important, so as to support the user in the action to take. For example, when an anomaly is detected, with weak confidence level, another source of information should be added (image, human intervention, etc.) before planning intervention actions. More generally, quantification of the prediction uncertainty allows to trust or not predictions. In fact, incorrect overconfident predictions can be harmful and lead to erroneous decision.
Goal of the thesis: The goal of this thesis is to develop a robust method to evaluate uncertainty for machine and deep learning algorithm predictions. Major of works focused on improving the algorithm performance, few works deal with measuring the uncertainty related to the predictions. In particular in this thesis we want to relax some hypothesis in the existing approach related to the distribution of the data and symmetry of the algorithm. This subject is challenging with many theoretical and applicatives difficulties. It is multidisciplinary including competences in probability, statistic and data processing. The two principal goal are: First, we aim to measure the impact of uncertainty miss evaluation on the decision. The second part is focused on developing new method to quantify uncertainty, that can be applied to different type of data and without restrictive constraint on distribution or the exchangeability. The third part, includes generalization of the proposed method when we have noisy and/or missing data.
The second part include study of the theoretical aspects: proof of convergence, complexityissue. In addition to practical aspects: independence from the chosen algorithm, architecture of the NN, implementation... Finally, a validation criterion is defined to attest the performance of the uncertainty measure.
Application: In this thesis, the aim is to propose a general approach in the sense to be applied for different machine and deep learning techniques. Moreover, the particular context of breast cancer diagnostic is considered to get proof of concept of the method. This thesis aims to: 1) Quantify the impact of ignoring the uncertainties on the decision. 2) Provide accurate uncertainty measure to get supplementary information to support the decision about the patient score. 3) Deal with noisy and missing data when evaluating uncertainties.
Bibliography:
[1] Joaquin Quinonero-Candela, Carl Edward Rasmussen, Fabian Sinz, Olivier Bousquet, and Bernhard Scholkopf, “Evaluating predictive uncertainty challenge,” Lecture Notes in Computer Science, vol. 3944, pp. 1–27, 2006.
[2] D J C. MacKay, “A practical bayesian framework for back-propagation networks.neural computation,” Neural computation, vol. 4, no. 3, 1992.
[3] Radford M Neal, Bayesian learning for neural networks, vol. 118, Springer Science & Business Media, 2012.
[4] Yarin Gal and Zoubin Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in international conference on machine learning. PMLR, 2016, pp. 1050–1059.
[5] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” Advances in neural linformation processing systems, vol. 30, 2017.
[6] John Platt et al., “Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods,” Advances in large margin classifiers, vol. 10, no. 3, pp. 61–74, 1999.
[7] Kuleshov.V, Fenner. N, and Ermon. S, “Accurate uncertainties for deep learning using calibrated regression,” CoRR, 2018.
[8] Gammerman. A Vovk. V and Shafer. G, Algorithmic Learning in a Random World, Springer-Verlag, Berlin, Heidelberg, 2005.
[9] A. N. Angelopoulos and S. Bates, “A gentle introduction to conformal prediction and distribution-free uncertainty quantification,” CoRR, 2021.
[10] Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi, “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” Information Fusion, vol. 76, pp. 243–297, 2021.
Applicant profile: Master in applied mathematics (or equivalent). Probability, statistic.
Good skills in Python programming. Experience in machine learning/deep learning
Financing Institution: MESRI
Application deadline: 15 June 2023
Start of contract: Fall 2023
Thesis Supervisor(s): Noura Dridi (noura.dridi@ens2m.fr), Zeina Al Masry (zeina.al.masry@ens2m.fr), Noureddine Zerhouni (zer- houni@ens2m.fr)
How to apply: Please send a motivation letter, recommendation letters, a detailed CV and transcript of results to the above email addresses