Proposition de stage M2 à Sophia Antipolis: Spiking neural networks for event-based stereovision, dans le cadre d'un projet européen. Thème: machine learning bio-inspiré, SNN, caméras évènementielles. Possibilité de continuer en thèse.
Spiking neural networks for event-based stereovision Master 2 internship proposal
This internship proposal takes part in the European CHIST-ERA APROVIS3D project (start scheduled for spring 2020) on the topic of bio-inspired machine learning for stereo event-based vision, using mixed analog-digital hardware.
In less than a decade, deep Artificial Neural Networks such as Inception and VGG-16 have successfully pulled state-of- the-are image classification performances to new levels on challenging computer vision benchmarks like ImageNet. The availability of both tremendous amounts of annotated data and huge computational resources have enabled remarkable progress. Therefore, this success comes with substantial human cost required for manually labeling data, and energy cost required for inference.
Spiking Neural Networks (SNN) represent a special class of artificial neural networks, where neurons communicate by sequences of spikes. Contrary to deep convolutional networks, spiking neurons do not fire at each propagation cycle, but rather fire only when their activation level (or membrane potential, an intrinsic quality of the neuron related to its membrane electrical charge) reaches a specific threshold value. When a neuron fires, it generates a non-binary signal that travels to other neurons, which in turn increases their potentials. The activation level either increases with incoming spikes, or decays over time. Regarding inference, SNN does not rely on stochastic gradient descent and backpropagation. Instead, neurons are connected through synapses, that implement a learning mechanism inspired from biology: it rests upon the “Spike-Timing-Dependent Plasticity”, a rule that updates synaptic weights (strength of connections) according to causal links observed between presynaptic and postsynaptic spikes. This updating rule reinforces incoming connections that cause the neuron to fire. Therefore, the learning process is intrinsically not supervised, and can be successfully used detect patterns in data in an unsupervised manner.
Beside static images, because of their asynchronous operation principle, SNN are allegedly likely to handle well temporal data such as video. Event-based cameras (or silicon retinas) bring a new vision paradigm by mimicking the biological retina. Instead of measuring the intensity of every pixel in a fixed time interval, it reports events of significant pixel intensity changes. Every such event is represented by its position, sign of change, and timestamp, accurate to the microsecond. Because of their asynchronous operation principle, they are a natural match for SNN.
State-of-the-art approaches in machine learning provide excellent results for vision tasks with standard cameras, however, asynchronous event sequences require special handling, and spiking networks can take advantage of this asynchrony. Moreover, primates and other mammals are given the ability to compute depth information from views acquired simultaneously from different points in space with stereopsis, which is a fundamental feature in environment 3D sensing. Bio-inspired models from binocular vision have also been used to solve the event-based stereo correspondence problem in the literature.
The objective of this internship is to design and implement Spiking-Neural-Network-based machine learning methods to extract vision features and infer useful information about the visual scene from stereo event-based cameras. In particular, the work will focus on two use cases: scene segmentation and depth estimation. The work will be carried out in collaboration with a leading neuroscience institute in Marseille, the Institute of Neuroscience of la Timone, that will be part of the supervision team.
Spiking Neural Networks, Event cameras, Computer Vision, Pattern Recognition
Université Côté d’Azur, Sophia Antipolis (Nice area) France
Type of contract
Temporary internship, 4-6 months
Master 2 in Computer Science (Machine Learning, Computer Vision, AI) or Computational Neuroscience
Programming skills in Python/C++, interest in research, machine learning, bio-inspiration and neurosciences are required.
Standard French “gratification” by CNRS (550 € per month)
Offer starting date
March 1, 2020 or before
Feb 10, 2020
How to apply
Send a resume (CV), transcript of grades, and motivation letter to the contact given below
Dr. Alexandre Muzy, I3S, CNRS
Dr. Laurent Perrinet, Institute of Neuroscience of la Timone, CNRS
Prof. Jean Martinet
Université Côte d'Azur / I3S / CNRS | Polytech Nice Sophia Campus SophiaTech, Sophia-Antipolis firstname.lastname@example.org, +334-89-15-43-86
(c) GdR 720 ISIS - CNRS - 2011-2020.