Intership Inria Lyon / INSA Lyon : Reinforcement Learning for Stochastic Resource Allocation in 6G Networks

20 Novembre 2023

---

Catégorie : Stagiaire

---

Context:

Internet of things (IoT) networks as well as 6G networks are expected to support a much higher number of devices compared to current networks. A key challenge of the deployment of such network is to avoid as much as possible collision between packets of the different devices. To this end, it is crucial to make a better utilization of the available resources (frequency band, time slots, power level...) that can be used for transmissions.

Currently, in wireless network, before transmitting data, devices need to ask for a grant (an exclusive allocation of resources) to the base station, and once they have this authorization, they can transmit data on the resources. This scheme is well suited is the device has lots of data to send or receive (for example, performing a video call). However in the context of Internet of Things, the transmission of the request of a grant can be longer than the actual data the device wants to send (for example, a smart sensor sending a temperature), making this scheme inefficient. A protocol, better suited for IoT networks, called grant free random access (GFRA) has been proposed and shows improvement in the energy efficiency of the devices. This protocol relies on the widely used assumption that devices in the network are equally likely to transmit and are statistically independent of each other. But this assumption is not holding in most of practical cases.

Assignment:

The objective of this internship, is, in line with some previous work done in our team, to investigate the possibility of modifying the GFRA protocol in order to exploit the different probability of transmissions or the possible correlation between devices. To this end, we wish to use reinforcement learning techniques to learn which resources should be assigned to which devices.

Objective:

The first step of the internship will be to formalize a reinforcement learning problem, where the environment, actions, rewards and states are clearly defined. Based on that, different algorithms (policy gradient, Q learning...) for solving RL problems will be tested and evaluated. Comparisons with the commonly used methods for resource allocation will also be carried. If time permits and depending on the wishes of the intern, either a mathematical regret analysis of the algorithms or an implementation of the proposed algorithms on the experiment platform of the lab (Cortex Lab) can be considered.