Reinforcement Learning for Self-organizing Wake-Up Scheduling inWireless Sensor Networks - Agents and Artificial Intelligence - page 377

Reinforcement Learning for Self-organizing Wake-Up
Scheduling in Wireless Sensor Networks

Mihail Mihaylov 1 , Yann-Ael Le Borgne 1 , Karl Tuyls 2 ,andAnnNowe 1

1 Vrije Universiteit Brussel, Brussels, Belgium
2 Maastricht University, Maastricht, The Netherlands
{mmihaylo,yleborgn,ann.nowe}@vub.ac.be
[e-mail]

Abstract. Wake-up scheduling is a challenging problem in wireless sensor net-
works. It was recently shown that a promising approach for solving this problem
is to rely on reinforcement learning (RL). The RL approach is particularly at-
tractive since it allows the sensor nodes to coordinate through local interactions
alone, without the need of central mediator or any form of explicit coordination.
This article extends previous work by experimentally studying the behavior of RL
wake-up scheduling on a set of three different network topologies, namely line,
mesh and grid topologies. The experiments are run using OMNET++, a the state-
of-the-art network simulator. The obtained results show how simple and compu-
tationally bounded sensor nodes are able to coordinate their wake-up cycles in
a distributed way in order to improve the global system performance. The main
insight of these experiments is to show that sensor nodes learn to synchronize if
they have to cooperate for forwarding data, and learn to desynchronize in order
to avoid interferences. This synchronization/desynchronization behavior, referred
to for short as (de)synchronicity, allows to improve the message throughput even
for very low duty cycles.

Keywords: Reinforcement learning, Synchronicity and desynchronicity, Wire-
less sensor networks, Wake-up scheduling.

1 Introduction

A Wireless Sensor Network is a collection of densely deployed autonomous devices,
called sensor nodes , which gather data with the help of sensors [4]. The untethered
nodes use radio communication to transmit sensor measurements to a terminal node,
called the sink . The sink is the access point of the observer, who is able to process
the distributed measurements and obtain useful information about the monitored en-
vironment. Sensor nodes communicate over a wireless medium, by using a multi-hop
communication protocol that allows data packets to be forwarded by neighboring nodes
to the sink. A typical multi-hop communication protocol is to rely on a shortest path
tree with respect to the hop distance [4]. Such a tree is obtained by letting nodes broad-
cast packets after deployment, in order identify their neighbors. The nodes then deter-
mine the neighbor node which is the closest (in terms of hops) to the sink, and use it