Reinforcement Learning for Self-organizing Wake-Up Scheduling inWireless Sensor Networks - Agents and Artificial Intelligence - page 386

(a) Mesh topology.

(b) An example of a learned wake-up schedule for duty
cycle of 5%.

(c) Average end-to-end latency for different duty cycles.

Fig. 4. Experimental results for the mesh topology

cycle of 5% after the actions of agents converge. One can observe that the state of
desynchronicity has been successfully achieved where each node is active at a different
time within a frame. Put differently, each node has chosen a different wake-up slot and
therefore belongs to different coalition. The benefit of this desynchronized pattern is
clearly evident in Figure 4(c) where we compare it to the average end-to-end latency
of the synchronized system. Error bars represent one standard deviation across 30 runs.
Since all nodes lie within one hop of the sink, the performance of the learning agents
is not dependent on the duty cycle for this topology. Each node independently learns to
hold its active period immediately after it generates a message, as long as no neighbor is
awake at the same time. Therefore, the average end-to-end latency is slightly more than
the duration of one transmission. Similarly to the line topology, when nodes use the
S-MAC protocol, the end-to-end latency of the system is on average half the sampling
period, for reasons outlined above. Moreover, for duty cycle of 0 . 5% , the S-MAC nodes
are unable to deliver their messages to the sink, since all nodes try to transmit during
the same short awake period and thus all messages collide. This effect is indicated with
the discontinued dashed line in Figure 4(c).
Lastly, we investigate a combination of the above two topologies, namely the grid
shown in Figure 5(a). Nodes here need to synchronize with those that lie on the same
branch of the routing tree to ensure high throughput, while at the same time desyn-
chronize with neighboring routing branches to avoid communication interference. An
example of the wake-up schedule of the learning nodes at 5% duty cycle is displayed in
Figure 5(b). As expected, the four columns of nodes belong to four different coalitions,
where nodes in one coalition are synchronized with each other (being active nearly at
the same time) and desynchronized with the other coalitions (sleeping while others are
active). This is the state of (de)synchronicity. Nodes in one coalition exhibit comparable
behavior to those in a line topology, i.e. they have synchronized with each other (while