Information Technology Reference
In-Depth Information
irregular data traffic, since nodes tightly adapt their schedules to match the expected
traffic. For the same reason, DESYDE is also vulnerable to clock drifts. The approach
presented in this paper, on the other hand, is more flexible in these respects, since most
nodes remain active longer than it is necessary to forward their packets.
4
Conclusions
In this paper we presented a decentralized reinforcement learning (RL) approach for
self-organizing wake-up scheduling in wireless sensor networks (WSNs). Our approach
improves the throughput of the system even for very low duty cycles, as compared to
the standard S-MAC protocol. When using our RL policy, agents independently learn
to synchronize their active periods with nodes on the same routing branch, so that mes-
sage throughput is improved. At the same time, nodes desynchronize with other routing
branches in order to reduce communication interference. We demonstrated how initially
randomized wake-up schedules successfully converge to the state of (de)synchronicity
based only on local interactions and without any form of explicit coordination. As a re-
sult, our approach makes it possible that sensor node coordination
emerges
rather than
is
agreed
upon.
The proposed approach provides a basis for a number of extensions that we are cur-
rently investigating. In particular, the wake-up schedules of individual nodes may be
adapted on the basis of their own traffic load, as illustrated by the DESYDE strategy.
This adapted version of the protocol allows to further reduce the convergence time and
the end-to-end latency of the system.
Acknowledgements.
The authors of this paper would like to thank the anonymous re-
viewers for their useful comments and valuable suggestions. This research is funded
by the agency for Innovation by Science and Technology (IWT), project DiCoMAS
(IWT60837); and by the Research Foundation - Flanders, Belgium (FWO), project
G.0219.09N.
References
1. CC2420: Data sheet,
2. Cohen, R., Kapchits, B.: An optimal wake-up scheduling algorithm for minimizing energy
consumption while limiting maximum delay in a mesh sensor network. IEEE/ACM Trans.
Netw. 17(2), 570-581 (2009)
3. Gutierrez, J., Naeve, M., Callaway, E., Bourgeois, M., Mitter, V., Heile, B.: IEEE 802.15. 4:
a developing standard for low-power low-cost wireless personal area networks. IEEE Net-
work 15(5), 12-19 (2002)
4. Ilyas, M., Mahgoub, I.: Handbook of sensor networks: compact wireless and wired sensing
systems. CRC (2005)
5. Langendoen, K.: Medium access control in wireless sensor networks. Medium Access Con-
trol in Wireless Networks 2, 535-560 (2008)
6. Leng, J.: Reinforcement learning and convergence analysis with applications to agent-based
systems. Ph.D. thesis, University of South Australia (2008)
