Coordination of Communication in Robot Teams by Reinforcement Learning - Foundations on Natural and Artificial Computation - page 162

Thus, we can conclude that on average reinforcement techniques provide a faster
convergence than the evolutionary strategies used in our previous work [5,6].

5 Conclusions and Further Research Work

In previous work [5,6] we showed that evolutionary algorithms can solve the
optimal Saussurean communication coordination problem in populations of au-
tonomous agents.
After the experimental results reported in this paper we can conclude that
Reinforcement Learning (RL) techniques, besides the advantage of their on-line
nature, are ecient enough to solve the communication coordination problem
for teams of autonomous robots, and even provide a faster convergence to the
optimal solution than evolutionary strategies. It is important to notice that we
have focused our experiments on obtaining a strict optimal Saussurean com-
munication system (i.e. we have established a 100% communication consensus
in the robot team for considering the convergence as reached, a truly strong
restriction).
As we have focused our experiments on the exclusive use of deterministic
RL algorithms we plan for the future to experiment also with probabilistic RL
algorithms, including the well-known stochastic learning automata methods, to
test their eciency on solving the strict optimal Saussurean communication
coordination problem.
After our previous experiments with both evolutionary algorithms and with
RL methods we can face our next step towards the development and building
of a physical multi-robot system based on machine vision for the cognitive part
(i.e. for the acquisition and processing of the sensory information related to the
meanings of the robots' language) and also based on the use of sound synthesizers
for the implementation of the symbols and signals emitted by the robots as their
artificial language's words.

References

1. Dorigo, M., Stutzle, T.: Ant Colony Optimization. The MIT Press, Cambridge
(2004)
2. Lenaerts, T., Jansen, B., Tuyls, K., De Vylder, B.: The evolutionary language
game: An orthogonal approach. J. Theor. Biol. 235, 566-582 (2005)
3. Lewis, D.K.: Convention. Harvard University Press (1969)
4. Loula, A., Gudwin, R., El-Hani, C.N., Queiroz, J.: Emergence of self-organized
symbol-based communication in artificial cretures. Cognitive Systems Re-

search 11(2), 131-147 (2010)
5. Maravall, D., de Lope, J., Domınguez, R.: Self-emergence of lexicon consensus
in a population of autonomous agents by means of evolutionary strategies. In:
Corchado, E., Grana Romay, M., Manhaes Savio, A. (eds.) HAIS 2010. LNCS,
vol. 6077, pp. 77-84. Springer, Heidelberg (2010)
6. Maravall, D., de Lope, J., Domınguez, R.: Self-emergence of a common lexicon by
evolution in teams of autonomous agents. Neurocomputing (in press)