An Incremental Model of Lexicon Consensus in a Population of Agents by Means of Grammatical Evolution, Reinforcement Learning and Semantic Rules - Foundations on Natural and Artificial Computation - page 42

object. The second specialized behaviour is supplied to play language games.
Languages games are a very useful tool when we study the language. Some
works in robotics propose language games as a model of communication (see [7],
[8], [12] or [14]). In the system we propose here a language game is a dialogue
between two agents. During the dialogue all agents communicate each other
their current symbol-meaning mappings. The more coincidence in the mappings
the more consensus the system will attain. When all symbol-meaning mappings
are equal we can arm that a lexicon consensus has emerged. Traditionally, in
the computational implementations of language games, the dialogues are realized
between all the agents in a hand-made design. However, in this case the language
games depend on interactions between agents, i.e. only when two agents are close
enough with each other a dialogue is produced. We think this choice is more
realistic.
3 Grammatical Evolution Guided by Reinforcement and
Semantic Rules
Grammatical Evolution Guided by Reinforcement (GER) is an extension of the
Grammatical Evolution (GE) [15] and it was proposed initially in [16]. GER
tries to merge evolution and learning in a simple way: it allows that individuals
of the population can rewrite its own code several times, trying different choices
with the same BNF grammar that they used for mapping the original genotype
in the phenotype. The process in GER consists of three stages:
1. Transcription is the process that transforms the original binary string into
an integer number string.
2. Translation uses the integer number string for getting a value that represents
the rule to apply for the current non terminal in the grammar.
3. Learning stage uses a reinforcement learning mechanism for generating new
programs. Specifically the reinforcement learning used in GER is based on
Q-Learning [17].
Stages 1 and 2 were defined in GE while stage 3 was added in GER. Essentially,
the learning process in GER allows to generate new programs by means of a
reinforcement mechanism instead of using the evolutionary mechanism. The re-
inforcement is based in the Q-Learning's idea and GER implements it by means
of an structure known as the Q-Tree. The Q-Tree is similar to the traditional Q-
table, but it allows to work with a potentially larger space of states and actions.
The Q-tree supports the production rules more suitable on each state. Nodes in
the Q-tree stands for states in the process of building a derivation tree which
represents a program. Q-tree keeps the next information on each node:
- A numeric value that represents the production rules used to reach the state.
- A group of numeric values called Q-values, which represent the reward asso-
ciated with the different production rules that can be applied in the state.