Information Technology Reference
In-Depth Information
On the other side from rational behavior there are some approaches based in
emotion modeling. These have the advantage to behave more human-like than
rational agents -property that is sometimes needed-. Furthermore emotions can
bias an agent decision, thus emotions can be seen as heuristics as well. The work
described in [6], based on
2apl
, is very clarifying. It does not provide a clear
mechanism for cooperation, but it doesn't prevent it either. The environment is
not restricted here, but since it has a strong basis on psychological models, the
agents are supposedly able to observe other agents actions and emotions.
Also in the reinforcement learning (RL) literature we can find an algorithm
which explicitly tackles the problem of exploration - exploitation: how much to
explore and when should the agent use the gathered information. The algorithm
E
3
[7]- does not have strong restrictions on
the environment, only that it should be static. That -and the fact that it does
not work with multiple agents- renders the algorithm unsuitable for MAL and
adaptation. Still it is a very significant approach to the specific treatment of the
probability of exploration.
The closest algorithm in spirit to the work presented here is the WoLF al-
gorithm [8] -there is a revision of the algorithm to allow high-dimensionality
environments [9]. Both require the agent to know its payoff matrix, and both al-
low dynamic environments since it
learns faster
when results are not as expected.
There is, though, a problem with the convergence of learning: if the agents do
not follow stationary strategies the algorithm will not ensure convergence.
Although there is some literature on
Social Reinforcement
[10,11], our ap-
proach is completely different. The reinforcement will always and only be given
by the environment, and the society will just give its opinion, aiming this opinion
towards changing the behavior of the rest of the agents.
3
-or extended version
MDP − E
3
Agent Modeling
3.1 Psychological Model
The psychological model is based in the OCC [2] emotional model. In human
emotional modeling we use emotions -plus some other factors- as the inputs
which govern the opinion the individual will give to the society. The opinions
of the people -which are in this individual's social network- will influence his
KAI index. The KAI index will determine whether to behave innovatively or
adaptively. The outcomes of the actions of the individual, again, will be evaluated
through his emotional system to produce an emotional response, which is a set
of emotions and their intensities. Figure 1(a) summarizes the relationships.
Another important step is the way each individual interprets the outcomes
of their actions. In Figure 1(b) a set of 11 functions are shown which transform
the
emotional stimuli
to a set of 11 emotions with intensities which is just the
emotional response
. This set of functions is peculiar for each person, so it is part
of their
personality
-which is invariant. The functions domains must follow the
guidelines described by the OCC model, such as some events should influence
some emotions and not others.
