Database Reference
In-Depth Information
12.2 The Realtime Analytics Framework of XELOPES
Neither the CWM nor the PMML standard supports realtime analytics functions. So
it was newly specified in XELOPES extending the existing framework. We first
give an introduction to the agent framework of XELOPES which is more general
than reinforcement learning only. Based on this, we then explain the reinforcement
package and finally the recommendation package which in turn extends the RL.
12.2.1 The Agent Framework
The agent framework which is implemented for every single agent in XELOPES is
inspired by the agent ansatz of artificial intelligence (see [RN02]). The heart of this
framework is the
Agent
, an object which interacts with an environment. This agent
consists of sensors to receive stimuli from the environment and actuators to perform
actions inside of the environment and with it response to the received stimuli.
We mention that nearly everything can be explained in terms of such agent
theory. For example, think of a calculator which gets the stimulus “2 + 2” and,
as a result, responses with the action “4.” Despite to this, we will consider the
agent concept for the analysis of systems mainly. In this context, examples
for agents are given through pack robots, interactive English teachers, systems
for medical diagnostics, and many more. To determine how the agent should
respond to a certain stimulus, some rules have to be introduced. Dependent on
the environment, especially the number of different stimuli, this can lead to an
innumerable number of rules that cannot even be stored on the best performing
computers of nowadays.
To help this out, rules are defined which handle more than one stimulus, actions
are chosen also randomly, and reward functions are defined. Randomly chosen
actions are actually necessary for environments which are not fully observable
which implicates that not the whole variety of possible stimuli is known. A reward
function measures the success of an action through a reward which, for instance,
could be a real number and is communicated to the agent. The aim of the agent is to
maximize this reward. Through the corresponding value function, the agent is able
to rate possible choices of actions in response to a stimulus. This allows him to
make a reasonable decision. The storing and applying of the corresponding rules are
described as learning, since these rules are not initially given.
The agent described until now is a so-called stateless agent. In contrast to this,
we can also consider stateful agents. These agents include an additional attribute,
the state. The state of the agent can change as a result of an action. The current state
of the agent is taken into account during the decision process.
The XELOPES
Agent
package consists of several utilities which provide a unique
access to the realtime applications of the XELOPES. We need to mention that the
environment discussed above, by now, is not modeled in XELOPES but will pre-
sumably be added in a future version. This means stimuli of the environment are only
