Information Technology Reference
In-Depth Information
4.2
ACogSim
Providing an environment for the evaluation of continual planning systems is not a
trivial task [2]. We implemented a simulator, namely
ACogSim
, for the environment
in order to make it possible to systematically evaluate the whole high-level control
architecture—including execution—described in Section 3. The ACogSim simulator
works similar to MAPSIM as described in [2]. In contrast to the agent, ACogSim has a
complete model of the domain. When the executor executes an action, then the action
is sent to ACogSim. ACogSim checks the precondition of actions at runtime prior to
the execution and updates its simulation model according to the effect of the actions. In
this way ACogSim simulates the execution of actions and guarantees that the executed
plans are correct.
The outcome of sensing actions is also simulated by ACogSim. Let
D
Msim
be the
(complete) domain model of the ACogSim instance. The result of a sensing action
sense
(
l,I,C,ks
)
is an additional instance
lσ
of
l
if such an instance can be derived
with respect to
D
Msim
;
impossible
if it can be derived that the existence of an additional
instance of
l
is impossible; or
indeterminable
otherwise.
4.3
Performing Tasks with a Decreasing Amount of Initial Knowledge
We used ACogSim in order to evaluate the behavior of the overall control system for
several domains. The objective of the conducted experiments is to determine the behav-
ior of the system in situations where an agent needs additional information to perform
a given task, but sufficient information can in principle be acquired by the agent.
Setup.
We used an adapted version of the rover domain with 1756 facts and an instance
of the depots domain with 880 facts from IPC planning competition 2002; an instance
of an adapted blocks world domain with 2050 facts; and a restaurant (109 facts) and an
office domain (88 facts) used to control a mobile service robot.
All domain model instances contain sufficient information to generate a complete
plan without the need to acquire additional information. The simulator (ACogSim) is
equipped with a complete domain model. In contrast, the agent has only an incomplete
domain model where a set of facts has randomly been removed. For each domain the
agent always had to perform the same task.
The objective of this experimental setup is to get deeper insights into the performance
of the proposed control system. In particular, we are interested in finding an answer to
the following questions: Is ACogControl always able to perform the given task? How
often switches ACogControl between planning and acting? How much time is necessary
for the whole planning and reasoning process? How long is an average planning phase?
How does the performance change with a decreasing amount of initial knowledge?
We conducted 10 experiments for all domains with 1000 runs per experiment, except
for the last experiment where 1 run was sufficient. Let
f
all
be the number of facts in
i
a domain, then
10
f
all
facts were removed in all runs of the
i
th experiment from the
domain model of the agent. Hence, in the last experiment all facts are removed (for
each domain) from the agent's domain model.
