Database Reference
In-Depth Information
To devise a GA for computing a partition of the set
S
such that the approximation
error of the corresponding factorization be minimized, we consider a partition of
cardinality
m
as a function
S ! m
. The fitness function is taken to be some norm
of the residual of an equivalent transformation of the Bellman equation evaluated at
an approximate solution in terms of the corresponding aggregation. Of course, there
is no way to evaluate such a residual directly in a model-free framework. It is,
however, possible to estimate the residual in virtue of the temporal differences.
As of the crossover step, we propose the common approach of interleaving
function values at random elements. More precisely,
to cross two partitions
f S
according to a distribution which
favors vectors with approximately equal numbers of entries with value 0 and 1.
The crossing
h
of
f,g
is then determined as
f,g
, we randomly pick a bit vector
b
0
;
∈
f ðÞ
,
b ðÞ¼
1,
h ðÞ:¼
g ðÞ
,
b ðÞ¼
0
:
The mutation may be performed with respect to the
Hamming metric
s
f ðÞ6¼ g ðÞ
d
H
f
ðÞ:¼
;
which is well known in the information and coding community. Specifically, for
each individual
f
, we pick a random element of the metric ball
: S !
m
d
H
f
g
j
ðÞε
;
for some small positive integer
ε
stipulated beforehand.
10.5.2 Switching Between Aggregation Bases
In the course of an adaptive computation of the partition underlying the aggregation
prolongator, the computation of the state-value function needs to be restarted for
each newly obtained partition. This gives rise to the question of whether the last
iterate with respect to the previous aggregation may somehow be exploited to
obtain a reasonable initial guess for the restarted iteration.
Given two partitions
G
β
no
G
β
of
S
with corresponding aggregation
β
∈
m
,
β
∈
m
prolongators
U
,
U
and a core matrix (tensor)
Θ
∈R
S
m
, what can be considered a
faithful representation of
Θ
U
in terms of
U
? We resort to a least squares approach
F
UΘ
UΘ
T
T
ð
10
:
12
Þ
min
Θ
