Information Technology Reference
In-Depth Information
Variants B, C & D also showed experimentally that intrinsic rewards can substan-
tially accelerate goal-directed learning and
external
reward intake of agents living
in environments providing external reward for achieving desirable goal states. See
(Schmidhuber
2010
) for a more detailed overview of the work 1990-2010. There
also are more recent implementation variants with applications to vision-based rein-
forcement learning/evolutionary search (Luciw et al.
2011
, Cuccu et al.
2011
), active
learning of currently easily learnable functions (Ngo et al.
2011
), black box optimi-
sation (Schaul et al.
2011b
), and detection of “interesting” sequences of Wikipedia
articles (Schaul et al.
2011a
).
Our previous computer programs already incorporated approximations of the ba-
sic creativity principle. But do they really deserve to be viewed as rudimentary sci-
entists and artists? The works of art produced by, say, the system of (Schmidhuber
2002a
), include temporary “dances” and internal state patterns that are novel with
respect to its own limited predictors and prior knowledge, but not necessarily rel-
ative to the knowledge of sophisticated adults (although an interactive approach
using human guidance allows for obtaining art appreciated by some humans—see
Fig.
12.1
). The main difference to human scientists or artists, however, may be only
quantitative by nature, not qualitative:
1. The unknown learning algorithms of humans are presumably still better suited to
predict/compress real world data. However, there already exist
universal,
math-
ematically optimal (not necessarily practically feasible) prediction and compres-
sion algorithms (Hutter
2005
, Schmidhuber
2009d
), and ongoing research is con-
tinually producing better
practical
prediction and compression methods, waiting
to be plugged into our creativity framework.
2. Humans may have superior RL algorithms for maximising rewards generated
through compression improvements achieved by their predictors. However, there
already exist
universal,
mathematically
optimal
(but not necessarily practically
feasible) RL algorithms (Hutter
2005
, Schmidhuber
2009d
), and ongoing re-
search is continually producing better
practical
RL methods, also waiting to be
plugged into our framework.
3. Renowned human scientists and artists have had decades of training experiences
involving a multitude of high-dimensional sensory inputs and motoric outputs,
while our systems so far only had a few hours with very low-dimensional experi-
ences in limited artificial worlds. This quantitative gap, however, will narrow as
our systems scale up.
4. Human brains still have vastly more storage capacity and raw computational
power than the best artificial computers. Note, however, that this statement is un-
likely to remain true for more than a few decades—currently each decade brings
a computing hardware speed-up factor of roughly 100-1000.
Section
12.6
will demonstrate that current computational limitations of artificial
artists do not prevent us from already using the Formal Theory of Creativity in
human-computer interaction to create art appreciable by humans.
