Biomedical Engineering Reference
In-Depth Information
circuit used as the main building block for
Figure 18.2
)
tend to have already large
computational power, provided that the reservoir from which their units are chosen
is sufficiently diverse.
Consider a class
B
of basis filters
B
(that may for example consist of the compo-
nents that are available for building filters
L
M
of LSMs). We say that this class
B
has the
point-wise separation property
if for any two input functions
u
(
·
)
,
v
(
·
)
with
.
‡
u
(
s
)
=
v
(
s
)
for some
s
≤
t
there exists some
B
∈ B
with
(
Bu
)(
t
)
=(
Bv
)(
t
)
There
exist completely different classes
B
of filters that satisfy this point-wise separation
property:
B
=
{
all delay lines
}
,
B
=
{
all linear filters
}
, and perhaps biologically
more relevant
(see [17]).
The complementary requirement that is demanded from the class
B
=
{
models for dynamic synapses
}
of functions
from which the readout maps
f
M
are to be picked is the well-known
universal ap-
proximation property
: for any continuous function
h
and any closed and bounded
domain one can approximate
h
on this domain with any desired degree of precision
by some
f
F
∈ F
. Examples for such classes are
F
=
{
feedforward sigmoidal neural
nets
}
, and according to [3] also
F
=
{
pools of spiking neurons with analog output
in space rate coding
.
A rigorous mathematical theorem [16], states that for
any
class
}
B
of filters that
satisfies the point-wise separation property and for
any
class
of functions that sat-
isfies the universal approximation property one can approximate any given real-time
computation on time-varying inputs with fading memory (and hence any biologi-
cally relevant real-time computation) by an LSM
M
whose filter
L
M
F
is composed of
, and whose readout map
f
M
finitely many filters in
.
This theoretical result supports the following pragmatic procedure: In order to im-
plement a given real-time computation with fading memory it suffices to take a filter
L
whose dynamics is
sufficiently complex
,andtraina
sufficiently flexible
readout to
transform at any time
t
the current state
x
B
is chosen from the class
F
.In
principle a memoryless readout can do this, without knowledge of the current time
t
, provided that states
x
(
t
)=(
Lu
)(
t
)
into the target output
y
(
t
)
t
)
t
)
(
t
)
and
x
(
that require different outputs
y
(
t
)
and
y
(
are
sufficiently distinct. We refer to [16] for details.
For physical implementations of LSMs it makes more sense to analyze instead of
the theoretically relevant point-wise separation property the following quantitative
separation property as a test for the computational capability of a filter
L
:Howdif-
ferent are the liquid states
x
u
(
t
)=(
Lu
)(
t
)
and
x
v
(
t
)=(
Lv
)(
t
)
for two different input
histories
u
are
Poisson spike trains and
L
is a generic neural microcircuit model. It turns out that
the difference between the liquid states scales roughly proportionally to the differ-
ence between the two input histories (thereby showing that the circuit dynamic is
not chaotic). This appears to be desirable from the practical point of view since it
implies that saliently different input histories can be distinguished more easily and
in a more noise robust fashion by the readout. We propose to use such evaluation
(
·
)
,
v
(
·
)
? This is evaluated in
Figure 18.1b
for the case where
u
(
·
)
,
v
(
·
)
‡
Note that it is
not
required that there exists a single
B
∈ B
which achieves this separation for any two
different input histories
u
(
·
)
,
v
(
·
)
.
Search WWH ::
Custom Search