Information Technology Reference
In-Depth Information
different radius values in the estimation of the multidimensional probabilities in the
state space correspond to different probability bins. The values of radius for which
the probabilities are not accurately estimated (typically large
r
values) may eventu-
ally lead to an erroneous estimate of TE.
15.2.2 Improved Computation of Transfer Entropy
15.2.2.1 Selection of
k
The value of
k
(order of the driven process) used in the calculation of TE
)
(see Equation (15.3)) represents the dependence of the state
x
n
+
1
of the system on
its past
k
states. A classical linear approach to autoregressive (AR) model order
selection, namely the Akaike information criterion (AIC), has been applied to the
selection of the order of Markov processes. Evidently, AIC suffers from substan-
tial overestimation of the order of the Markov process order in nonlinear systems
and, therefore, is not a consistent estimator [12]. Arguably, a method to estimate
this parameter is the delayed mutual information [13]. The delay
d
at which the mu-
tual information of
X
reaches its first minimum can be taken as the estimate of the
interval within which two states of
X
are dynamically correlated with each other.
In essence, this value of
d
minimizes the Kullback-Leibler divergence between the
d
th and higher order corresponding probabilities of the driven process
X
(see Equa-
tion (15.1)), i.e., there is minimum information gain about the future state of
X
by
using its values that are more than
d
steps in the past. Thus, in units of the sampling
period,
d
would be equal to the order
k
of the Markov process.
If the value of
k
is severely underestimated, the information gained about
x
n
+
1
will erroneously increase due to the presence of
y
n
and would result to an incorrect
estimation of TE. A straightforward extension of this method for estimation of
k
from real-world data may not be possible, especially when the selected value of
k
is large (i.e., the embedding dimension of state space would be too large for finite
duration data in the time domain). This may thus lead to an erroneous calculation of
TE. From a practical point of view, a statistic that may be used is the correlation time
constant
t
e
, which is defined as the time required for the autocorrelation function
(AF) to decrease to 1
(
Y
→
X
e
of its maximum value (maximum value of AF is 1) (see
Fig. 15.1d) [13]. AF is an easy metric to compute over time, has been found to be
robust in many simulations, but detects only linear dependencies in the data. As we
show below and elsewhere [18, 19], the derived results from the detection of the
direction and level of interactions justify such a compromise in the estimation of
k
.
/
15.2.2.2 Selection of
l
The value of
l
(order of the driving system) was chosen to be equal to 1. The jus-
tification for the selection of this value of
l
is the assumption that the current state
