System analysis - Signals, Processes, and Systems - page 170

If we have defined information as an arranged meaningful pattern (see Chapter 2)
stochastic noise appears to be the only signal which according to our definition does not
possess any information. A pattern in the signal gives it a tendency to preserve itself. This
means that as the transmission of a pattern requires a certain period of time in the case of
two time segments A and B which directly follow each other something in segment B
must remind one of segment A. To put it in another way: there must be a certain similarity
or relation between time segments A and B.
Similarity or relation are complex concepts. In the case of signals they may refer to three
areas:
Signal patterns - may resemble each other in the time and/or
frequency domain, the relation/similarity may also refer to
statistical information.
An example: radioactive decay occurs in a random manner. Pronounced radioactive decay
- made audible or visible via a detector - makes itself felt as noise. The time intervals
between two processes of decay which follow each other or between two clicks statisti-
cally subject to an “exponential distribution”. Brief intervals between two clicks occur
very frequently, long intervals occur rarely. Accordingly two signals which have come
into being as a result of radioactive processes of decay may have a certain (statistical)
relationship with each other.
Such considerations unavoidably take us to information theory . We cannot leave out the
theoretical aspect in the case of test signals. While a noise-signal u in does not contain any
information about the features of the system, the response of the system, i.e. u out , should
contain and make transparent all the information about the system. The test signals dealt
with up to now have a certain tendency towards preservation, that is, a certain regularity
in the time and frequency domain. This is not the case with the noise-signal - apart from
statistical features.
All information which the noise response u out contains derives
originally from the system, all the information says something
about the system. Unfortunately this information about the system
is not directly recognizable either in the time or frequency domain
as the noise response still has a random component.
In order to obtain the features of the system in a pure form from noise response we must
have recourse to statistics. For example, it would be possible to add a few noise responses
and calculate the average. The FT of this mean noise response is a clear improvement.
The average value plays a decisive role in statistics, partly because the mean value of a
series of stochastic measurement readings must be equal to zero. If this mean value were
not zero certain quantities would occur more frequently than others. The measurement
readings or the signal would than not be purely random.