Digital Signal Processing Reference
In-Depth Information
as suitable to be applicable to a wide range of practical problems. The
Russian mathematician Andrei Nikolaevich Kolmogorov established a solid
landmark in 1933, by proposing the axiomatic approach that forms the
basis of modern probability theory [169]. A few years later, in his clas-
sical paper [272], Shannon made use of probability in the definition of
entropy, in order to “play a central role in information theory as measures of
information, choice and uncertainty.” This fundamental link between uncer-
tainty and information raised many possibilities of using statistical tools in
the characterization of signals and systems within all fields of knowledge
concerned with information processing.
2.1 Signals and Systems
Information exchange has been a vital process since the dawn of mankind. If
we consider for a moment our routine, we will probably be able to point out
several sources of information that belong to our everyday life. Nevertheless,
“information in itself” cannot be transmitted. A message must find its proper
herald; this is the idea of signal.
We shall define a signal as a function that bears information, while a
system shall be understood as a device that produces one or more output
signals from one or more input signals. As mentioned in the introduction
of this chapter, the proper way to address signals and systems in the mod-
ern theory of filtering and signal processing is by means of their statistical
characterization, due to the intrinsic relationships between information and
randomness. Nevertheless, for the sake of systemizing such theory, we first
consider signals that do not have incertitude in their nature.
2.1.1 Signals
In simple terms, a signal can be defined as an information-bearing function.
The more we probe into the structure of a certain signal, the more informa-
tion we are able to extract. A cardiologist can find out a lot about your health
by simply glancing at an ECG. Conversely, someone without an adequate
training would hardly avoid a commonplace appreciation of the same data
set, which leads us to a conclusion: signals have but a small practical value
without the efficient means to interpret their content. From this it is easy
to understand why so much attention has been paid to the field of signal
analysis.
Mathematically, a function is a mapping that associates elements of two
sets—the domain and the codomain. The domain of a signal is usually,
although not necessarily, related to the idea of time flow. In signal processing,
there are countless examples of temporal signals: the electrical stimulus pro-
duced by a microphone, the voltage in a capacitor, the daily peak temperature
