Information Technology Reference
In-Depth Information
Fig. 2.4
The role of
hypothesis in communication
Information
Source
Destination
Message
Inferred
Message
Representation
Interpretation
Coded
Message
Received
Coded Message
Transmier
Receiver
Signal
2.3
Insight
So far I have suggested that the message is represented by some characteristic of
the transmitted symbols, and this is considered to be a single event. Because of the
additive properties of information, the entropy of a sequence of (say) N symbols
that
are independent
is the sum of the entropy for each of the symbols.
It could be the case that the sequence of symbols is significant (as in Morse Code)
and that each message is identified by a different sequence. If we at the destination
'know' the key of this code, the sequences can be interpreted and the information
measure relates to the number of encodable messages (see Fig.
2.4
). This will usually
be less than the sum of the entropy for the individual and independent symbols.
However, if the key is not 'known', the information is perceived to be that of
the uncertainty of the independent symbols rather than the potential messages. This
greater entropy we will call
Perceived Entropy
.
•
We can thus say that the perceived entropy is either higher than or equal to the
actual entropy of a system.
In the original sequence of digits above, the probability of any digit occurring is
about 0.1, but the insight that this sequence is the value of
p
means that the sequence
of numbers can be calculated from an equation such as:
π
4
=
1
3
+
1
5
−
1
7
+
1
9
−
1
11
+
1
13
−
1
15
+
1
−
.....
•
In this case the perceived entropy falls from that of approaching infinity to zero in
a single moment, and it is this insight that characterises the intelligence process.
Insight
can now be seen to involve at least two processes (see Fig.
2.5
). The first
process is the identification of the symbol (Wittgenstein L
1921
). Since the symbol
is to be abstracted from the signal we will call this process
abstraction
. The symbol
is not always the obvious sign such as a digit, but may be a feature of the sign such as
the notions of even, odd and prime. Abstraction may be considered formed through
the
perception
of significant features. In this case a perception is a concept that
involves bringing together the features into a single unit. That is the identification of
the elements observed that carry the important information.
