Mystery Number One - Computational Linguistics and Talking Robots - page 23

2. The modality-dependent external surfaces are much better suited for (i)
content transfer and (ii) agent-external long-term storage than the associ-
ated agent-internal modality-free meanings.
The advantage of modality-free meanings referred to in (1) may be illustrated
by a sign-theoretic comparison of symbols and icons (cf. FoCL'99, Sect. 6.4).
Icons in the visual modality, for example, are limited to a visual representation
of meaning, which makes it often impossible to represent concepts from an-
other modality, for example, the meaning of sweet . Symbols, in contrast, have
no such limitation because their meaning representations are modality-free. 17
The advantage of modality-dependent external surfaces for transfer and stor-
age referred to in (2) is that an unanalyzed external surface is simple and ro-
bust compared to the often complicated meanings, attached by convention to
the surfaces inside the cognitive agent. This holds especially for the written
representation of language, which has long been the medium of choice for the
agent-external storage of content. 18
Naturally, the highly specialized and powerful technique characterized by
MoC-1 also has an apparent disadvantage: because the surface for a given
meaning can take any shape within the limits of its modality, communities of
natural agents can and do evolve their own languages. This results in the diffi-
culty of communicating in foreign language environments, which this chapter
began with.
2.3 Modality Conversion in the Speak and Hear Modes
MoC-1 is not only a conceptual insight into the working of natural language
communication between agents viewed from the outside, but constitutes a
well-defined technical challenge, namely the construction of machines which
can recognize and produce external language surfaces. The basic task of these
machines may be viewed as an automatic conversion between a modality-
dependent realization of an external surface and its modality-free counterpart
17 A modality-free representation of meaning is procedural in the sense that it is based on the recognition
and action procedures of the cognitive agent. Whether such a meaning representation in an artificial
agent is adequate or not is decided by the agent's performance. For example, an artificial agent's
concept of shoe may be considered adequate if the agent picks out the same object(s) from a collection
of different things as a human would (NLC'06, Chap. 4).
18 Recently, storage by means of written language has been complemented by the technologies of tape
recording, video, CDs, DVDs, etc., which allow to preserve also spoken and signed language for
indefinite periods of time.
On the relation between modalities and media see FoCL'99, pp. 23 f., and NLC'06, pp. 23 f.