Mystery Number One - Computational Linguistics and Talking Robots - page 25

A notable difference between the auditory and the visual modality in inter-
agent communication is that the interpretation(s) (read, hear mode) of a letter,
for example, may be far removed in time and space from the point of produc-
tion (write, speak mode). 19 Another difference is that written language can be
corrected by the author, while spontaneous speech cannot:
Speech is irreversible. That is its fatality. What has been said cannot be unsaid, except by
adding to it: to correct here is, oddly enough, to continue.

R. Barthes (1986, p. 76)

Of course, as soon as spoken language is recorded, an interpretation may be
arbitrarily distant in time and space from the production. Also, a recording may
be “doctored” - which from a certain point of view is a form of correction.
If the speak mode and the hear mode utilize different modalities and are
realized by the same agent, we have a modality conversion. In nature, modality
conversion is illustrated by reading aloud (conversion from the visual to the
auditory modality) and taking dictation (conversion from the auditory to the
visual modality).

2.3.2 T WO KINDS OF MODALITY CONVERSION

agent reading aloud external world agent taking dictation external world

auditory auditory

modality modality

modality free
internal coding

modality free

internal coding

visual
modality

visual
modality

In technology, reading aloud is modeled by combining optical character recog-
nition and speech synthesis, which is an important application for the blind.
Conversely, taking dictation is modeled by a machine called “electronic secre-
tary,” which combines speech recognition with optical character synthesis. 20

19 As shown in Chap. 11, the content of a text must be interpreted relative to the STAR-1 (Space,
Time, Agent, Recipient) values of the author when producing the written sign. Because the reader
is normally not present when a text is written - in contradistinction to the hearer in face-to-face
communication - it is in the interest of the writer to code the STAR-1 into the text itself. Otherwise,
the reader will have difficulty anchoring the written sign to its context of use.
20 Infra-red cameras are another technical means of modality conversion, representing temperature
(what is called temperature modality , Dodt and Zotterman 1952) as color (visual modality).