Information Technology Reference
In-Depth Information
linguistics and information theory, informed by an understanding of the
primitive computational operations, suggests constraints on determinis-
tic selection from written language relevant to information retrieval. At
this stage of conceptual development, constraints can be stated only as
provisional generalizations, not confirmed. To create index descriptions,
deterministic operations on written language may involve tearing words
or other syntactically differentiated units from the syntagma and releas-
ing them into paradigmatic multivalency. Multivalency may emerge most
clearly in syntagmas retrieved by Boolean operations in retrieval, but it
could remain present with other algorithmic procedures for description
and retrieval. Extending the syntagma cut in description or searching can
reduce the potential for multivalency. A preference for nondeterminis-
tic modes in searching can be supported theoretically as well as expe-
rientially. The human capacity for choice—earlier connected with the
etymology of
intelligence
(
inter-legere
, or to choose between) (Scholarly
and Ordinary Discourses, chapter 2)—exceeded the possibility of com-
putational modeling, particularly for combining and understanding the
syntagma. Information theory yielded an understanding of the relatively
frequent occurrence of word and multiword sequences, extracted from
the message.
The respective domains of linguistics and information theory have been
sustained, although some convergence emerged from the dynamics of
each discourse: in the linguistic view of speech, as marked by freedom
of combination, and in the elements of semantic selection by the source
and interpretation by the destination in information theory; Shannon's
own experiments on language did not separate semantic from syntactic
processes (1951/1993). Insights into full-text retrieval have been obtained
from the correspondence of the syntagma and paradigm, and the mes-
sage and messages for selection, to processes of representation in full text
retrieval, which could not have been obtained from any of the discourses
in isolation. The computational transformations possible on written lan-
guage continued to be understood from automata theory, with an inde-
pendent and consistent account given by information retrieval experience
and research. Practice and theory have been brought into a dialectical
relation to each other.
