Linguistic Computing with UNIX Tools - Natural Language Processing and Text Mining - page 250

is to combine the selection schemes with elementary statistical operations. We have
shown that the latter can easily be implemented with awk .
A third class of application of sed and awk is lexical-etymological analysis. Using
sed and awk , dictionaries of related languages can be compared and roots of words
determined through rule-based and statistical analysis.
Various selection schemes can easily be formulated and implemented using set
and vector operations on files. We have shown the implementation of set union, set
complement, vector addition, and other such operations.
Finally, all the above shows that sed and awk are ideally suited for the develop-
ment of prototype programs in certain areas of language analysis. One saves time in
formatting the text source into a suitable database for certain types of programming
languages such as prolog . One saves time in compiling and otherwise handling C ,
which is required if one does analysis with lex and yacc . In particular, if the devel-
oped program runs only a few times this is very e cient.
Disclaimer
The authors do not accept responsibility for any line of code or any programming
method presented in this work. There is absolutely no guarantee that these methods
are reliable or even function in any sense. Responsibility for the use of the code and
methods presented in this work lies solely in the domain of the applier/user.
References
1. H. Abramson, S. Bhalla, K.T. Christianson, J.M. Goodwin, J.R. Goodwin, J.
Sarraille (1995): Towards CD-ROM based Japanese ↔ English dictionaries:
Justification and some implementation issues. In: Proc. 3rd Natural Language
Processing Pacific-Rim Symp. (Dec. 4-6, 1995), Seoul, Korea
2. H. Abramson, S. Bhalla, K.T. Christianson, J.M. Goodwin, J.R. Goodwin, J.
Sarraille, L.M. Schmitt (1996): Multimedia, multilingual hyperdictionaries: A
Japanese ↔ English example. Paper presented at the Joint Int. Conf. Associa-
tion for Literary and Linguistic Computing and Association for Computers and
the Humanities (June 25-29, 1996), Bergen, Norway, available from the authors
3. H. Abramson, S. Bhalla, K.T. Christianson, J.M. Goodwin, J.R. Goodwin, J.
Sarraille, L.M. Schmitt (1996): The Logic of Kanji lookup in a Japanese ↔
English hyperdictionary. Paper presented at the Joint Int. Conf. Association
for Literary and Linguistic Computing and Association for Computers and the
Humanities (June 25-29, 1996), Bergen, Norway, available from the authors
4. A.V. Aho, B.W. Kernighan, P.J. Weinberger (1978): awk — A Pattern Scan-
ning and Processing Language (2nd ed.). In: B.W. Kernighanm, M.D. McIl-
roy (eds.), UNIX programmer's manual (7th ed.) , Bell Labs, Murray Hill,
http://cm.bell-labs.com/7thEdMan/vol2/awk
5. A.V. Aho, B.W. Kernighan, P.J. Weinberger (1988): The AWK programming
language . Addison-Wesley, Reading, MA
6. B.T.S. Atkins (1992): Acta Linguistica Hungarica 41 :5-71
7. J. Burstein, D. Marcu (2003): Computers and the Humanities 37 :455-467