Detecting Part of Speech - Natural Language Processing with Java - page 175

Phrase Textese Phrase Textese

Away from keyboard AFK You're on your own YOYO

Thanks THNX or THX As soon as possible ASAP

Today 2day What do you mean by that WDYMBT

Before B4 Be back in a minute BBIAM

See you C U Can't CNT

Haha hh Later l8R

Laughing out loud LOL On the other hand OTOH

Rolling on the floor laughing ROFL or ROTFL I don't know IDK

Great GR8 Cleaning the house CTH

At the moment ATM In my humble opinion IMHO

Note

Although there are several list of textese, a large list can be found at ht-
tp://www.ukrainecalling.com/textspeak.aspx .

Tokenization is an important step in the POS tagging process. If the tokens are not split
properly, we can get erroneous results. There are several other potential problems includ-
ing:

• If we use lowercase, then words such as "sam" can be confused with the person or
for the System for Award Management ( www.sam.gov )
• We have to take into account contractions such as "can't" and recognize that dif-
ferent characters may be used for the apostrophe
• Although phrases such as "vice versa" can be treated as a unit, it has been used
for a band in England, the title of a novel, and as the title of a magazine
• We can't ignore hyphenated words such as "first-cut" and "prime-cut" that have
meanings different from their individual use