Java Reference
In-Depth Information
Chapter 3. Finding Sentences
Partitioning text into sentences is also called
Sentence Boundary Disambiguation
(
SBD
).
This process is useful for many downstream NLP tasks that require analysis within sen-
tences; for example, POS and phrase analysis typically work within a sentence.
In this chapter, we will explain in further detail why SBD is difficult. Then we will exam-
ine some core Java approaches that may work in some situations, and move on to the use of
models by various NLP APIs. We will also examine training and validating approaches for
sentence detection models. We can add additional rules to refine the process further, but
this will work only up to a certain point. After that, models must be trained to handle both
common and specialized situations. The later part of this chapter focuses on these models
and their use.
