Database Reference
In-Depth Information
Precision
is defined as the percentage of documents in the results that are
relevant. If by entering keyword
bPhone
, the search engine returns 100
documents, and 70 of them are relevant, the precision of the search engine result
is 0.7%.
Recall
is the percentage of returned documents among all relevant documents
in the corpus. If by entering keyword
bPhone
, the search engine returns 100
documents, only 70 of which are relevant while failing to return 10 additional,
relevant documents, the recall is
.
Therefore, the naïve Bayes classifier from
Table 9.7
receives a recall of
and a precision of
.
Precision and recall are important concepts, whether the task is about information
retrieval of a search engine or text analysis over a finite corpus. A good classifier
ideally should achieve both precision and recall close to 1.0. In information
retrieval, a perfect precision score of 1.0 means that every result retrieved by a
search was relevant (but says nothing about whether all relevant documents were
retrieved), whereas a perfect recall score of 1.0 means that all relevant documents
were retrieved by the search (but says nothing about how many irrelevant
documents were also retrieved). Both precision and recall are therefore based on
an understanding and measure of relevance. In reality, it is difficult for a classifier
to achieve both high precision and high recall. For the example in
Table 9.7
, the
naïve Bayes classifier has a high recall but a low precision. Therefore, the Data
Science team needs to check the cleanliness of the data, optimize the classifier, and
find if there are ways to improve the precision while retaining the high recall.
Classifiers determine sentiments solely based on the datasets on which they are
trained. The domain of the datasets and the characteristics of the features
determine what the knowledge classifiers can learn. For example,
lightweight
is a positive feature for reviews on laptops but not necessarily for reviews on
wheelbarrows or textbooks. In addition, the training and the testing sets should
share similar traits for classifiers to perform well. For example, classifiers trained
on movie reviews generally should not be tested on tweets or blog comments.
