Databases Reference
In-Depth Information
CHAPTER 4
Spam Filters, Naive Bayes,
and Wrangling
The contributor for this chapter is
Jake Hofman
. Jake is at
Microsoft
Research
after recently leaving Yahoo! Research. He got a PhD in
physics at Columbia and regularly teaches a
fantastic course
on data-
driven modeling at Columbia, as well as a newer course in computa‐
tional social science.
As with our other presenters, we first took a look at Jake's data science
profile. It turns out he is an expert on a category that he added to the
data science profile called “data wrangling.” He confessed that he
doesn't know if he spends so much time on it because he's good at it
or because he's bad at it. (He's good at it.)
Thought Experiment: Learning by Example
Let's start by looking at a bunch of text shown in
Figure 4-1
, whose
rows seem to contain the subject and first line of an email in an inbox.
You may notice that several of the rows of text look like spam.
How did you figure this out? Can you write code to automate the spam
filter that your brain represents?
