Introducing Positive Discrimination in Predictive Models - Discrimination and Privacy in the Information Society - page 246

not discriminate with respect to a given sensitive attribute, e.g., for every sex, the
probability of being in the positive class should be roughly the same. For a more
detailed description of the problem domain and some algorithmic solutions, see
Chapters 3 and 12 of this topic. This chapter will discuss different techniques of
learning and adapting probabilistic classifiers to make them discrimination-free.
Initially, we concentrate on Naive Bayes classifiers, see, e.g., (Bishop, 2006).
These are simple probabilistic models with strong independence assumptions. The
main benefit of these assumptions is that they make the problem of learning a
Naive Bayes classifier easy. Intuitively, a Naive Bayes classifier can be used to
compute the probability that a given combination of attribute values (features or
characteristics) obtains a positive class value. If this value is larger than a given
threshold (typically 50%), the classifier outputs “yes”, otherwise it outputs “no”.
Consider, e.g., the following example of a spam filter.
Example
Suppose that we have a collection of emails, each of which is marked either as a
spam mail or a regular mail. In order to learn a predictive model for spam, we
have to transform every message into a vector of values. This is typically done by
selecting all words that appear in the emails, order them, and transform every
email in a sequence of 0-1 where a 1 in the ith position indicates that the ith word
appeared in the mail. Otherwise, the ith position is 0. E.g., suppose that the or-
dered list of words appearing in the collection of emails is:
(of, a, the, man, $, win, price, task).
Then the vector (1,1,1,0,0,1,0,0) would indicate an email in which the words “of”,
“a”, “the”, and “win” appear, but not “man”, “$”, or “price”. Based on the da-
ta vectors obtained, a model will be learned that can be used to predict for a new,
unlabeled email if it is spam or not. For the Naive Bayes classifier, the model es-
sentially corresponds to assigning a positive or negative score to every word, and
setting a threshold. The scores for all words present in the email to be classified
are added up, and only if the total score exceeds the threshold, the mail will be
classified as spam. The scores of the words and the threshold are the parameters
of the model. The Naive Bayes classification algorithm learns optimal values for
these parameters based upon the data. For example, suppose that the Naive Bayes
algorithm learns the following scores: (-0.5,-0.5,-0.5,0.5,2,1.5,2,-3) and threshold
2, then an email with content “win a price” corresponds to the vector
(0,1,0,0,0,1,1,0) and gets a score of -0.5+1.5+2 = 3, which exceeds the threshold.
Therefore, the mail is classified as spam.
A more exact definition of Bayesian models will be given in Section 2 of this
chapter. The decision of a Naive Bayes classifier is based on a given data set,
which is used to fit the parameters of the Naive Bayes model, and the strong class-
independence assumption. Although this assumption is often violated in practice,
even then good results can be obtained using a Naive Bayes approach (Langley et
al., 1992).