Explainable and Non-explainable Discrimination in Classification - Discrimination and Privacy in the Information Society - page 153

8.3 Conditional Non-discrimination in Decision Making

Even though the historical data contains discrimination, new classifiers trained on
this data should not discriminate in the future decision making. The first solution that
comes to mind to address this discrimination-aware classification problem is that we
should remove the sensitive attribute from the training data before learning a new
classifier. Unfortunately, this approach does not remove discrimination from future
decision making if some of the attributes in the training data are correlated with the
sensitive attribute. For instance, postal code may be highly correlated with race. A
classifier will be able indirectly (internally) predict the race from the postal code and
then it will still use race in the acceptance decisions. That is indirect discrimination,
known as redlining .
To get rid of such discriminatory relations among attributes, one would also need
to remove the attributes that are correlated with the sensitive attribute. It is not a
good solution if these attributes carry the objective information about the outcome,
as the predictions will become less accurate. For instance, a postal code in addition
to the racial information may carry information about real estate prices in the neigh-
borhood, which is objectively informative for loan decisions. The aim is to use the
objective information, but not the sensitive information of such attributes.
The explanatory attribute is the attribute in the training data that is correlated
with the sensitive attribute, and at the same time gives some objective information
about the outcome. In general there is no objective truth which attribute is more
reasonable to use as the explanation for discrimination. For instance, in case gender
is the sensitive attribute, some attributes, such as relationships ('wife' or 'husband')
are not a good explanation, as semantically they are closely related to gender. On
the other hand, difference in working hours may be an appropriate reason to have
different monthly salaries. What is discriminatory and what is legal to use as an
explanation of the outcome depends on the law and goals of the anti-discrimination
policies. Thus, the law or domain experts define, which attributes are sensitive and
which are allowed to be treated as explanatory.

8.3.1 An Example on University Admission

Consider a model of the admission decisions to a fictitious university 2 , that will help
to analyze the difference between the explainable and illegal discrimination. Note
that the model presents a simplified version of reality and is intended to cover the
key mechanisms of decision making, and does not cover a full application process.
Gender is the sensitive attribute; male (m) and female (f) are the sensitive groups,
against which discrimination may occur. There are two programs: medicine (med)
and computer science (cs) with potentially different acceptance standards. Program
is considered to be the explanatory attribute. In this example, we assume that the
differences in acceptance statistics between male and female that can be attributed to

2

This model does not express our belief how the admission procedures is modeled. We use
it for the purpose of illustration only.