Databases Reference
In-Depth Information
literally
keeping track of the truth
. The way the article frames this is by
claiming that the new approach of Big Data is letting “N=ALL.”
Can N=ALL?
Here's the thing: it's pretty much never all. And we are very often
missing the very things we should care about most.
So, for example, as this
InfoWorld post
explains, Internet surveillance
will never really work, because the very clever and tech-savvy criminals
that we most want to catch are the very ones we will never be able to
catch, because they're always a step ahead.
An example from that very article—election night polls—is in itself a
great counter-example: even if we poll absolutely everyone who leaves
the polling stations, we still don't count people who decided not to vote
in the first place. And those might be the very people we'd need to talk
to to understand our country's voting problems.
Indeed, we'd argue that the assumption we make that N=ALL is one
of the biggest problems we face in the age of Big Data. It is, above all,
a way of excluding the voices of people who don't have the time, energy,
or access to cast their vote in all sorts of informal, possibly unan‐
nounced, elections.
Those people, busy working two jobs and spending time waiting for
buses, become invisible when we tally up the votes without them. To
you this might just mean that the recommendations you receive on
Netflix don't seem very good because most of the people who bother
to rate things on Netflix are young and might have different tastes than
you, which skews the recommendation engine toward them. But there
are plenty of much more insidious consequences stemming from this
basic idea.
Data is not objective
Another way in which the assumption that N=ALL can matter is that
it often gets translated into the idea that data is
objective
. It is wrong
to believe either that data is objective or that “data speaks,” and beware
of people who say otherwise.
We were recently reminded of it in a terrifying way by
this New York
Times article
on Big Data and recruiter hiring practices. At one point,
a data scientist is quoted as saying, “Let's put everything in and let the
data speak for itself.”
