Extracting Meaning from Data - Doing Data Science - page 168

Organizational factors can also hinder success. A competition that is
not run well conflates participants with mechanical turks : in other
words, they assume the competitors to be somewhat mindless and give
bad questions and poor prizes. This precedent is just bad for everyone
in that it demoralizes data scientists and doesn't help businesses answer
more essential questions that get the most from their data. Another
common problem is when the competitions don't chunk the work into
bite-sized pieces. Either the question is too big to tackle or too small
to be interesting.
Learning from these mistakes, we expect a good competition to have
a feasible, interesting question, with an evaluation metric that is trans‐
parent and objective. The problem is given, the dataset is given, and
the metric of success is given. Moreover, the prizes are established up
front.
Let's get a bit of historical context for crowdsourcing, since it is not a
new idea. Here are a few examples:
• In 1714, the British Royal Navy couldn't measure longitude , and
put out a prize worth $6 million in today's dollars to get help. John
Harrison, an unknown cabinetmaker, figured out how to make a
clock to solve the problem .
• In 2002, the TV network Fox issued a prize for the next pop solo
artist, which resulted in the television show American Idol , where
contestants compete in an elimination-round style singing com‐
petition.
• There's also the X-prize company , which offers “incentivized prize
competitions…to bring about radical breakthroughs for the ben‐
efits of humanity, thereby inspiring the formation of new indus‐
tries and the revitalization of markets.” A total of $10 million was
offered for the Ansari X-prize, a space competition, and $100
million was invested by contestants trying to solve it. Note this
shows that it's not always such an efficient process overall—but
on the other hand, it could very well be efficient for the people
offering the prize if it gets solved.