Database Reference
In-Depth Information
What was interesting—and the goal of our work together—was that we were
able to tell if certain programs that they had instituted were actually working.
For example, Grameen gave their teams bicycles to travel further, which was
an expensive program to put in place. They wanted to know if and how well
that investment was working. Our volunteer team discovered by looking at
the GPS data from their services, that there wasn't a statistically significant
increase in the range for the groups that had the bikes. Based on that insight,
Grameen cut the program and chalked the learning up to an experiment that
didn't work. This allowed them to put the funding to other great uses.
Gutierrez:
What were some insights from this project for the data
scientists?
Porway:
I think an interesting thing that people may not think about when
just getting into data science is that you always need to question your assump-
tions. I know that sounds trite, but specifically here, Grameen and the data
scientists started with the premise that more searches means better perfor-
mance. They built a histogram of search volume over all volunteers and found
“For this month, this person did 200 searches. Everyone else did 10.” And at
that moment, as a data scientist or statistician, you might say, great, they're my
good performers. I'm going to go back to Grameen and say, “Here are your
good performers.”
But a good and even better data scientist is one who doesn't just compute
but thinks hard about all of the potential interactions of people and data. A
good statistician really thinks about all the confounding variables, all of the
things that are being said and not being said. So they didn't say, “These are the
best people.” They said, “Grameen, this is what we got back. Some people did
200 searches. Does this sound right? This is what we've observed.” And the
people at Grameen sort of scratched their heads. “Those look high.” And the
data scientist quickly went back—this was all happening at the same time
because they were in the same room—and looked at it simply hour by hour,
and it turned out that some of these people were doing 200 searches an hour.
And we all said, “Wait, that's impossible.” Grameen left committed to finding
the issue with their data, which could have been that people were gaming the
system or that there was an issue in their cellphone systems.
Another lesson from this project was that we shouldn't underestimate how
much little things like that can transform an organization. And so finding out
about the data was just a simple analysis that found a problem in data quality.
Grameen could help so many more people by fixing their data collection, but
without having a data scientist available, it just wouldn't have been possible.
Gutierrez:
In addition to the great lessons from that project, what were
other lessons you've been able to extend to other projects?
