Database Reference
In-Depth Information
So data scientists need this intuitive part that revolves around data. It is some-
thing that is part of their personality, kind of like skepticism, as well as having
expectations. I had a long talk with my friend Andreas Weigend, who was the
first guy who ever got me excited about data. His belief was even stronger.
He felt that, “You have to have an emotional reaction.” I said, “You just have to
have expectations. You need to know what you would expect to see, and then
you can see whether it deviates.” He felt that this is good, but he felt much
stronger about having to have the emotional response.
Gutierrez:
Is this intrinsic to a person or is it something that can be taught
and/or learned?
Perlich:
Something I have consistently seen in myself and other data scientists
is an ability to say, “You know, something does not look right,” even though it
may take a while to translate that feeling into something you can communicate
and make a formal case for it being wrong. It typically starts with “Hmmm, I
am not sure about this.” To develop this, I think takes good apprenticeship.
You need to be shown that process. You need to make peace with it. I have
had the blessing of having really good mentors, starting with Andreas, and then
Foster, who has a very similar pedigree and attitude toward these things. The
interesting thing is that the process is not terribly formal, but having seen the
process a few times, you start to get the hang of it.
That said, you cannot force it down everybody's throat. It takes experience
and apprenticeship, but that is a necessary condition, not a sufficient condition.
In addition to the experience and apprenticeship, you still need the intuition
about what is happening.
Gutierrez:
How do you think about whether you are solving the right prob-
lem or modeling the right thing? How do you even know you have the right
data?
Perlich:
This is one of the biggest generic problems to arise in data science.
It takes more than technical skill to be able to answer these questions. I may
know how to solve a problem, but the ability to provide feedback on whether
or not the question I am being posed with is meaningful in the first place—
that is a very difficult problem.
One of the reasons I love working here at Dstillery is that there is a lot of
appreciation for what data science does and how we can help. We are part
of almost all of the decisioning from a business perspective in the first place.
What I have seen in some of the consulting engagements of IBM was that if
a conversation does not involve somebody who actually knows what can be
done with data, you often end up solving the wrong problem or no problem
at all.
