Database Reference
In-Depth Information
Gutierrez:
How do you think about solving the right problem?
Radinsky:
I do a lot of analysis before I start solving the problem because
I think the most important thing is to solve the right problem. So when I
approach a problem, I need to make sure that this is the problem that needs
solving. The next thing I do is to make the smallest possible prototype to
make sure that my solution actually addresses the problem. We learn in one
environment and we test in a completely different environment because we
have a temporal problem, so we keep a lot of historical data. The first thing
I do is to take a lot of historical data and create a staging environment for
us to do all of the relevant experiments without waiting a week. I do a lot of
experimentation because this is the way to work with data.
If there's an issue from a perception kind of problem, I talk to our customers.
I'll ask them, “What's going on?” We do a lot of logging in our system, so I see
the usage data and I speculate on different things based on why they did or
didn't click on something. Because we have so many clicks, eventually I'll build
classifiers to help me understand where to focus. I want to understand what
the differentiation is between places where they do click versus places they
don't click. Based on that, I try to explore where to get our hypothesis. So,
with most of the ideas, I just make hypotheses and check them.
Gutierrez:
How do you evaluate the work you and you group doing?
Radinsky:
Well, engineering has very specific guidelines, so if somebody's
doing scaling, we can just look at the success criteria for that problem. We
assign success criteria to every engineering task before we assign the task.
This helps us understand the problem and ensure we know what it means
for it to be a successful task. This is how I measure the success from an
engineering point of view. In data science problems like, “We currently have a
30 percent decision rule. We need to go up.” And if does, it does. Eventually,
the numbers don't lie. So we look at success on the data science side through
numbers as well.
Gutierrez:
What do you look for when hiring people?
Radinsky:
First of all, I always want good engineers. I ask questions like, “In
this problem, with this architecture, what are the tools you're going to use?
How are you going to solve it?” I also ask them to write code. Eventually,
and more than that, it's important for me to know that they know the tools
they're using in depth. So if it's someone from an engineering background, I'll
ask about Java or JVM, or ask something about how does memory work—
something very deep to show me that this person is willing to go deep into
things. This is important for me.
If it's in data science, I'll just show them one of the problems we have and ask
how they would go about solving it. For me, it's very important for them to
actually ask the right questions, to see that they are not stuck on some idea,
