Erin Shellman - Data Scientists at Work - page 66

Gutierrez: How has it been working with developers?
Shellman: I've been an R programmer for a decade, but you don't develop
general-purpose software in R, so I've had to learn a ton and think about
things differently. We released Recommendo 2.0, the second version of our
product recommendations API, earlier this year and I worked on a real-time
component of that called “the scorer.” In a nutshell, the scorer receives mes-
sages about customer actions like product page views or add-to-bag events.
It then re-orders recommendations in real-time based on the customer
behavior right now in the session. I initially wrote the scorer using the Python
library pandas because we'd been using it for our nightly batch recommenda-
tions. Well, I learned that the conveniences of the pandas data frame that we
were enjoying for batch jobs had subpar performance in real-time applica-
tions. Each time the scorer runs, a message needs to be parsed, scores need
to be computed and updated, and the process requires multiple reads and
writes to our Dynamo tables on AWS. In this situation the set-up costs of the
data frame objects were too high, and I ended up having to re-write the whole
thing without pandas.
In the past I didn't really have to worry much about the performance of my
programs, so I didn't recognize the performance limitations of the scorer
I'd written. It was really amazing to have talented developers around when
I needed advice.
Gutierrez: What's been the progression from undergrad to Nordstrom?
Shellman: I started undergrad as an economics and philosophy major because
I really liked thinking about complexity in human behavior and interactions.
I was drawn specifically to economics because I liked the process of describing
that complexity with math. As I got further along in my program, I realized
that I couldn't see myself at the age of 40 still working in economics, and knew
I needed a change. I was young and didn't realize that you don't have to do the
thing you majored in for the rest of your life.
Part of what drove my shift from social science was reading Richard Dawkins'
The Selfish Gene 1 and Robert Axelrod's The Evolution of Cooperation 2 , which is
a classic text about evolutionary game theory. I developed an admiration and
love of applying econ-like math models to biology. So rather than continue to
focus on economics, I decided that I wanted to study biological sciences while
retaining the math focus. To really explore this area, I got an internship at the
National Institutes of Health in the Division of Computational Biosciences.
I worked for Jim Malley, a mathematician who taught me machine learning and
1 Richard Dawkins, The Selfish Gene, 2nd ed. (Oxford University Press, 1990) .
2 Robert Axelrod, The Evolution of Cooperation (Basic Books, 2006).