Database Reference
In-Depth Information
of people that you're working with, like I did, all the traditional startup van-
ity metrics are ignored in favor of the question of “Are dollars coming in?”
That measure of success is all that matters, especially when you're VC-backed,
because the dollars-going-out number is typically very large. So for us at
P(K), the primary focus was: How many customers, doing real things with the
system, are paying us money? Of course, when we were very early on in the
process, we took the wrong view of measuring success by focusing on much
more technically questions, like: What's our uptime look like? How long does
it take to process a job? What's our predictive accuracy? and similar questions.
However, we very quickly realized that no one gives a damn. What really
matters is who's actually using and paying for it.
For example, there's all this talk about predictive analytics. Kaggle became this
big thing because everyone seems to think that predictive accuracy matters.
In reality, almost no one actually cares about predictive accuracy because in
almost all the cases, their starting point is nothing. If you have something that
gets them 80 percent of their way there, it's an infinite improvement and they
will be so happy. The number of industries where the difference between
85 versus 90 percent accuracy is the rate-limiting factor is very small.
Sometime in the future, after everyone has adopted these sorts of technolo-
gies, the predictive accuracy will start to matter, but at this point it doesn't
matter as much as people think it does. Sure there are some areas like quan-
titative hedge funds that are fighting tooth and nail over that last epsilon, but
most people are not in that position. So it really comes back to the question
of “What value are we providing?”
Gutierrez:
How do you view and measure success now that you've transi-
tioned back to research?
Jonas:
As I've transition back to research, it's been very important for me to
keep the startup experience view and measurement of success at the top of
my mind. Our first employee who we hired out of Berkeley, Jonathan Glidden,
wants to go back to graduate school. I'm really excited for him because I don't
think he's going to make any of these cognitive errors in graduate school,
having now gone through this process. He really understands, in some sense,
how to ship. But it's hard because a lot of academia tends to value novelty in
a way that I think is actually very counterproductive.
One of the things I struggle with is that comparative advantages are actually
complicated, because they intersect with your utility curves. There are
problem domains where I know that the models that I have would actually
be transformative. But I feel that that's not the most important domain for
me to be working on right now, or someone else will do that, or that I can
come back later. And so it's hard when you've been so trained by graduate
school to think that what really matters are papers. So it's novel to recognize
that, yes, some papers aren't actually worth your time to write. So a lot of it
