Database Reference
In-Depth Information
more connections with neuroscience. That was a way of getting papers pub-
lished, because if you publish another paper on supervised convolutional nets,
people say, “You talked about this in 1991. Why don't you do something else?”
Whereas unsupervised learning was kind of new, so that's what actually got
people interested in deep learning.
Then what happened is that the practical applications of the deep learning are
all purely supervised. They're all basically just back prop on convolutional nets.
And the funny thing is that Geoff was one of the people who changed their
minds about this. I mean, he didn't change his mind in the sense that he still
thinks unsupervised learning is the way to go in the long run, and I believe this,
and Andrew Ng believes this, and Yoshua Bengio believes this. But in terms of
practical applications, he changed his mind in the sense that he started work-
ing on purely supervised convolutional nets like me. He applied this to speech
recognition, image recognition, and contributed to changing the opinion of the
community about the whole idea of deep learning.
We've been sort of oscillating between the case of Geoff thinking unsuper-
vised learning is the way to go in the long run, but sort of holding his nose and,
you know, making supervised learning work because it actually works. And for
me, it's been more of a change of opinion over the years. I'm more like Geoff
now. I'm still convinced that in the long run for things like natural language and
video, unsupervised learning will have to play a big role, but the stuff we use
today in practice is all back prop.
Gutierrez:
What does the field of data science look like in the future?
LeCun:
What I say very often in regards to the future of data science is that
one of most important things to notice is that the amount of data that's being
collected and stored is growing exponentially. It either grows at the speed at
which our communication network increases in bandwidth or at the speed at
which our hard drives increase in capacity. It's always one of the two, depend-
ing on whether it is streaming data or it is stored data. And so that's an
exponential with a pretty big rate. Currently, when you try to extract knowl-
edge out of that data, there are humans in the loop. The amount of human
brainpower on the planet is actually increasing exponentially as well, but with
a very, very, very small exponent. It's very slow growth rate compared to the
data growth rate.
What this means is that inevitably—in fact, this has already happened—there
is a point where there are just not enough brain cells on the planet to even
look or even glance at that data, let alone analyze it and extract knowledge
from it. So it's clear that most of the knowledge in the world in the future is
going to be extracted by machines and will reside in machines. It's probably
already the case actually, depending on what your definition of knowledge is.
For me, knowledge is some compilation of data that allows you to make deci-
sions, and what we find today is that computers are making a lot of decisions
automatically. That's not going to get any better in the future.
