Database Reference
In-Depth Information
mathematical consistency?” My wife happens to work in the same field, but
on the experimental side. She was working at accelerators while I was off in
the academic clouds.
After I left academics, I worked for an enjoyable couple of years as an assistant
editor for the
Physical Review
in Brookhaven. It was very interesting to see how
the review and referee process works, and how the sausage is made in the
academic publishing world. It set me to thinking a lot more about computation
and data. And I was in the vicinity of New York and the financial industry.
Going to work in the quantitative finance industry was definitely my first jump
into a very data-intensive world. Quantitative finance broadly divides into two
fields. In the first field, people work with very complicated financial instru-
ments, such as over-the-counter derivatives, which are traded very simply,
usually over the phone. There are mathematical frameworks around these
complicated instruments, which have theoretical, analytical, and computational
aspects. The second field is high-frequency algorithmic trading, in which you
have very simple financial instruments, such as equities, which are traded in
very complicated ways. This field is much more computational and algorithm-
driven. There's no analytical structure around this type of trading; it's essen-
tially driven by experimental data. I originally wanted to go into quantitative
finance to work in the first field dominated by analysis of complicated deriva-
tive products, but I ended up in the second field dominated by algorithmic
trading of simple financial instruments.
Gutierrez:
What made you change your plan from doing purely analytical
work to data-driven work?
Lenaghan:
I quickly learned that the empirical style of work in algorithmic
trading suited me very well. I liked the experimental design—figuring out what
works, what doesn't work, and how not to trick yourself with data. There are
lots of challenges working with data in the algorithmic world. One of them is
that thousands and thousands of other people are looking at the exact same
data sets, and basically they're all just squeezing everything they possibly can
out of it. Another challenge is that people under pressure to find patterns
are prone to fall into the common human fallacies of overfitting models with
insufficient data and overreading correlation as causation.
Gutierrez:
How do the data challenges you faced in the algorithmic trading
world compare to the data challenges you face at PlaceIQ?
Lenaghan:
The initial data challenge when I came to PlaceIQ was that geospa-
tial data was a data type that I had never worked with. The second challenge
was that the data volume was scaled up by a couple of orders of magnitude.
The volume of data in the algorithmic trading I was doing was quite large—say,
a terabyte a year. But the PlaceIQ environment generates hundreds and hun-
dreds of terabytes a year. Making these adjustments were exciting challenges
