Databases Reference
In-Depth Information
Adding other predictors.
What we just looked at was simple linear re‐
gression—one outcome or dependent variable and one predictor. But
we can extend this model by building in other predictors, which is
called
multiple linear regression
:
y
=
β
0
+
β
1
x
1
+
β
2
x
2
+
β
3
x
3
+ϵ.
All the math that we did before holds because we had expressed it in
matrix notation, so it was already generalized to give the appropriate
estimators for the
β
. In the example we gave of predicting time spent
on the website, the other predictors could be the user's age and gender,
for example. We'll explore feature selection more in
Chapter 7
, which
means figuring out which additional predictors you'd want to put in
your model. The R code will just be:
model <- lm(y ~ x_1 + x_2 + x_3)
Or to add in interactions between variables:
model <- lm(y ~ x_1 + x_2 + x_3 + x2_*x_3)
One key here is to make scatterplots of
y
against each of the predictors
as well as between the predictors, and histograms of
y x
for various
values of each of the predictors to help build intuition. As with simple
linear regression, you can use the same methods to evaluate your
model as described earlier: looking at
R
2
, p-values, and using training
and testing sets.
Transformations.
Going back to one
x
predicting one
y
, why did we
assume a linear relationship? Instead, maybe, a better model would be
a polynomial relationship like this:
y
=
β
0
+
β
1
x
+
β
2
x
2
+
β
3
x
3
Wait, but isn't this
linear
regression? Last time we checked, polyno‐
mials weren't linear. To think of it as
linear
, you transform or create
new variables—for example,
z
=
x
2
—and build a regression model
based on
z
. Other common transformations are to take the log or to
pick a threshold and turn it into a binary predictor instead.
If you look at the plot of time spent versus number friends, the shape
looks a little bit curvy. You could potentially explore this further
by building up a model and checking to see whether this yields an
improvement.
