Regression techniques thus can be used for prediction, estimation, modeling rela-
tionships, or hypothesis testing. The following figure shows a standard representa-
tion and the terminology used:
When we want to model a dependent variable Y as a function of three different in-
dependent variables (X1, X2, and X3), we usually would not have enough data to
estimate the relationship or the function, and hence we start with an assumption of
linear dependency. Here, most of the effort is in studying variables that are determ-
inistically related to each other.
The following is a representation of linear probabilistic regression model:
The limitations are:
• Linear regression does not handle the missing values well. It assumes that
variables that affect the outcome nonlinearly and the relationships are not ac-
tually additive, the model does not fit well.
• It is recommended to take the log of monetary amounts or any variable with
a wide dynamic range. It cannot handle variables that affect the outcome in
a discontinuous way.
• Also, when we have discrete drivers with a large number of distinct values
the model becomes complex and computationally inefficient.