Database Reference
In-Depth Information
B[j] += alpha*t*x[i][j];
err2 += t*t;
}
err2 = Math.
sqrt
(err2);
if
(err2 - lastError < 1e-6)
break
;
lastError = err2;
}
return this
;
}
The iteration continues until either a maximum number of iterations are
reached or the change in error becomes so small it is not worth continuing.
A more general application of this approach is used to fit the parameters of
the artificial neural network models introduced in the next section.
Tip
The gradient descent approach works for all types of regression, not
just logistic regression. Although there are other options for
multivariate regression, this allows for streaming implementations of
the more complicated forms of regression. When working with
streaming data it is assumed that data in the future will be equivalent to
data in the past, which eliminates the need to iterate over the data
multiple times.
Neural Network Models
Neural network models, also known as the artificial neural network (ANN),
are a collection of nonlinear models used to predict some outcome given
some set of input variables. The first examples of neural networks date to
the early 1940s where they were used to solve classification problems. It
is generally accepted that neural networks represent a type of nonlinear
regression and can be applicable to the same problem domains.
Inspired by the activity of physical neurons in the brain, the artificial
neuron, called a unit in neural network jargon, computes a function of the
weighted sum of its inputs. This function is called the activation function
and is usually chosen from the family of sigmoid functions, which range
