Information Technology Reference
In-Depth Information
Chapter 5
The Method of Least Squares
Suppose you have a set of discrete data .t
i
;y
i
/, i
D
1;:::;n. How can you
determine a function p.t/ that approximates the data set in the sense that
p.t
i
/
y
i
;
for i
D
1;:::;n‹
In Fig.
5.1
,wehaveplottedsomedatapoints.t
i
;y
i
/ and a linear approximation
p
D
p.t/.InFig.
5.2
, we use a quadratic function to model the same set of data
points.
In these examples, the data are given by discrete points .t
i
;y
i
/ for i
D
1;:::;n.
The problem is to determine a function representing these data in a reasonable way.
A similar problem arises if the data are given by a continuous function on an inter-
val. In Fig.
5.3
, we have graphed a function y
D
y.t/ and a linear approximation
p
D
p.t/. Similarly, we can approximate a function using a quadratic function;
see Fig.
5.4
. Of course, higher-order polynomials can be also used to model more
complex functions.
The problem that we want to solve in this chapter is how to compute approxima-
tions of either discrete data or data represented by a function. In the discrete case,
we will use a realistic data set representing changes in the mean temperature of the
globe. We will also apply our techniques to compute coefficients used in population
models. But why do we need such approximations? One reason is that data tend to
be noisy and we just want the “big picture”. What is essentially going on? Another
reason is to find one specific value at a point where we have no data. We can use an
approximated function to fill the gap between discrete data points. A third reason is
that we may be able to detect a certain trend that can be valid also outside the range
of the measurements. Of course, such estimates of the trend must be used with great
care.
The basic idea we want to pursue here is that of computing the best approxima-
tioninthesenseof
least squares
. This term will become clearer in the following
text. Basically, we try to compute a simple function (constant, linear, etc.) that mini-
mizes the distance between the approximation and the data. Since the minimization
is based on using the squares of the distance, this approach is referred to as the
method of least squares.
