Introduction to Optimization - Numerical Methods in Engineering with MATLAB - page 391

way to tackle the problemistouse a succession of one-dimensional minimizations
to close in on the optimal point. The basicstrategy is

Choose apoint x 0 in the design space.

loopwith i ,...

Choose avector v i .
Minimize F ( x ) along the linethrough x i − 1 in the direction of v i .Let theminimum
point be x i .

= 1 , 2 , 3

if | x i − x i − 1 | <ε exit loop

end loop

The minimizationalong a linecan be accomplishedwith any one-dimensional
optimizationalgorithm (such as the golden section search). The only question left
openishow to choose the vectors v i .

Conjugate Directions

Consider the quadraticfunction

2
i

1

F ( x ) = c − b i x i + A i j x i x j

i j

1
2 x T Ax

b T x

= c − + (10.5)

Differentiationwith respect to x i yields

∂ F

x i =−

b i + A i j x j

∂

j

which can be writteninvectornotationas

∇ F =− b + Ax (10.6)

where F is the gradient of F .

Now consider the change in the gradient as wemovefrompoint x 0 in the direction
of avector u . The motion takes place along the line

∇

x = x 0 + s u

where s is the distance moved. Substitutioninto Eq. (10.6) yields the expression for
the gradient along u :

∇ F | x 0 + s u =− b + A ( x 0 + s u ) = ∇ F | x 0 + s Au