Closed-Loop Control Learning - Neural Networks: Methodology and Applications - page 289

5
Closed-Loop Control Learning
M. Samuelides
In the previous chapter, we showed how to use training in order to model
controlled dynamical systems, with emphasis on neural modeling. This chapter
extends that presentation to the problem of designing a closed-loop control
law by training. Nonlinear control has been a growing field during the past
twenty years. However, there is no methodology based on first principles, in
contrast to linear control. A number of practical methods have been proposed,
starting from various points of view. Some results are essentially theoretical,
addressing the problems of controllability, existence of stabilizing control law,
and validity of linearization techniques. Such results are beyond the scope of
this topic.
However, we will recall some elements of control theory in the following
section, emphasizing the connections between linear and nonlinear control
laws. Actually, “neural” techniques extend “classical” techniques of nonlinear
control to systems, which has been previously modeled using neuronal identi-
fication and training. Those techniques are described in the section “Design of
Neural Control by Inverse Model”, where several techniques are successively
studied: straightforward inversion, simple but too often ine cient, reference
model control, which is more frequently used, and recurrent models whose
implementation may be more di cult.
Further sections are devoted to optimal decision problems in the classical
framework of dynamic programming (section “Dynamic Programming and
Optimal Control”) and to its counterpart in learning theory (section “Re-
inforcement Learning and Neuro-Dynamic Programming”). Those techniques
were in existence long before neural networks became popular; they addressed
the problems of control in discrete spaces. Neural networks provided good ap-
proximations of those methods. Meanwhile, reinforcement learning can be
applied now to continuous state spaces avoiding “combinatorial explosion”
which was a drastic limit to the field of application of classical reinforcement
learning. That set of more modern techniques was termed recently “Neuro-
Dynamic Programming”.