The Tandem Truck Backer-Upper Problem - Design of Experiments for Reinforcement Learning - page 126

Fig. 7.1 The state of the
tandem trailer truck is defined
by the rear trailer position
( x , y ), the cab angle ʸ 0 , the
first trailer angle ʸ 2 , and the
second trailer angle ʸ 4 . The
direction of the cab wheels is
defined by u .

u

Cab

ʸ 0

Trailer 1

ʸ 2

Trailer 2

ʸ 4

( x , y )

ʽʔt cos ʸ 3 cos ʸ 4 +

ʸ 4

x = x +

2

where ʽ 2.

The general problem and goal for the tandem trailer problem is basically the same
as that for the single trailer problem: to back up a trailer truck to a specific location
and orientation by controlling the orientation of the front wheels. As with the single
trailer problem, the truck can only back up, and forward actions are not allowed in this
implementation. The increased difficulty of adding another trailer is quite significant
however, and the tandem trailer problem is a very challenging control problem,
especially considering that the methods we employ are straightforward and relatively
simple with no heuristics. We therefore believe that a sequential training paradigm is
most appropriate and reasonable for learning this problem. By a sequential training
paradigm, we are referring to a procedure in which the problem is learned in stages,
where in each sequential stage, the learned control scheme becomes more and more
efficient and/or precise. An example of this could be where in each training stage,
the tolerances to the goal in terms of the desired location and orientation of the truck,
are reduced. Note that in this work, however, we focus on only the first phases of
such a sequential training scheme, such that the truck is only required to back up
to a specific location with a relatively loose tolerance. We consider a full sequential
training procedure to be a heuristic, which is beyond the scope of this work, and we
therefore reserve full development of a sequential scheme for future work.

=− 1, ʔt = 0 . 1, L = 5, and l =