Agent-Based Models and Optimal Control in Biology: A Discrete Approach - Mathematical Concepts and Methods in Modern Biology - page 151

of an infection. We refer to variables of this type as state variables because they help
to describe the state of the system at any given time.
In this example, our only control variable is a binary decision whether or not to
administer the drug on a given day, thus there are only two possible values for this
control variable at each time step. For any given day, we may represent that the drug
was administered that day by assigning that variable the value 1, and represent that
the drug was not administered by assigning that variable the value 0. In this situation,
then, a treatment schedule is a vector of length 365, of which each entry is either 0 or
1. Thus there are a total of 2 365 possible treatment schedules. However, it is likely that
not all possible treatment schedules will reduce the number of infected cells below the
fixed level. The treatment schedules that do achieve this are solutions to the optimal
control problem. The solution space is the set of all such solutions. Note that this
space is entirely separate from the state space , which consists of all possible states a
patient may be in throughout any simulation of the model.
Recall that we are trying to find the best solution among many candidates. In order
to do this, we must have some way of ranking individual solutions. We do this by
introducing a cost function . We are attempting to steer the system to a particular
state, and each solution achieves that goal at a certain cost. We wish to determine
the best solution (i.e., the optimal control); thus we wish to find the solution that
minimizes the cost function. Earlier we noted that in the immune system example
the best solution was the solution that involved the least number of treatment days
throughout the treatment schedule; thus, in this example a reasonable cost function
might involve the sum of the entries in the treatment schedule (i.e., the number of
days on which the drug was administered). For the purposes of this chapter, the goal
of the optimization and the optimal control problems will always be to minimize the
associated cost function.
In our example, we wish to minimize T , the total number of days the drug is taken,
and maximize I , the number of healthy cells. Then a reasonable cost function for a
treatment schedule S which contains S t days on which the drug is administered and
results in an expected value of S i healthy cells might be c S i . Here we want

to maximize S i over all treatment schedules S ; this is the same as minimizing

( S ) = S t −

S i (in
this way we can always formulate our problem so that the goal is to minimize the cost
function). However, it is perhaps more realistic that one of our optimization goals has
a higher priority than the other. For example, we may suppose that maximizing the
number of healthy cells is more important than minimizing the number of treatment
days. In that case, we introduce weighting coefficients to alter the cost function. For
example, it may become c

−

5 S i . This has the effect of increasing the relative
importance of the healthy cell count when evaluating the associated cost of a particular
treatment schedule.
In order to use our model to obtain results, we simulate themodel a number of times
and make observations on the results. We might, for example, administer a treatment
every day. This particular solution is represented as a string of 365 consecutive 1's;
we implement this in the agent-based model and count how many healthy cells are
there after one year of simulated time. We then evaluate the cost function based on this

( S ) = S t −