Trust, Control and Autonomy: A Dialectic Relationship - Trust Theory: A Socio-Cognitive and Computational Model - page 202

It has to be clear that this problem is central in any collaboration - between individuals,
organizations, etc. - and that this theory is aimed at the general, not just for AI. Our claim
in fact is that: in designing how to adjust the level of autonomy and how to arrive at a
dynamic level of control, it is necessary to have an explicit and general theory of (trust-based)
delegation, which specifies different dimensions and levels of delegation, and relates the latter
to the notion and the levels of autonomy .
Thus, we propose our plan-based analysis (Pollack, 1990) of levels of delegation and levels
of help, and discuss a related notion of autonomy. In several cases of collaboration among
agents an open delegation is required, that is the delegation 'to bring it about that

...

'. The
agent is supposed to use its knowledge, intelligence, ability, to exert some degree of discretion.
Given that the knowledge of the delegating agent/user (client) concerning the domain and
the helping agents is limited (both incomplete and incorrect) the 'delegated task' (the request
or the elicited behavior) might not to be so useful for the delegator itself. Either the expected
behavior is useful but cannot be executed, or it is useless or self-defeating, or dangerous for
the delegator's other goals, or else there is a better way of satisfying the delegator's needs; and
perhaps the helping agent is able to provide greater help with its knowledge and ability, going
beyond the 'literally' delegated task. We will call extension of help or critical help this kind
of help. To be really helpful this kind of agent must take the initiative of opposing (not for
personal reasons/goals) the other's expectations or prescriptions, either proposing or directly
executing a different action/plan. To do this it must be able to recognize and reason with the
goals, plans and interests of the delegator, and to have/generate different solutions.
Open delegation and over/critical help distinguish a collaborator from a simple tool, and
presupposes intelligence and autonomy (discretion) in the agent. However, of course, there
is a trade-off between pros and cons both in open delegation and in extended(critical)-help :
the more intelligent and autonomous the delegee (able to solve problems, to choose between
alternatives, to think rationally and to plan) the less passively obedient it is. 16 So, possible
conflicts arise between a client and its contractor; conflicts which are due either to the intel-
ligence and the initiative of the contractor or to an inappropriate delegation by the client.
We are interested here only in the conflicts originating from the agent's willingness to
collaborate and to help the other in a better and more efficient way: a kind of collabora-
tive conflict . We do not consider the contractor's selfish reasons for modifying delegation
(because the nature of the conflict, negotiation, etc. would be different). 17
It is worth specifying that this work is aimed at providing a theoretical framework, i.e.
the conceptual instruments necessary for analyzing and understanding interaction with au-
tonomous entities. As has just been said, we assume that this framework is useful not only
for organization theory or management, but also for a principled engineering, i.e. for getting
systems designed not only on the basis of empirical data and practical experience, but also on
the basis of a more complete view and typology, and of some prediction and explanation. The
role of the dynamics of trust (Chapter 6) and control in this conflict and in the adjustment of
the level of Y 's autonomy is clear.
We also suggest some criteria about when and why to adjust the autonomy of an agent, and
preliminary hints about necessary protocols for adjusting the interaction with agents.

16 Obviously a very autonomous but stupid agent is even worse.
17 In this chapter there is at least one of these reasons that should be taken into account: when the contractor/trustee
adjusts the delegation for having more autonomy.