What is tool AI?

A tool AI is a type of artificial general intelligence that is limited in its range of action and can only function as an assistant to human use. One kind of system people often have in mind when talking about tool AI is one which generates information and then displays it in a user-friendly way, allowing the user to decide how to use it or whether to discard it entirely. It is contrasted with an agent, which takes actions in order to maximize a utility function.

Tool AIs were suggested as a safer alternative to agents. However, they are not necessarily safer, for a number of reasons. For example, the choices they offer may ultimately prove beyond human capacity to evaluate carefully, thereby reducing human oversight to a rubber stamp.

Furthermore, tool AIs could develop into agents for a number of reasons, including:

1. intentionally, since agents are more economically competitive. For example, requiring a human to make choices will critically slow down the system.

2. inherently, since navigating a complex domain will put pressure on a system to become a general optimizer. If the world is complex, the system will be able to carry out its function more effectively through becoming more agent-like. Thus it is likely to produce a mesa-optimizer as part of its development.

Eric Drexler’s comprehensive AI services (CAIS) is a theory about how many disparate tool AI systems could interact, jointly achieving many of the goals of AGI, but with none being agents in their own right.