What is an agent?
Informally, an "agent" is something whose actions can be understood as directed at achieving a goal.
One way of identifying agency is to ask how a system would behave if its environment changed. For example, imagine a system which gives advice, but imagine that (for whatever reason) people start to consistently do the opposite of what it suggests. If the system is non-agentic, it will continue to give the same advice, but if it is an agent with the goal of causing people to be successful, it will try to find other ways of achieving that goal. For example, it might start giving the opposite advice.
Some researchers focus on the system’s ability to act in the world on its own. For example, Richard Ngo argues that there are six factors which contribute to a system being agent-like:
- Self-awareness: it understands that it’s a part of the world, and that its behavior impacts the world;
- Planning: it considers a wide range of possible sequences of behaviors (let’s call them “plans”), including long plans;
- Consequentialism: it decides which of those plans is best by considering the value of the outcomes that they produce;
- Scale: its choice is sensitive to the effects of plans over large distances and long time horizons;
- Coherence: it is internally unified towards implementing the single plan it judges to be best;
- Flexibility: it is able to adapt its plans flexibly as circumstances change, rather than just continuing the same patterns of behavior.
These features
A feature of a region of input space that corresponds to a useful pattern. For example, in an image detector, a set of neurons that detects cars might be a feature.
In addition to disagreements over which properties are part of agency, there are also disagreements over whether there is an objective distinction between agents and non-agents: is everything in the world either one or the other, or is agency (merely) a "stance" that an observer can take towards a system as a way of understanding and predicting its behavior? As an example of agency as a stance, we might think of another human being as an agent because we don’t know all of the psychological factors which go into their decisions; that is, we might interpret their behavior through understanding the goals they are pursuing.
However, a superintelligence
An AI with cognitive abilities far greater than those of humans in a wide range of important domains.
The agent foundations A research agenda which tries to understand the nature of agents and their properties.
Further reading:
-
Deepmind’s paper on Discovering Agents
-
Sayash Kapoor and Arvind Narayanan on agents that matter
-
Owen Cotton-Barratt and Raymond D on Decomposing Agency
-
CSET’s report on Preparing for AI Agents