Take AISafety.info’s 3 minute survey to help inform our strategy and priorities

Take the survey

What is a subagent?

A subagent is a part of a larger system or agent which itself has agent-like properties. For example, we can model a corporation as an agent, since it makes decisions and pursues goals, but it is also made up of subagents: the people who work for it, who have their own goals.

There are numerous theories of human psychology, such as internal family systems, which model the human mind as also being made up of subagents. This is used to explain thoughts like “one part of me wants this, but the other parts are getting in the way”.

An agentic AI may also contain subagents, and insofar as subagents are involved, they present an additional alignment challenge: to make sure that they are also aligned with our values.

A related concept, which should not be confused with a subagent, is a mesa-optimizer.



AISafety.info

We’re a global team of specialists and volunteers from various backgrounds who want to ensure that the effects of future AI are beneficial rather than catastrophic.