Take AISafety.info’s 3 minute survey to help inform our strategy and priorities

Take the survey
Beyond the basics

Language models
Mesa-optimizers and subagents
Decision theory
Mathematics of agents
Strategy and outcomes
Brain emulation
Human intelligence enhancement
Computer science
Values
AI consciousness

What are "mesa-optimizers"?

Machine learning

is used to create computer programs to solve problems. Some programs created this way are very simple, capable of solving the specific problem they were trained to solve. Other programs created this way are more advanced, problem-solvers in their own right. When a machine learning process results in a piece of software that is a problem-solver itself, we call the created problem-solver a "mesa-optimizer".

"Mesa" is Greek for inner/within/inside. "Optimizer

" is the term for something that solves problems. So "mesa-optimizer" means something like "inner problem-solver".

In this context, the "outer problem-solver" is the machine learning process that created the mesa-optimizer.

For example, a human being can be thought of as a mesa-optimizer of evolution. Evolution is an optimizer, which fine-tunes creatures for reproductive fitness. Most creatures have a set program which helps them survive and reproduce, but human beings developed intelligence, itself an optimizing power. While intelligence initially emerged as a solution of the optimization problem of evolution, humans now use intelligence to optimize their own goals, sometimes at the expense of reproductive success.

Keep Reading

Continue with the next entry in "Beyond the basics"
What is a subagent?
Next
Or jump to a related question


AISafety.info

We’re a global team of specialists and volunteers from various backgrounds who want to ensure that the effects of future AI are beneficial rather than catastrophic.