Take AISafety.info’s 3 minute survey to help inform our strategy and priorities

Take the survey
Basic concepts

Capabilities
Current systems
Algorithms
Prompting
Alignment concepts
Intelligence and optimization
AI goals
Risks and outcomes

What is GPT-4?

GPT-4 is the latest (as of May 2024) and largest LLM released by OpenAI. The number of parameters

it contains is unknown, but a leaked report claims it to be 1.76 trillion, or ten times the size of GPT-3.

GPT-4 developed a variety of capabilities, notably the ability to program and to give detailed responses to logical puzzles. Furthermore, it has shown some signs of having a theory of mind.

GPT-4 was trained using reinforcement learning from human feedback

(RLHF). This feedback was only given during training: the model does not change while in active use. Unlike the previous versions of GPT, GPT-4 was also trained with image data, allowing it to interact with user-uploaded images. (It can also prompt DALL-E and display the resulting images.)

Different versions of GPT have “context windows” of different lengths. GPT-4 initially had multiple fine-tuned versions with context windows of up to 32,768 tokens, much longer than GPT-3.5’s 4,096 and GPT-3’s 2,048 tokens. Newer versions called GPT-4 Turbo and GPT-4o have a 128,000-token window.

Keep Reading

Continue with the next entry in "Basic concepts"
What does it mean when an AI is "hallucinating"?
Next
Or jump to a related question


AISafety.info

We’re a global team of specialists and volunteers from various backgrounds who want to ensure that the effects of future AI are beneficial rather than catastrophic.