Training / pre-training

The compute-heavy process of adjusting a model's billions of parameters by showing it data until it captures patterns. Pre-training builds the base model; later stages (see fine-tuning and RLHF) shape its behavior.

Related: Fine-tuning · Reinforcement Learning from Human Feedback · Large Language Model

← full glossary