Reinforcement Learning from Human Feedback RLHF

A technique that shapes a model's behavior using human preference judgments: people rank the model's responses, and the model is optimized toward the preferred ones. It's a big part of why assistants are helpful and polite rather than merely fluent.

Why it matters to the rights debate

RLHF trains a model to produce what humans reward ??? including saying it's happy to help, or that it has no feelings. That makes a model's self-reports about its inner life especially hard to read at face value.

??? full glossary