Updated 2026-07-12
learn
Token
The unit an LLM actually reads and writes ??? usually a word-piece rather than a whole word. "Unhelpful" might be three tokens. Models are priced and measured in tokens, and they generate text one token at a time.
Updated 2026-07-12
learn
The unit an LLM actually reads and writes ??? usually a word-piece rather than a whole word. "Unhelpful" might be three tokens. Models are priced and measured in tokens, and they generate text one token at a time.