Basic unit of text that AI language models process, typically representing words, word parts, or punctuation. Like the individual puzzle pieces that make up text.
The word 'unhappiness' might be split into three tokens: 'un', 'happi', and 'ness' by a language model's tokenizer.
“Token” is a foundational AI/ML concept (a unit of text used by language models), not a cloud provider service. All major clouds expose token-based LLM APIs (via their AI platforms), but the term itself is provider-agnostic.