AI reads in tokens. And its desk is finite.
Two ideas explain most AI costs, limits and “it forgot what I said” moments.
Tokens
The Unit Of Everything
- Chunks of text: a word, part of a word, or punctuation
- In English, roughly 750 words ≈ 1,000 tokens
- Usage limits and API pricing are counted in tokens
- Other languages often need more tokens per word
Context window
What It Can See At Once
- Instructions, chat history, documents and its reply all share it
- When it fills up, older material gets dropped or summarised
- Detail buried mid-document is easier to miss
- Bigger windows help, but cost more per message
Source: OpenAI API docs ↗
Short, focused context beats long and messy.