Tokens

A token is the unit of text the model actually operates on: a run of characters somewhere between a letter and a word. Common English words are usually one token each, rarer words are split into several, and whitespace and punctuation usually ride along with a neighbouring word. As a working rule for English prose, one token is about four characters, or three quarters of a word; a statute, with its numbered parts and defined terms, runs denser.

Tokens govern three separate things:

1. You are billed per token, at one rate for tokens going in and a higher rate for tokens coming out. That is what the usage line at the bottom of talk.ts is reporting.
2. max_tokens is measured in output tokens. Set it too low and answers get truncated; set it high and nothing bad happens directly, because you are billed for what is produced, not for the ceiling.
3. There is a limit on the total, and that limit is what constrains how you design.

The context window is the maximum number of tokens that can be in a single request and its response combined. Exceed it and the API returns an error rather than silently dropping the oldest part. Current Anthropic models have very large context windows, large enough that nothing in this book comes near one; but “large” is not “free”, and Chapter 4 is where you stop sending the whole corpus with every question because the bill, not the window, says to.