A reasoning model’s thinking tokens are the internal computation it performs before writing its visible answer — a chain-of-thought you pay for but never see. A single complex query can generate 10,000 thinking tokens. On GPT-5.5 at $30 per million output tokens, those invisible tokens cost $0.30. The visible answer might cost $0.006. You just paid 50x more for the thinking than the answer.
I learned this the hard way. I’d been running a code review pipeline through GPT-5.5 for two weeks before checking the billing breakdown. Every review was generating roughly 8,000 reasoning tokens before producing a 300-token response. Those 8,000 tokens — roughly $0.24 per review at GPT-5.5 rates — were invisible in the response. I found them buried in output_tokens_details.reasoning_tokens. Switching to a lower reasoning effort cut my monthly bill by $200 with no measurable drop in review quality.
This guide explains what thinking tokens are, which models use them, what they cost, and how to control them without sacrificing output quality. If you’re new to the fundamentals, start with what AI tokens are and how they work.
What are thinking tokens?
A standard model generates output one token at a time, directly producing the text you see. A reasoning model inserts an invisible step: before writing the answer, it generates a chain of internal reasoning — the thinking tokens — that break the problem into smaller steps, consider alternatives, and self-verify before committing to a final response.
You type a prompt. The model spends several seconds (or minutes) “thinking.” During that time it’s generating tokens — sometimes thousands of them — that you never see. Then it produces the visible answer. You’re billed for all of it at the output rate.
Thinking tokens are distinct from regular input and output tokens in one critical way: they don’t persist across conversation turns. When you send the next message in a conversation, the thinking tokens from the previous turn are discarded. Only the visible text remains in context. This means you don’t pay to re-process old thinking tokens, but it also means the model doesn’t “remember” its reasoning from previous turns.
Chain-of-thought reasoning can mislead — as explored in the article on the illusion of AI reasoning, what looks like step-by-step logic is often sophisticated pattern matching. Thinking tokens don’t guarantee correct reasoning. They do guarantee higher bills.
How thinking tokens work internally
Here is what happens during a reasoning model request, step by step:
- Your prompt enters the system with all conversation history and system instructions.
- The model begins generating thinking tokens — internal reasoning that breaks the problem into substeps, tests assumptions, and evaluates alternatives.
- These thinking tokens consume context window space alongside your input. A 100K-token context window with 10K tokens of input and 15K tokens of thinking has 75K tokens remaining for additional reasoning and the final response.
- The model generates the visible response tokens after the thinking phase completes.
- The API returns the visible response. Thinking tokens appear in billing details (
output_tokens_details.reasoning_tokenson OpenAI, usage object on Anthropic) but not in the response content. - When you send the next message, the thinking tokens are stripped. Only the visible answer text stays in the conversation history.
The context window constraint matters. As covered in the context windows guide, reasoning models consume window space faster than standard models because of their invisible token generation. A conversation that runs fine on GPT-5.4 might hit the context limit on GPT-5.5 solely because of thinking tokens.
Which models use thinking tokens
| Model | Provider | Thinking control | Output rate | Max output (incl. thinking) |
|---|---|---|---|---|
| GPT-5.5 | OpenAI | reasoning.effort (none/minimal/low/medium/high/xhigh) | $30.00/M | 128,000 |
| GPT-5.4 | OpenAI | reasoning.effort | $15.00/M | 128,000 |
| GPT-5.4-mini | OpenAI | reasoning.effort | $4.50/M | 128,000 |
| Claude Opus 4.7 | Anthropic | thinking.type: "adaptive" + effort param | $25.00/M | 128,000 |
| Claude Sonnet 4.6 | Anthropic | thinking.type: "adaptive" + effort param; or budget_tokens (deprecated) | $15.00/M | 64,000 |
| Gemini 2.5 Pro | Thinking config via API | $10.00/M | 65,536 | |
| DeepSeek V4-Flash | DeepSeek | Automatic chain-of-thought; thinking tokens shown to user | $0.28/M | 384,000 |
OpenAI uses reasoning.effort as a knob from none to xhigh. Anthropic uses adaptive thinking with an effort parameter on Opus 4.7 and Sonnet 4.6 (the older budget_tokens approach is deprecated). Google exposes thinking through the Gemini API with its own configuration. DeepSeek V4-Flash is the only model that shows you the actual thinking tokens — everyone else hides them, returning at most a summary.
GPT-5.5 defaults to medium effort. Claude Opus 4.7 and Sonnet 4.6 use adaptive thinking when explicitly configured — without setting thinking: {"type": "adaptive"}, they operate in standard mode. Many developers enable extended thinking and never adjust the effort parameter, paying for thinking tokens on requests that don’t benefit.
What thinking tokens cost in practice
Here is a concrete comparison of the same prompt run through a reasoning model versus a standard model. The prompt: “Debug this SQL query that’s returning duplicate rows. Explain the root cause and provide the fix.”
import math
def thinking_cost(model, input_tokens, output_tokens, thinking_tokens, calls_per_month):
rates = {
"gpt-5.5": (5.00, 30.00),
"gpt-5.4": (2.50, 15.00),
"gpt-5.4-mini": (0.75, 4.50),
"claude-opus": (5.00, 25.00),
"claude-sonnet": (3.00, 15.00),
"gemini-flash": (0.30, 2.50),
}
input_rate, output_rate = rates[model]
# Output cost includes both visible output AND thinking tokens
total_output = output_tokens + thinking_tokens
cost_per_call = (input_tokens / 1_000_000 * input_rate) + (total_output / 1_000_000 * output_rate)
monthly = cost_per_call * calls_per_month
return {
"cost_per_call": round(cost_per_call, 5),
"thinking_cost_per_call": round(thinking_tokens / 1_000_000 * output_rate, 5),
"monthly": round(monthly, 2),
}
# SQL debugging: 800 input, 400 visible output, 5K thinking tokens, 1000 calls/month
result = thinking_cost("gpt-5.5", 800, 400, 5000, 1000)
print(f"GPT-5.5 with thinking: ${result['cost_per_call']}/call, ${result['monthly']}/month")
print(f"Thinking alone: ${result['thinking_cost_per_call']}/call")
result = thinking_cost("gpt-5.4", 800, 400, 5000, 1000)
print(f"GPT-5.4 with thinking: ${result['cost_per_call']}/call, ${result['monthly']}/month")
# Standard model (no thinking): GPT-5.4-mini, 800 in, 400 out
standard_cost = (800 / 1_000_000 * 0.75) + (400 / 1_000_000 * 4.50)
print(f"GPT-5.4-mini (no thinking): ${round(standard_cost, 5)}/call, ${round(standard_cost * 1000, 2)}/month")Output:
GPT-5.5 with thinking: $0.166/call, $166.0/month
Thinking alone: $0.15/call
GPT-5.4 with thinking: $0.083/call, $83.0/month
GPT-5.4-mini (no thinking): $0.002/call, $2.4/monthThe thinking tokens cost $0.15 of the $0.166 GPT-5.5 call — 90% of the total. The same query on GPT-5.4-mini without thinking costs $0.0024. That’s a 69x cost difference for the same SQL debug. If the debug output is identical or near-identical (which it often is for well-scoped technical problems), you paid 69x more for no measurable gain.
Here are five real-world scenarios with cost comparisons:
| Scenario | Model | Thinking tokens | Visible tokens | Calls/day | Monthly cost |
|---|---|---|---|---|---|
| SQL debugging | GPT-5.5 medium | 5,000 | 400 | 200 | ~$996 |
| SQL debugging | GPT-5.4-mini none | 0 | 400 | 200 | ~$55 |
| Code review | Claude Sonnet adaptive | 8,000 | 300 | 100 | ~$1,245 |
| Code review | Claude Sonnet standard | 0 | 300 | 100 | ~$144 |
| Research agent | GPT-5.5 high | 20,000 | 1,500 | 50 | ~$3,225 |
| Research agent | GPT-5.4 medium | 8,000 | 1,500 | 50 | ~$825 |
| Support chatbot | GPT-5.5 low | 2,000 | 500 | 5,000 | ~$3,825 |
| Support chatbot | GPT-5.4-mini none | 0 | 500 | 5,000 | ~$788 |
| Contract analysis | Claude Opus adaptive | 12,000 | 800 | 20 | ~$645 |
| Contract analysis | Claude Sonnet standard | 0 | 800 | 20 | ~$29 |
The support chatbot comparison is the most dramatic: $3,825/month on GPT-5.5 versus $788 on GPT-5.4-mini. For classification, extraction, and straightforward Q&A, the reasoning model adds cost without adding value. The token pricing guide covers full pricing across all providers if you need the complete comparison tables.
When thinking tokens help
Thinking tokens measurably improve output quality for specific categories of tasks. Here is where the extra computation earns its cost:
Complex multi-step reasoning. Math proofs, logic puzzles, legal reasoning chains, and scientific problem-solving where the model needs to hold intermediate conclusions and verify them before producing the final answer. GPT-5.5 at medium effort outperforms GPT-5.4 on MATH benchmark problems by roughly 15-20 percentage points.
Architecture decisions. The model needs to weigh tradeoffs, compare approaches, and justify a recommendation. Thinking tokens let it explore “option A vs. option B” internally before committing. This is where Claude Opus with extended thinking justifies its premium.
Contradiction detection. Comparing two documents and finding where they conflict. The model must hold both documents in mind, identify claims, and cross-reference. Thinking tokens give it the space to do this systematically.
Autonomous agent loops. When an AI agent needs to plan a multi-step workflow, evaluate tool results, and decide next actions, thinking tokens help it avoid dead ends. This is why GPT-5.5 is OpenAI’s recommended model for agentic coding.
Code debugging with confounding factors. A bug that involves multiple files, race conditions, or non-obvious interactions between systems. The thinking phase lets the model trace execution paths before proposing a fix.
The pattern across all these cases: the task requires holding multiple pieces of information in mind and reasoning across them simultaneously. When the model can get the answer by pattern-matching against a single piece of context, thinking tokens add nothing.
When thinking tokens are wasted
These tasks do not benefit from reasoning models. The thinking phase burns tokens without improving output quality:
Classification. “Is this email spam or not?” “Which department should handle this ticket?” The model does not need to think about the answer — it needs to match patterns.
Extraction. “Pull the invoice number and date from this PDF.” Single-pass pattern extraction. No multi-step reasoning required.
Summarization. “Summarize this article in three paragraphs.” The model identifies key sentences and compresses them. Thinking about how to summarize doesn’t produce a better summary.
Translation. Moving text from one language to another. A task where fluency matters more than reasoning depth.
Simple Q&A. “What is the capital of France?” “How do I reverse a list in Python?” These are single-fact retrievals. Using a reasoning model is like hiring a research scientist to look something up on Wikipedia.
Boilerplate generation. “Write a React component for a login form.” Standard components are well-represented in training data. The model does not need extended thinking to produce them.
The routing strategy from the pricing guide applies here: classification, extraction, summarization, translation, and simple Q&A go to the cheapest model available. Code generation, content writing, and multi-turn conversations go to mid-tier models. Architecture decisions, complex debugging, and multi-step reasoning go to expensive reasoning models — and only when the extra thinking actually changes the output.
The token waste article covers the optimization tactics that move the needle fastest. Model routing is number one.
How to control thinking token costs
1. Use the reasoning effort parameter
Every provider exposes a knob for controlling how much the model thinks.
OpenAI: Set reasoning.effort to low or minimal for tasks where medium overkill. Use none for classification and extraction tasks. Only use high or xhigh when your evals show a measurable quality gain that justifies the cost.
response = client.responses.create(
model="gpt-5.5",
reasoning={"effort": "low"}, # Start here for most tasks
input=...
)Anthropic: On Claude Opus 4.7 and Sonnet 4.6, use adaptive thinking with the effort parameter. Lower effort = fewer thinking tokens. On older models, set budget_tokens explicitly (1,024 for simple tasks, up to 32,000 for complex ones).
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=16000,
thinking={"type": "adaptive", "effort": "low"},
messages=...
)The key insight: don’t accept the default. Default reasoning effort is medium on GPT-5.5 and adaptive on Claude 4 models. Both produce thinking tokens on every request. Experiment with lower effort levels and compare output quality before scaling.
2. Route simple tasks to non-reasoning models
This is the highest-leverage cost reduction. Every request that goes to GPT-5.5 or Claude Opus should earn its premium. If a GPT-5.4-mini or Claude Haiku call would produce the same output, route there instead.
A practical routing table:
| Task type | Model | Thinking? | Output cost/M tokens |
|---|---|---|---|
| Classification, extraction, translation | GPT-5.4-mini | No | $4.50 |
| Simple Q&A, boilerplate | Claude Haiku 4.5 | No | $5.00 |
| Code generation, content writing | Claude Sonnet 4.6 | No (disable thinking) | $15.00 |
| Multi-step debugging, architecture | GPT-5.5 | Low effort | $30.00 |
| Research, contradiction detection | GPT-5.5 or Claude Opus | Medium/high effort | $25-30.00 |
3. Set max_output_tokens
This caps the total tokens the model can generate, including thinking tokens. On OpenAI:
response = client.responses.create(
model="gpt-5.5",
reasoning={"effort": "medium"},
max_output_tokens=8000, # Cap total generation
input=...
)OpenAI recommends reserving at least 25,000 tokens for reasoning and output when experimenting. Set a lower cap once you know how many tokens your prompts typically need. If the model hits the cap, you get an incomplete response with incomplete_details.reason: "max_output_tokens". The thinking tokens generated up to that point are still billed.
4. Monitor thinking token usage
OpenAI exposes output_tokens_details.reasoning_tokens in the response usage object. Anthropic includes thinking token count in the usage payload. Log this metric alongside your regular input/output token counts.
A Python monitoring snippet:
response = client.responses.create(...)
usage = response.usage
visible_output = usage.output_tokens - usage.output_tokens_details.reasoning_tokens
thinking = usage.output_tokens_details.reasoning_tokens
thinking_ratio = thinking / usage.output_tokens
if thinking_ratio > 0.8:
print(f"WARNING: {thinking_ratio:.0%} of output is thinking tokens")
print(f" Thinking: {thinking} tokens, Visible: {visible_output}")
print(f" Consider reducing reasoning effort")A thinking ratio above 80% means the model is spending far more on internal reasoning than on the final answer. This is a strong signal to either reduce the reasoning effort or question whether the task needs a reasoning model.
5. Don’t anthropomorphize the thinking process
The model is not “thinking harder.” It is generating more tokens that statistically correlate with more accurate outputs on certain task types. More thinking tokens do not guarantee better results on every task. They guarantee higher costs on every task.
Test with your actual data. If low effort produces the same output quality as medium on your specific use case, use low. If a non-reasoning model produces the same output quality, disable thinking entirely. Only pay for the tokens that change the output.
FAQ
Do thinking tokens count toward the context window?
Yes. Thinking tokens occupy context window space during generation. They are discarded before the next conversation turn, so they don’t accumulate from turn to turn — but during a single request, thinking tokens and output tokens compete for the same context window. OpenAI recommends reserving at least 25,000 tokens for reasoning when experimenting with GPT-5.5.
Can I see the actual thinking tokens?
DeepSeek V4-Flash is the only major model that exposes raw thinking tokens in the API response. OpenAI returns reasoning summaries with the summary parameter set to auto. Anthropic returns summarized thinking on Claude 4 models (a condensed version, not the raw chain of thought — the full thinking content requires a special arrangement with Anthropic). The thinking tokens are still generated and billed regardless of whether you see them.
How do thinking tokens differ from regular chain-of-thought prompting?
With chain-of-thought prompting, you add “think step by step” to your prompt and the model includes its reasoning in the visible output. You see the reasoning, it stays in the conversation history, and you’re billed for it as output tokens. With thinking tokens, the model generates reasoning internally — it’s invisible, doesn’t persist across turns, and is billed at the same output rate as visible tokens. The advantage of thinking tokens: they don’t clutter the conversation history. The disadvantage: you can’t inspect them for errors.
Do all providers charge the same rate for thinking tokens?
All major providers bill thinking tokens at the same output rate as visible tokens. On GPT-5.5, that’s $30/M. On Claude Opus 4.7, $25/M. On Gemini 2.5 Pro, $10/M. There are no discounts for thinking tokens, no separate pricing tiers. One thinking token costs exactly what one visible output token costs.
Can I use prompt caching with thinking tokens?
Prompt caching reduces input token costs but does not affect thinking token costs. Thinking tokens are always billed at the full output rate. Caching your system prompt and document set reduces the total cost per call but doesn’t change the thinking token math.
What happens if max_output_tokens is lower than the thinking tokens needed?
The model stops generating before completing thinking. The API returns status: "incomplete" with reason: "max_output_tokens". You’re billed for all generated tokens up to the cutoff. If the cutoff happens during the thinking phase, you may receive no visible text at all — you paid for invisible reasoning with no answer.
Do thinking tokens improve when I use a higher effort level?
They can, but not proportionally. GPT-5.5 at high effort might use 3x the thinking tokens of low effort but only improve accuracy by 5-10 percentage points on the hardest problems. On easy problems, the difference is often zero. The relationship between thinking tokens and output quality is non-linear and task-dependent. Test with your own evals.
All pricing as of May 2026. Check provider pricing pages before budgeting. Reasoning effort defaults, thinking token behavior, and pricing are subject to frequent changes.
Related topics
- What are AI tokens? A complete guide to tokenization, costs, and context windows. Understand the fundamentals first.
- LLM context windows explained: limits, costs, and developer workarounds. How thinking tokens consume context window space.
- AI token pricing guide: What LLMs actually cost per model. Full pricing comparison and routing strategies.
- The illusion of AI reasoning: Why your thinking model isn’t really thinking. Whether chain-of-thought reasoning is genuine or sophisticated pattern matching.
- What is an AI agent?. Why autonomous agents especially need thinking tokens — and how they burn through them.
- How to stop wasting tokens in Claude Code: 7 data-backed fixes. Optimization for the most common dev environment.