Think before you answer

The quick answer is often wrong

What is the last digit of 7¹⁰⁰?

Ask a standard LLM and it'll probably answer "7" in a fraction of a second. Makes sense — 7 starts with 7, 7² = 49, and without thinking too hard, you might assume it stays 7. That answer is wrong — it's 1.

But ask the same question to a reasoning model and it hesitates. It "thinks" for 10, 20, sometimes 60 seconds. And it arrives at the right answer. OpenAI opened that path with o1 in late 2024, DeepSeek-R1 made it public in early 2025; today every major model has a mode of this kind.

The difference isn't in the architecture — it's the same Transformer. It's in how the model was trained, and in what it's allowed to do before responding.

Thinking tokens

Every LLM generates tokens, one at a time, left to right. What sets reasoning models apart is that they first generate a long sequence of working tokens — an internal monologue — before producing the final answer. Depending on the provider, you see all of it, only a summary, or nothing at all.

These hidden tokens are called thinking tokens.

The model can write anything there: intermediate calculations, hypotheses it then disproves, abandoned exploration branches, double-checks. It's a scratchpad it erases before showing you the clean result.

It's not magic. It's just extra space to work through a hard problem.

Try it yourself

Set the thinking budget to "None" and click "Start reasoning." Watch the instant answer. Then switch to "Full" and run it again.

The greyed-out blocks are the internal chain of thought — the model hypothesizes, verifies, sometimes backtracks. These thinking tokens cost latency and money, but unlock problems that direct mode cannot solve.

The difference between the two isn't in the model's capacity — it's in the inference-time compute it's allowed to use.

How it works technically

This isn't a different architecture. It's the same Transformer, the same attention mechanism, the same autoregressive generation.

What changes is the training and decoding. And contrary to what you'd assume, it isn't shown thousands of well-written reasoning traces to imitate.

Something smarter happens: it's left to search. The model produces its own attempts, we mechanically verify which ones land on the right result — a calculation can be checked, a program passes its tests — and we reinforce the paths that worked. Nobody wrote those chains of reasoning: the model found them, and we kept the ones that got there. That's the RLVR described in chapter 08. DeepSeek even showed, with R1-Zero, that a model can develop these strategies from nothing, without a single annotated reasoning example.

At inference, it's given a thinking token budget — a limit on how many hidden tokens it can generate. The larger the budget, the more it can explore. Beyond a certain budget, quality improvements on hard tasks start to plateau.

An important detail: thinking tokens are generated before the answer, in the same token stream. The model doesn't "think" in parallel — it thinks in series, and that costs tokens just like everything else.

Extended reasoning vs. chain-of-thought

You may have seen the chain-of-thought (CoT) technique, where you explicitly ask the model to "think step by step." That's different, but related.

	Chain-of-Thought (prompted)	Extended reasoning (native)
Who triggers it	The user, in the prompt	The model itself
Visibility	Visible in the response	Hidden (thinking tokens)
Control	User can guide the steps	Model chooses its plan
Examples	GPT-4 with "let's think step by step"	o1, o3, Claude with extended thinking

Prompted CoT also improves performance — but native reasoning goes further, because the model isn't constrained to write readable steps. It can explore messy paths, run calculations it later discards, contradict itself and self-correct, all within the hidden space.

When it's worth it

Extended reasoning meaningfully improves performance on:

• Math and logic — proofs, combinatorics, exact arithmetic
• Complex code — multi-file debugging, non-trivial algorithms
• Structured reasoning — puzzles, chained deductions
• Planning — tasks that require laying out a strategy before acting

For simple factual questions ("what's the capital of France?"), creative text, or translation, extended reasoning adds nothing — and costs more.

On hallucinations (chapter 13), the effect needs stating carefully, because intuition misleads here. Where reasoning genuinely helps is on verifiable errors: a wrong calculation, a shaky deduction, a forgotten case. The model unrolls, rereads itself, catches itself.

On pure factuality — a name, a date, a quote — the gain isn't guaranteed. OpenAI published its own measurements showing that o3 hallucinated more than its predecessor on certain knowledge tests. Thinking longer doesn't conjure up information the model doesn't have: it just gives it more room to build a coherent story around what it believes it knows.

The cost is the real constraint. Thinking tokens are billed like regular tokens. A model that generates 1,000 thinking tokens before a 30-token answer bills you for 1,030. At millions of requests, that adds up.

Test-time compute scaling

What reasoning models revealed is that you can buy intelligence at inference time: the more thinking tokens you allocate, the better the answers get on hard tasks.

This is called test-time compute scaling — as opposed to the usual scaling that increases model parameters during training.

The curve looks similar to classical scaling laws: doubling the thinking budget improves performance, but with diminishing returns. At some point, thinking longer no longer compensates.

And this is a significant discovery: a LLM's intelligence isn't a fixed constant set by its weights. It also depends on the compute given to it at the moment of responding.

A model that thinks long and hard about a difficult problem can outperform a larger model that answers quickly. Speed isn't always a virtue.

But "thinking for a long time", concretely, means generating thousands of tokens one by one before even starting the answer. And we saw in chapter 09 that each token has to look at every token before it. If the thousandth token really had to reread everything from the start, none of these reasoning modes would be billable. What makes the thousandth token as fast as the second? The next chapter opens the hood on inference.

Updated August 2, 2026