chain

Chain-of-thought is the discovery that a language model solves hard multi-step problems far more reliably when it writes out its reasoning before committing to an answer, rather than producing a final answer directly. Wei and colleagues' 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" showed this concretely: on GSM8K, a benchmark of grade-school math word problems, prompting a large model to answer directly scored in the 50-to-60-percent range, while the identical model prompted to reason step-by-step first — either through worked examples or simply the instruction "let's think step by step" — jumped to roughly 90 percent on the same benchmark, with no change to the model's weights at all. The gain came purely from giving the model somewhere to put intermediate reasoning before being asked for a final answer.

Question asked
e.g. a multi-step math word problem
Direct answer
one forward pass, no intermediate steps — often wrong on hard problems
Chain-of-thought
reasoning steps generated first, each one building on the last
Final answer
grounded in the reasoning trace that came before it

Why writing the reasoning out actually helps, rather than just looking more thorough, comes down to what a single forward pass can and can't do. A model answering directly has to reach the correct answer within one fixed amount of computation; writing out "step 1, step 2, step 3" turns each step into new context the model reads back in before generating the next one, effectively letting the model spend more total computation on a harder problem by chaining several smaller, more tractable sub-problems together instead of demanding one large leap in a single pass. A word problem asking how long nine shirts take to dry, if three take two hours, trips up a model answering directly because it pattern-matches "3 scales to 9, so 2 scales to 6" — wrong, because shirts dry in parallel, not sequentially — while a chain-of-thought response that explicitly reasons "all the shirts fit on the line at once" catches exactly the insight the direct answer skipped.

Several variants extend the same basic idea in different directions, and each addresses a different failure mode of plain prompted reasoning. Self-consistency, introduced alongside the original chain-of-thought research, samples several independent reasoning chains for the same question and takes the majority answer across them, which smooths over the cases where any single chain goes wrong partway through; Tree-of-Thought generalizes further by letting the model explore and compare multiple reasoning branches rather than committing to one linear chain; ReAct interleaves reasoning steps with actual tool calls, letting the model check a calculation or look something up mid-reasoning rather than reasoning from memory alone.

Prompted Reasoning vs. Trained Reasoning the same basic idea, at two very different points in a model's lifecycle Prompted chain-of-thought Wei et al., 2022 A capability already inside the model, simply elicited by the right instruction or examples GSM8K: ~55% → ~90% direct vs. chain-of-thought Works on any sufficiently large model — no training required Reasoning is visible, in the chat itself Trained reasoning models OpenAI's o1, DeepSeek-R1 Trained with reinforcement learning specifically to reason at length before answering Spends variable compute — harder problem, longer thinking Learns to backtrack and try alternate approaches on its own Reasoning trace often hidden from the user Same core idea — think before answering — baked into training instead of requested in the prompt.

Reasoning models turned chain-of-thought from a prompting trick into a second, genuinely new axis for scaling capability. Where most of deep learning's progress came from scaling training — more data, more parameters, more pretraining compute — a model trained to generate a long internal reasoning trace before answering can be made more accurate on a specific hard problem simply by letting it think longer at inference time, a cost-and-capability trade-off distinct from training a bigger model in the first place; OpenAI's o1 and DeepSeek's R1 are both built around this test-time compute idea, allocating more generated "thinking" tokens to harder problems and fewer to easy ones.

None of this comes free, and the honesty of the visible reasoning is a genuinely open question, not a settled guarantee. Every reasoning token generated is a token that costs time and money, which is why reasoning models are reserved for problems that actually need the extra computation rather than used by default on everything; more fundamentally, research on faithfulness has found that a model's written reasoning doesn't always reflect the actual computation that produced its answer — a chain of thought can look entirely plausible and still not be a true account of why the model landed on that particular answer, which means a convincing-looking reasoning trace is evidence in favor of a correct answer, not proof of one.

Chain-of-thought's real significance is the reframing it forced: a model's accuracy on a hard problem turned out to be a dial, not a fixed property of its size or training, and reasoning models are what happens when that dial gets built into the model itself — trading tokens, latency, and dollars for accuracy as a deliberate, controllable choice rather than something fixed at training time.

Go deeper with CFSGPT

Get AI-powered deep-dives, save terms, and run advanced simulations — free account.

Create Free Account