Overview
Chain-of-thought (CoT) prompting elicits step-by-step reasoning from language models rather than jumping directly to answers. By asking the model to “think step by step” or providing examples that demonstrate intermediate reasoning, CoT dramatically improves accuracy on tasks requiring logic, math, multi-step planning, and complex analysis. The technique works because it forces the model to decompose problems and make intermediate conclusions explicit, reducing the chance of logical errors that compound silently. Variants include zero-shot CoT (just adding “think step by step”), few-shot CoT (providing reasoning examples), and structured CoT (using XML or markdown headers to organize thinking).
Key Concepts
- Zero-shot CoT — Appending “Let’s think step by step” to a prompt. Surprisingly effective despite requiring no examples.
- Few-shot CoT — Providing worked examples with explicit reasoning chains. More reliable for domain-specific tasks.
- Self-consistency — Generating multiple CoT paths and selecting the most common answer. Improves accuracy at the cost of multiple API calls.
- Tree of Thought — Exploring multiple reasoning branches simultaneously and evaluating which paths are most promising.
- Scratchpad pattern — Giving the model a designated space for intermediate calculations separate from the final answer.
- Structured thinking — Using XML tags, numbered steps, or section headers to organize reasoning into predictable phases.
Trade-offs
- Reasoning tokens consume context window and increase cost
- Not all tasks benefit — simple lookups and classifications don’t need step-by-step reasoning
- Longer reasoning chains can introduce errors that propagate
- Models may generate plausible-sounding but incorrect reasoning (faithfulness problem)
Related
- Prompt Engineering encompasses CoT as a key technique
- Agent Patterns use CoT in the reasoning phase of ReAct loops
- Context Windows must budget space for reasoning tokens