Chain of Thought (CoT)

Complex problems often trip up an LLM that answers immediately. Chain-of-thought (CoT) prompting fixes this by asking the model to reason step by step before giving its final answer. Getting the model to "show its work" dramatically improves accuracy on maths, logic, and multi-step tasks — the same model, made far more reliable just by changing how it responds.

💡 In one line: Chain-of-thought prompting asks the model to reason step by step before answering, which sharply improves multi-step accuracy.

What is Chain-of-Thought?

CoT is a technique where the model generates intermediate reasoning steps before the final answer, instead of jumping straight to it. It mirrors how a person works through a problem rather than blurting out a guess.

Why It Helps

  • It breaks a hard problem into smaller, manageable steps.
  • Each step conditions the next, so the model spends more of its "thinking" on the actual reasoning.
  • It reduces careless errors on multi-step tasks.
  • It makes the reasoning visible and checkable.

Two Ways to Trigger CoT

  1. Zero-shot CoT — simply add a phrase like "Let's think step by step." No examples needed; remarkably effective.
  2. Few-shot CoT — provide examples that include the reasoning, so the model imitates that step-by-step style.

The Reasoning Flow

Chain-of-Thought (CoT) Reasoning Flow
Chain-of-Thought (CoT) Reasoning Flow
Chain-of-Thought (CoT) Reasoning Flow

When to Use CoT

  • Maths, logic, and multi-step reasoning.
  • Planning and analysis.
  • Any time a direct answer is unreliable.

It's not needed for simple factual or lookup tasks — there it just wastes tokens.

Example

Question: "A shop had 23 apples, sold 8, then received 12 more. How many now?"

  • Without CoT: the model may slip and answer wrongly.
  • With CoT: "23 − 8 = 15; 15 + 12 = 27. Answer: 27."

Laying out the arithmetic makes the correct answer far more likely.

Code Example


 
Limitations & Cautions
 
     
  • It uses more tokens (longer output).
    •  
  • Reasoning can still be flawed — one wrong step gives a wrong answer.
    •  
  • The shown reasoning may be post-hoc — not a perfect trace of the true computation.
    •  
  • In user-facing apps, you may hide the reasoning and show only the final answer.
    •  
 
Related: Self-Consistency
 
CoT pairs naturally with self-consistency — sampling several reasoning chains and taking the majority answer for extra reliability (the next subtopic).
 
Summary
 
     
  • Chain-of-thought makes the model reason step by step before answering.
    •  
  • It improves maths, logic, and multi-step accuracy by breaking problems into steps.
    •  
  • Trigger it with zero-shot ("Let's think step by step") or few-shot reasoning examples.
    •  
  • Use it for hard reasoning, not simple lookups (it costs extra tokens).
    •  
  • Reasoning can still be wrong — and it pairs well with self-consistency. EOF echo "created"