Self-Consistency

Self-consistency is a powerful upgrade to chain-of-thought. Instead of trusting a single reasoning path, you sample several — each reasoning through the problem a little differently — and then take the majority answer. The intuition is simple: a correct answer tends to show up across many independent reasoning paths, while mistakes are usually one-off flukes that get outvoted.

💡 In one line: Self-consistency samples multiple reasoning chains for the same question and picks the most common final answer.

What is Self-Consistency?

It's a technique built on top of chain-of-thought:

  1. Prompt the model to reason step by step.
  2. Generate N separate answers, using some randomness (temperature > 0) so the reasoning paths differ.
  3. Extract the final answer from each.
  4. Take the majority vote.

Rather than one "greedy" chain, you get a crowd of reasoning attempts and trust their consensus.

Why It Works

  • Different valid reasoning paths tend to converge on the same correct answer.
  • Errors are idiosyncratic — a slip in one chain rarely repeats identically in others.
  • Voting cancels out the occasional flawed chain.

How It Works

The process is a straightforward pipeline.

Whiteboard
Whiteboard diagram

Benefits

  • Higher accuracy on maths, logic, and multi-step reasoning than a single chain-of-thought.
  • Simple to add on top of an existing CoT prompt.
  • More robust to the occasional bad reasoning path.

Costs & Trade-offs

  • N× the compute, cost, and latency — you run the model several times.
  • Only helps when there's a discrete answer to vote on.

It's a classic accuracy-for-cost trade.

When to Use It

  • High-stakes reasoning where accuracy matters more than cost.
  • Problems with a clear, checkable final answer (a number, a label, a choice).

Skip it for simple tasks or open-ended text, where there's nothing clean to vote on.

Code Example


 
Each call reasons independently; the most common answer wins.
 
Limitations
 
     
  • Expensive — several full generations per question.
    •  
  • Needs a votable answer; doesn't directly help free-form writing.
    •  
  • If the model is consistently wrong (a shared bias), voting won't rescue it.
    •  
 
Summary
 
     
  • Self-consistency samples multiple reasoning chains and takes the majority answer.
    •  
  • It works because correct answers converge while errors are idiosyncratic.
    •  
  • It builds on chain-of-thought and boosts accuracy on reasoning tasks.
    •  
  • The cost is N× compute, and it needs a discrete answer to vote on.
    •  
  • Use it for high-stakes reasoning, not open-ended generation. EOF echo "created"