Self-Consistency Improves Chain of Thought Reasoning in Language Models

The greedy decoding of a single reasoning path. Self-consistency samples many diverse reasoning paths from the decoder and returns the answer most of them agree on.

A complex reasoning problem usually admits several different ways of thinking that lead to the same correct answer, while wrong reasoning paths tend to scatter over different wrong answers; if several independent lines of thought agree, the answer deserves more confidence.

The repetitiveness and local optimality of greedy decoding, and the randomness of relying on one sampled generation.

It needs no extra training, verifier, re-ranker, human annotation or fine-tuning, and it works off the shelf with one pretrained model, acting as a “self-ensemble” rather than aggregating several trained models.

(1) Prompt the model with manually written chain-of-thought exemplars; (2) sample a diverse set of reasoning paths from the decoder (e.g. temperature, top-\(k\) or nucleus sampling); (3) marginalize out the reasoning paths by choosing the most frequent final answer.

Each sample is a pair \((r_i, a_i)\): a reasoning path \(r_i\), treated as a latent variable, and a parsed answer \(a_i\) from a fixed answer set. The output is the majority vote \(\arg\max_a \sum_{i=1}^{m} \mathbb{1}(a_i = a)\).

Weighting each \((r_i, a_i)\) by its length-normalized probability gives nearly the same accuracy as an unweighted majority vote (unnormalized weighting is much worse), because the model assigns the sampled paths similar normalized probabilities: it is poorly calibrated at telling right from wrong solutions.

The final answer must come from a fixed answer set so that answers can be compared and counted. In principle it extends to open-ended generation if a good consistency metric between generations is defined, e.g. whether two answers agree or contradict each other.

Sampling is normally used for open-ended generation, and fixed-answer tasks have usually been decoded greedily; Wang et al. found that even when the answer is fixed, diversity in the reasoning paths is highly beneficial.

Arithmetic, commonsense and symbolic reasoning benchmarks; four models (UL2-20B, GPT-3 175B via Codex engines, LaMDA-137B, PaLM-540B); few-shot prompting with the same chain-of-thought exemplars as Wei et al. (2022) and no training; 40 sampled paths per question, averaged over 10 runs.

Large gains on all four models, e.g. GSM8K +17.9, SVAMP +11.0, AQuA +12.2, StrategyQA +6.4 and ARC-challenge +3.9 points (absolute), reaching new state of the art on most tasks without task-specific training.

They grew with scale: about +3–6 points for UL2-20B but +9–23 for LaMDA-137B and GPT-3; for small models gains are lower because abilities such as arithmetic only emerge at sufficient scale.

It rises consistently from 1 to 40 paths, but most of the gain comes early: in practice 5 or 10 paths capture most of the benefit at modest cost.

On NLI and closed-book QA tasks (e.g. ANLI-R1, e-SNLI, RTE), adding rationales via chain-of-thought sometimes did worse than standard prompting, but self-consistency recovered and surpassed standard prompting, making rationales safe to add.

Sample-and-rank keeps only the single most probable sequence, which helps far less than voting. Beam search produces less diverse outputs; even self-consistency over beam-search paths does worse than over sampled paths, since diversity of reasoning is the key.

Majority votes over greedy outputs from 40 prompt-order permutations or from 3 hand-written prompt sets gave much smaller gains than 40 sampled paths from one prompt; the methods are compatible and can be combined.

The share of sampled paths agreeing with the final answer is highly correlated with accuracy, so low consistency signals low confidence: the model gains some ability to “know when it doesn’t know”.

It helps with imperfect prompts (exemplars with wrong intermediate numbers), with equation-only reasoning (less gain, since short equations leave less room for diversity), and with zero-shot chain-of-thought (+26.2 points on GSM8K with PaLM-540B), and across sampling settings and model sizes.

It multiplies inference cost by the number of sampled paths, and models still sometimes produce incorrect or nonsensical reasoning paths even when the voted answer is right, so rationales need better grounding.