Self-Consistency Improves Chain of Thought Reasoning in Language Models

2 Self-Consistency over Diverse Reasoning Paths

1. State the three steps of self-consistency.
   (1) Prompt the model with manually written chain-of-thought exemplars; (2) sample a diverse set of reasoning paths from the decoder (e.g. temperature, top-\(k\) or nucleus sampling); (3) marginalize out the reasoning paths by choosing the most frequent final answer.

   • Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2
2. Formally, how is the final answer chosen?
   Each sample is a pair \((r_i, a_i)\): a reasoning path \(r_i\), treated as a latent variable, and a parsed answer \(a_i\) from a fixed answer set. The output is the majority vote \(\arg\max_a \sum_{i=1}^{m} \mathbb{1}(a_i = a)\).
   • Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2
3. How does majority voting compare with probability-weighted aggregation?
   Weighting each \((r_i, a_i)\) by its length-normalized probability gives nearly the same accuracy as an unweighted majority vote (unnormalized weighting is much worse), because the model assigns the sampled paths similar normalized probabilities: it is poorly calibrated at telling right from wrong solutions.
   • Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2
4. What is the main restriction on where self-consistency applies?
   The final answer must come from a fixed answer set so that answers can be compared and counted. In principle it extends to open-ended generation if a good consistency metric between generations is defined, e.g. whether two answers agree or contradict each other.
   • Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2
5. Why use sampling for reasoning tasks that have a single correct answer?
   Sampling is normally used for open-ended generation, and fixed-answer tasks have usually been decoded greedily; Wang et al. found that even when the answer is fixed, diversity in the reasoning paths is highly beneficial.
   • Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2