Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023))
Subscriber deck — free during early access
2 cards in “3.4 Compare to Other Existing Approaches” · whole deck
3.4 Compare to Other Existing Approaches
- Why does self-consistency beat sample-and-rank and beam search?Sample-and-rank keeps only the single most probable sequence, which helps far less than voting. Beam search produces less diverse outputs; even self-consistency over beam-search paths does worse than over sampled paths, since diversity of reasoning is the key.
- Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023)), §3.4
- How does self-consistency compare with prompt ensembles?Majority votes over greedy outputs from 40 prompt-order permutations or from 3 hand-written prompt sets gave much smaller gains than 40 sampled paths from one prompt; the methods are compatible and can be combined.
- Wang et al., 2023, Self-Consistency Improves Chain of Thought Reasoning in Language Models (The Eleventh International Conference on Learning Representations (ICLR 2023)), §3.4
Source
- Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In The Eleventh International Conference on Learning Representations (ICLR 2023). https://arxiv.org/abs/2203.11171