On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

Bender et al., 2021, On the Dangers of Stochastic Parrots (Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21))

Subscriber deck — free during early access

1 cards in “5 Down the Garden Path” · whole deck

5 Down the Garden Path

What is the “garden path” risk of large LMs for research?
Research effort flows into beating NLU benchmarks with LMs that manipulate form well enough to pass tests without understanding (shown by removing spurious cues), an opportunity cost against meaning-capturing approaches and against technology built from carefully curated data for more languages.
- Bender et al., 2021, On the Dangers of Stochastic Parrots (Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21)), §5, pp. 615–616

Source

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), pp. 610–623. License: CC BY 4.0. Flashcards adapted from this work. doi:10.1145/3442188.3445922