On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

1 Introduction

1. What question does “On the Dangers of Stochastic Parrots” ask, and what does it recommend?
   How big is too big for language models, what risks come with ever larger LMs, and how can they be mitigated? It recommends weighing environmental and financial costs first, curating and documenting data sets instead of ingesting the web, pre-development exercises on how a project fits research goals and stakeholder values, and research beyond ever larger LMs.
   • Bender et al., 2021, On the Dangers of Stochastic Parrots (Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21)), §1, pp. 610–611
2. Which four families of risk does the Stochastic Parrots paper discuss?
   Environmental and financial costs (§3); unfathomable training data that encodes hegemonic views and biases (§4); misdirected research effort, mistaking form manipulation for understanding (§5); and harms from people taking seemingly coherent synthetic text as meaningful (§6).
   • Bender et al., 2021, On the Dangers of Stochastic Parrots (Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21)), §1, pp. 610–611