Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

4 The octopus test

1. Describe Bender and Koller’s octopus test.
   Two English speakers, A and B, stranded on islands, chat by underwater telegraph. A hyper-intelligent octopus O taps the cable, learns the statistics of their messages, then cuts B off and impersonates B. Can O fool A, having seen only form?
   • Bender & Koller, 2020, Climbing towards NLU (Proceedings of the 58th Annual Meeting of ACL), §4, pp. 5188–5189
2. Where does the octopus succeed and where does it fail?
   Social small talk, which needs only internally coherent text, may fool A. With A’s coconut catapult instructions, O can only reply generically (“Cool idea, great job!”), and A does the work of attributing meaning. When A urgently asks how to fight a bear with sticks, O fails: that needs mapping words to real-world entities.
   • Bender & Koller, 2020, Climbing towards NLU (Proceedings of the 58th Annual Meeting of ACL), §4, pp. 5188–5189
3. What does the octopus test show about who creates meaning in such exchanges?
   It isn’t that O’s utterances make sense, but that A can make sense of them: A assumes the sender has communicative intents (as English speakers usually do) and attributes intent from conventional meaning and her model of B.
   • Bender & Koller, 2020, Climbing towards NLU (Proceedings of the 58th Annual Meeting of ACL), §4, pp. 5188–5189