ReAct: Synergizing Reasoning and Acting in Language Models

4 Decision Making Tasks

1. How did ReAct do on ALFWorld and WebShop?
   With one or two in-context examples it beat imitation and reinforcement learning agents trained on \(10^3\)–\(10^5\) instances: on ALFWorld 71% success (best trial) vs 45% for Act and 37% for BUTLER; on WebShop a 10-point absolute gain in success rate over the previous best, though still far below expert humans.
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §4
2. Why did Act fail on ALFWorld without thoughts?
   Without thoughts it couldn’t decompose goals into subgoals and lost track of the environment state, e.g. repeatedly trying to take an object from a place where it wasn’t.
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §4
3. How does ReAct differ from Inner Monologue, and what did the comparison show?
   Inner Monologue feeds back only environment observations and what remains to be done; ReAct’s thoughts are flexible and sparse (goal decomposition, commonsense, progress). An IM-style variant reached 53% vs ReAct’s 71% on ALFWorld, often misjudging subgoals or where items were.
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §4