ReAct: Synergizing Reasoning and Acting in Language Models

2 ReAct: Synergizing Reasoning + Acting

1. How does ReAct formalize thoughts within an agent’s action space?
   It augments the action space to \(\hat{\mathcal{A}} = \mathcal{A} \cup \mathcal{L}\), where \(\mathcal{L}\) is the space of language. A thought \(\hat a_t \in \mathcal{L}\) does not affect the environment and yields no observation; it only updates the context, \(c_{t+1} = (c_t, \hat a_t)\), to support later reasoning or acting.

   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2
2. What kinds of useful thoughts appear in ReAct trajectories?
   Decomposing the goal into a plan, injecting commonsense knowledge (e.g. where an object is likely to be), extracting the important part of an observation, tracking progress and moving to the next subgoal, and handling exceptions by adjusting the plan or reformulating a search.
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2
3. How dense are thoughts in ReAct trajectories for different tasks?
   For reasoning-heavy tasks like QA, thoughts and actions alternate (thought–action–observation steps). For decision-making tasks with many actions, thoughts are sparse and the model decides itself when to insert one.
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2
4. How were ReAct prompts built, and why is that an advantage?
   Annotators wrote a few (one to six) task-solving trajectories, simply typing their thoughts in language on top of their actions; no special format, thought design or example selection was needed, and humans can later inspect and even edit thoughts to correct the agent.
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §2