ReAct: Synergizing Reasoning and Acting in Language Models

3.3 Results and Observations

1. How did ReAct compare with Act and CoT on HotpotQA and FEVER (PaLM-540B)?
   ReAct beat Act on both (27.4 vs 25.7 EM; 60.9 vs 58.9 accuracy). It beat CoT on FEVER (60.9 vs 56.3), where retrieving exact facts matters, but trailed it slightly on HotpotQA (27.4 vs 29.4). The combinations did best: 35.1 EM (ReAct → CoT-SC) and 64.6 accuracy (CoT-SC → ReAct).
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §3.3
2. How did ReAct’s and CoT’s errors differ on HotpotQA?
   Hallucination was CoT’s main failure (56% of failures; 14% of its successes were false positives vs 6% for ReAct). ReAct was more grounded but less flexible: more reasoning errors, including loops repeating earlier thoughts and actions, and uninformative searches that derailed it (23% of errors).
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §3.3
3. What happened when smaller models were fine-tuned on ReAct trajectories?
   Prompting ReAct was worst for PaLM-8B/62B, but after fine-tuning on 3,000 correct trajectories it was best: fine-tuned 8B ReAct beat all 62B prompting methods. Fine-tuning on Standard or CoT outputs teaches memorizing facts; ReAct teaches how to look information up, a more general skill.
   • Yao et al., 2023, ReAct (The Eleventh International Conference on Learning Representations (ICLR 2023)), §3.3