ReAct: Synergizing Reasoning and Acting in Language Models

It prompts a language model to produce reasoning traces and task actions interleaved: reasoning helps it create, track and adjust plans and handle exceptions (“reason to act”), while actions query external sources such as a Wikipedia API or an environment to bring new information into the reasoning (“act to reason”).

Chain-of-thought is a static black box: the model reasons only from its internal representations, ungrounded in the external world, so it can’t update its knowledge and suffers from hallucinated facts and error propagation.

Those systems mapped observations to actions or plans using language priors but didn’t use the model to reason abstractly about high-level goals or keep a working memory to support acting.

People interleave actions with inner speech, e.g. while cooking: tracking progress (“everything is cut, now heat the water”), adjusting plans (“no salt, so use soy sauce”), and noticing when outside information is needed (“how do I make dough? let me look it up”).

It augments the action space to \(\hat{\mathcal{A}} = \mathcal{A} \cup \mathcal{L}\), where \(\mathcal{L}\) is the space of language. A thought \(\hat a_t \in \mathcal{L}\) does not affect the environment and yields no observation; it only updates the context, \(c_{t+1} = (c_t, \hat a_t)\), to support later reasoning or acting.

Decomposing the goal into a plan, injecting commonsense knowledge (e.g. where an object is likely to be), extracting the important part of an observation, tracking progress and moving to the next subgoal, and handling exceptions by adjusting the plan or reformulating a search.

For reasoning-heavy tasks like QA, thoughts and actions alternate (thought–action–observation steps). For decision-making tasks with many actions, thoughts are sparse and the model decides itself when to insert one.

Annotators wrote a few (one to six) task-solving trajectories, simply typing their thoughts in language on top of their actions; no special format, thought design or example selection was needed, and humans can later inspect and even edit thoughts to correct the agent.

A deliberately simple Wikipedia API: search[entity] (first five sentences of the page, or five similar titles), lookup[string] (next sentence containing the string, like Ctrl+F) and finish[answer]. It is far weaker than real retrievers, forcing retrieval to be driven by explicit reasoning.

Standard prompting (no thoughts, actions or observations), chain-of-thought (thoughts only), CoT with self-consistency (21 samples, majority vote) and Act (actions and observations only, without thoughts).

Two back-off heuristics: ReAct → CoT-SC falls back to self-consistency when ReAct gives no answer within a step limit (7 for HotpotQA, 5 for FEVER); CoT-SC → ReAct switches to ReAct when the majority answer gets fewer than half of the samples, i.e. internal knowledge seems unsure.

ReAct beat Act on both (27.4 vs 25.7 EM; 60.9 vs 58.9 accuracy). It beat CoT on FEVER (60.9 vs 56.3), where retrieving exact facts matters, but trailed it slightly on HotpotQA (27.4 vs 29.4). The combinations did best: 35.1 EM (ReAct → CoT-SC) and 64.6 accuracy (CoT-SC → ReAct).

Hallucination was CoT’s main failure (56% of failures; 14% of its successes were false positives vs 6% for ReAct). ReAct was more grounded but less flexible: more reasoning errors, including loops repeating earlier thoughts and actions, and uninformative searches that derailed it (23% of errors).

Prompting ReAct was worst for PaLM-8B/62B, but after fine-tuning on 3,000 correct trajectories it was best: fine-tuned 8B ReAct beat all 62B prompting methods. Fine-tuning on Standard or CoT outputs teaches memorizing facts; ReAct teaches how to look information up, a more general skill.

With one or two in-context examples it beat imitation and reinforcement learning agents trained on \(10^3\)–\(10^5\) instances: on ALFWorld 71% success (best trial) vs 45% for Act and 37% for BUTLER; on WebShop a 10-point absolute gain in success rate over the previous best, though still far below expert humans.

Without thoughts it couldn’t decompose goals into subgoals and lost track of the environment state, e.g. repeatedly trying to take an object from a place where it wasn’t.

Inner Monologue feeds back only environment observations and what remains to be done; ReAct’s thoughts are flexible and sparse (goal decomposition, commonsense, progress). An IM-style variant reached 53% vs ReAct’s 71% on ALFWorld, often misjudging subgoals or where items were.

Complex tasks with large action spaces need more demonstrations than fit in the context window, so more training data and fine-tuning are needed; and connecting language models to real actions risks retrieving private information or doing harm, which they limited by using only Wikipedia and a simulated shop.