Athina Rewoo

Rewoo

llmsathina-rag-cookbooksagent_techniquesopenaiAItutorialsChromaDBcookbooksfaissqdrantLLMPythonpineconeraglangchainweaviate

alph-notebooks/athina-rag-cookbooks / rewoo.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

ReWOO (Reasoning WithOut Observation)

ReWOO (Reasoning WithOut Observation) is a novel prompting paradigm designed to enhance the efficiency and accuracy of Augmented Language Models (ALMs) by decoupling reasoning from observation-dependent interactions with external tools. This approach mitigates issues such as high computational cost and token redundancy prevalent in conventional ALM methodologies.

Key Concepts of ReWOO:

Decoupled Reasoning and Observation: Unlike traditional ALMs, which require LLMs to halt their reasoning process to fetch external tool responses, ReWOO enables models to generate a structured reasoning plan before interacting with external data sources.
Modular Plan-Work-Solve Framework:
- Planner: Constructs a comprehensive plan outlining logical steps required to solve the given task.
- Worker: Retrieves the necessary information from external tools based on the Planner’s blueprint.
- Solver: Synthesizes retrieved evidence and plans to generate the final answer.
Token Efficiency: ReWOO reduces token consumption by eliminating redundant context repetitions, achieving up to 5× token efficiency compared to interleaved reasoning paradigms like ReAct.
Scalability and Specialization: By decoupling parametric reasoning from non-parametric tool calls, ReWOO allows for instruction fine-tuning, enabling smaller models like LLaMA 7B to approximate the reasoning capabilities of larger models such as GPT-3.5.
Robustness Against Tool Failures: In scenarios where external tools fail or return incomplete results, ReWOO remains resilient by structuring logical inferences before calling tools, reducing dependency on external observations.

Advantages of ReWOO:

Lower Computational Overhead: Reduces prompt redundancy, cutting down costs in commercial LLM APIs.
Improved Accuracy: Achieves a 4% improvement in reasoning accuracy on complex tasks like HotpotQA.
Enhanced Generalization: Facilitates the transition of reasoning abilities from large models to smaller, fine-tuned models, making ALMs more accessible and scalable.
Better Multi-Step Task Handling: Suitable for multi-hop reasoning and fact-checking applications that require structured workflows rather than continuous interruptions.

ReWOO marks a significant shift in how LLMs interact with external knowledge, optimizing efficiency and enhancing performance for real-world AI applications.

Paper link: https://arxiv.org/pdf/2305.18323

Code Reference: https://github.com/langchain-ai/langgraph/blob/main/docs/docs/tutorials/rewoo/rewoo.ipynb

[ ]

Plan: Begin by identifying the winner of the 2024 Men's Australian Open using websearch. This will provide the name of the player who won the tournament.
#E1 = websearch["2024 Men's Australian Open winner"]

Plan: Once we have the winner's name, search for the winner's exact hometown using websearch, as this specific information might not be widely known but potentially available from reliable sources like the player's official profiles or other reputable databases.
#E2 = websearch["hometown of 2024 Men's Australian Open winner #E1"]

[ ]

[ ]

{'plan': {'steps': [("Search for the winner of the 2024 Men's Australian Open.", '#E1', 'websearch', '"2024 Men\'s Australian Open winner"'), ("Retrieve details about the winner's hometown using the search result from #E1.", '#E2', 'LLM', '"Find the hometown of #E1, the 2024 Men\'s Australian Open winner"')], 'plan_string': 'To solve this task, we need to find out the winner of the 2024 Men\'s Australian Open and then determine their exact hometown. Here\'s how we can approach it step-by-step:\n\nPlan: Search for the winner of the 2024 Men\'s Australian Open.\n#E1 = websearch["2024 Men\'s Australian Open winner"]\n\nPlan: Retrieve details about the winner\'s hometown using the search result from #E1.\n#E2 = LLM["Find the hometown of #E1, the 2024 Men\'s Australian Open winner"]\n\nNow, with these steps, we will be able to identify both the winner\'s name and their exact hometown.'}}
---
{'tool': {'results': {'#E1': '[{\'url\': \'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-19-23.pdf\', \'content\': "2024 MEN\'S AUSTRALIAN OPEN WINNER. PLAYER VS FIELD ODDS. FIELD. PLAYER. EV 42202 -130 JANNIK SINNER. 42203 +365 42204 -470 DANIIL MEDVEDEV."}, {\'url\': \'https://vsin.com/wp-content/uploads/2024/01/circa-sheets-1-21-24.pdf\', \'content\': "2024 MEN\'S AUSTRALIAN OPEN WINNER. PLAYER. 42201. 42203. PLAYER VS FIELD ODDS. FIELD. EV 42202 -130 JANNIK SINNER. +365 42204 -470 DANIIL"}, {\'url\': \'https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/\', \'content\': \'[](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-1)Plan: Use Google to search for the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-4)Plan: Retrieve the name of the 2024 Australian Open winner from the search results. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-7)Plan: Use Google to search for the hometown of the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-1){\\\'plan\\\': {\\\'plan_string\\\': "Plan: Use Google to search for the 2024 Men\\\'s Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-3){\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\', \\\\\\\'content\\\\\\\': "Qinwen Zheng, 6-3, 6-2\\\\\\\\nOur Latest Tennis Stories\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\n2024 Australian Open odds, Sinner vs. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-5){\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\', \\\\\\\'content\\\\\\\': "Qinwen Zheng, 6-3, 6-2\\\\\\\\nOur Latest Tennis Stories\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\n2024 Australian Open odds, Sinner vs.\'}, {\'url\': \'https://blog.csdn.net/qq_41472205/article/details/144353195\', \'content\': \'\\\', \\\'#E3\\\', \\\'Google\\\', \\\'"[Winner\\\\\\\'s Name\\\')], \\\'plan_string\\\': \\\'Plan: Search for the winner of the 2024 Men\\\\\\\'s Australian Open tennis tournament. {\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\'s Australian Open winner]\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown"}, {\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\'s Australian Open winner]\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown"}, {\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\'s Australian Open winner]\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown"}, {\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER.\'}, {\'url\': \'https://www.bets.com.au/tennis/australian-open/2024-mens-australian-open-winner-betting-odds-20230206-0009/\', \'content\': "Australian Open 2025 Tips, Predictions & Live Streaming Australian Open 2025 Betting Tips, Predictions & Live Streaming ### Aus Open Jannik Sinner vs Alexander Zverev Tips, Prediction & Live Stream - Sinner to defend Australian Open title ### Aus Open Aryna Sabalenka vs Madison Keys Tips, Prediction & Live Stream - Keys to defy the odds and win first Grand Slam title Our team of tennis experts at Bets.com.au provides betting tips, match previews and live streaming guides for the 2025 Australian Open. 2025 Women\'s Australian Open Betting Odds Betting On The Australian Open Online bookmaker\\xa0bet365\\xa0offer their customers a comprehensive live streaming service, which broadcasts Grand Slam tennis all year - including the Australian Open. ATP Tour Australian Open"}]'}}}
---
{'tool': {'results': {'#E1': '[{\'url\': \'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-19-23.pdf\', \'content\': "2024 MEN\'S AUSTRALIAN OPEN WINNER. PLAYER VS FIELD ODDS. FIELD. PLAYER. EV 42202 -130 JANNIK SINNER. 42203 +365 42204 -470 DANIIL MEDVEDEV."}, {\'url\': \'https://vsin.com/wp-content/uploads/2024/01/circa-sheets-1-21-24.pdf\', \'content\': "2024 MEN\'S AUSTRALIAN OPEN WINNER. PLAYER. 42201. 42203. PLAYER VS FIELD ODDS. FIELD. EV 42202 -130 JANNIK SINNER. +365 42204 -470 DANIIL"}, {\'url\': \'https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/\', \'content\': \'[](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-1)Plan: Use Google to search for the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-4)Plan: Retrieve the name of the 2024 Australian Open winner from the search results. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-7)Plan: Use Google to search for the hometown of the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-1){\\\'plan\\\': {\\\'plan_string\\\': "Plan: Use Google to search for the 2024 Men\\\'s Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-3){\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\', \\\\\\\'content\\\\\\\': "Qinwen Zheng, 6-3, 6-2\\\\\\\\nOur Latest Tennis Stories\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\n2024 Australian Open odds, Sinner vs. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-5){\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\', \\\\\\\'content\\\\\\\': "Qinwen Zheng, 6-3, 6-2\\\\\\\\nOur Latest Tennis Stories\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\n2024 Australian Open odds, Sinner vs.\'}, {\'url\': \'https://blog.csdn.net/qq_41472205/article/details/144353195\', \'content\': \'\\\', \\\'#E3\\\', \\\'Google\\\', \\\'"[Winner\\\\\\\'s Name\\\')], \\\'plan_string\\\': \\\'Plan: Search for the winner of the 2024 Men\\\\\\\'s Australian Open tennis tournament. {\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\'s Australian Open winner]\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown"}, {\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\'s Australian Open winner]\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown"}, {\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\'tool\\\': {\\\'results\\\': {\\\'#E1\\\': \\\'[{\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\'s Australian Open winner]\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown"}, {\\\\\\\'url\\\\\\\': \\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\', \\\\\\\'content\\\\\\\': "2024 MEN\\\\\\\'S AUSTRALIAN OPEN WINNER.\'}, {\'url\': \'https://www.bets.com.au/tennis/australian-open/2024-mens-australian-open-winner-betting-odds-20230206-0009/\', \'content\': "Australian Open 2025 Tips, Predictions & Live Streaming Australian Open 2025 Betting Tips, Predictions & Live Streaming ### Aus Open Jannik Sinner vs Alexander Zverev Tips, Prediction & Live Stream - Sinner to defend Australian Open title ### Aus Open Aryna Sabalenka vs Madison Keys Tips, Prediction & Live Stream - Keys to defy the odds and win first Grand Slam title Our team of tennis experts at Bets.com.au provides betting tips, match previews and live streaming guides for the 2025 Australian Open. 2025 Women\'s Australian Open Betting Odds Betting On The Australian Open Online bookmaker\\xa0bet365\\xa0offer their customers a comprehensive live streaming service, which broadcasts Grand Slam tennis all year - including the Australian Open. ATP Tour Australian Open"}]', '#E2': 'content="Jannik Sinner won the 2024 Men\'s Australian Open. He is from San Candido, Italy." additional_kwargs={\'refusal\': None} response_metadata={\'token_usage\': {\'completion_tokens\': 23, \'prompt_tokens\': 1527, \'total_tokens\': 1550, \'completion_tokens_details\': {\'accepted_prediction_tokens\': 0, \'audio_tokens\': 0, \'reasoning_tokens\': 0, \'rejected_prediction_tokens\': 0}, \'prompt_tokens_details\': {\'audio_tokens\': 0, \'cached_tokens\': 0}}, \'model_name\': \'gpt-4o-2024-08-06\', \'system_fingerprint\': \'fp_523b9b6e5f\', \'finish_reason\': \'stop\', \'logprobs\': None} id=\'run-90236759-f030-4f91-a108-6d89e34bf0c6-0\' usage_metadata={\'input_tokens\': 1527, \'output_tokens\': 23, \'total_tokens\': 1550, \'input_token_details\': {\'audio\': 0, \'cache_read\': 0}, \'output_token_details\': {\'audio\': 0, \'reasoning\': 0}}'}}}
---
{'solve': {'result': 'San Candido, Italy.'}}
---

[ ]

{'plan': {'steps': [('Identify the main ingredient in The Hennchata utilizing general awareness about the drink.', '#E1', 'LLM', 'What is the main ingredient in The Hennchata?'), ('Once the main ingredient is identified, find out which cognac house produces this ingredient by using a web search.', '#E2', 'websearch', 'Which cognac house makes #E1?')], 'plan_string': 'To determine the cognac house responsible for making the main ingredient in The Hennchata, I will begin by understanding what the main ingredient in The Hennchata is. From this, I can identify the cognac house.\n\nPlan: Identify the main ingredient in The Hennchata utilizing general awareness about the drink.\n#E1 = LLM[What is the main ingredient in The Hennchata?]\n\nPlan: Once the main ingredient is identified, find out which cognac house produces this ingredient by using a web search.\n#E2 = websearch[Which cognac house makes #E1?]'}}
---
{'tool': {'results': {'#E1': 'content=\'The main ingredients in "The Hennchata" are typically Hennessy cognac and horchata. This cocktail combines the flavors of the smooth, bold cognac with the sweet, creamy, and spiced rice drink, horchata. Some variations of the drink might include additional spices or sweeteners, but Hennessy and horchata are the key components.\' additional_kwargs={\'refusal\': None} response_metadata={\'token_usage\': {\'completion_tokens\': 78, \'prompt_tokens\': 19, \'total_tokens\': 97, \'completion_tokens_details\': {\'accepted_prediction_tokens\': 0, \'audio_tokens\': 0, \'reasoning_tokens\': 0, \'rejected_prediction_tokens\': 0}, \'prompt_tokens_details\': {\'audio_tokens\': 0, \'cached_tokens\': 0}}, \'model_name\': \'gpt-4o-2024-08-06\', \'system_fingerprint\': \'fp_50cad350e4\', \'finish_reason\': \'stop\', \'logprobs\': None} id=\'run-29d0f354-7f44-4573-8a89-3cc6ee70aed5-0\' usage_metadata={\'input_tokens\': 19, \'output_tokens\': 78, \'total_tokens\': 97, \'input_token_details\': {\'audio\': 0, \'cache_read\': 0}, \'output_token_details\': {\'audio\': 0, \'reasoning\': 0}}'}}}
---
{'tool': {'results': {'#E1': 'content=\'The main ingredients in "The Hennchata" are typically Hennessy cognac and horchata. This cocktail combines the flavors of the smooth, bold cognac with the sweet, creamy, and spiced rice drink, horchata. Some variations of the drink might include additional spices or sweeteners, but Hennessy and horchata are the key components.\' additional_kwargs={\'refusal\': None} response_metadata={\'token_usage\': {\'completion_tokens\': 78, \'prompt_tokens\': 19, \'total_tokens\': 97, \'completion_tokens_details\': {\'accepted_prediction_tokens\': 0, \'audio_tokens\': 0, \'reasoning_tokens\': 0, \'rejected_prediction_tokens\': 0}, \'prompt_tokens_details\': {\'audio_tokens\': 0, \'cached_tokens\': 0}}, \'model_name\': \'gpt-4o-2024-08-06\', \'system_fingerprint\': \'fp_50cad350e4\', \'finish_reason\': \'stop\', \'logprobs\': None} id=\'run-29d0f354-7f44-4573-8a89-3cc6ee70aed5-0\' usage_metadata={\'input_tokens\': 19, \'output_tokens\': 78, \'total_tokens\': 97, \'input_token_details\': {\'audio\': 0, \'cache_read\': 0}, \'output_token_details\': {\'audio\': 0, \'reasoning\': 0}}', '#E2': "HTTPError('400 Client Error: Bad Request for url: https://api.tavily.com/search')"}}}
---
{'solve': {'result': 'Hennessy'}}
---

[ ]

{'plan': {'steps': [('To find the capital of the United Kingdom, use general world knowledge to provide a confident answer.', '#E1', 'LLM', 'What is the capital of the United Kingdom?')], 'plan_string': 'Plan: To find the capital of the United Kingdom, use general world knowledge to provide a confident answer.\n#E1 = LLM[What is the capital of the United Kingdom?]'}}
---
{'tool': {'results': {'#E1': "content='The capital of the United Kingdom is London.' additional_kwargs={'refusal': None} response_metadata={'token_usage': {'completion_tokens': 10, 'prompt_tokens': 16, 'total_tokens': 26, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-2024-08-06', 'system_fingerprint': 'fp_076d6600c1', 'finish_reason': 'stop', 'logprobs': None} id='run-6cfca7e5-eb02-4478-b251-765dbf7ed925-0' usage_metadata={'input_tokens': 16, 'output_tokens': 10, 'total_tokens': 26, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}"}}}
---
{'solve': {'result': 'London'}}
---