Main
SuperAgent_Autogenagentsllmsvector-databaselancedbgptopenaiAImultimodal-aimachine-learningembeddingsfine-tuningexamplesdeep-learninggpt-4-visionllama-indexragmultimodallangchainlancedb-recipes
Export
Super Agent: Integrating Autogen Technologies and Vector Databases
The Super Agent project showcases the integration of cutting-edge technologies including Ollama, LiteLLM, Autogen, LanceDB, and LangChain to create a powerful AI agent. This agent leverages the strengths of vector databases and conversational AI to provide sophisticated data management and context-aware interactions. The goal is to harness the capabilities of LanceDB's vector database and Autogen's conversational AI framework to build a Super Agent that excels in understanding and processing complex queries.
This notebook illustrates how you can use the Langchain with custom PDF data & chat it using an Autogen Agent.
[1]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 88.8/88.8 kB 3.8 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 20.4 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 28.0 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.0/77.0 kB 3.6 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 13.5 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 27.4/27.4 MB 9.3 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.5/30.5 MB 28.5 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.0/298.0 kB 16.8 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 409.5/409.5 kB 22.1 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.5/45.5 kB 2.6 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 313.9/313.9 kB 16.2 MB/s eta 0:00:00 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.5/49.5 kB 2.8 MB/s eta 0:00:00
[5]
Requirements AutoGen requires Python>=3.8.
To run this notebook example, please install pyautogen:
[3]
Requirement already satisfied: pyautogen in /usr/local/lib/python3.10/dist-packages (0.1.14) Requirement already satisfied: diskcache in /usr/local/lib/python3.10/dist-packages (from pyautogen) (5.6.3) Requirement already satisfied: flaml in /usr/local/lib/python3.10/dist-packages (from pyautogen) (2.3.2) Requirement already satisfied: openai<1 in /usr/local/lib/python3.10/dist-packages (from pyautogen) (0.28.1) Requirement already satisfied: python-dotenv in /usr/local/lib/python3.10/dist-packages (from pyautogen) (1.0.1) Requirement already satisfied: termcolor in /usr/local/lib/python3.10/dist-packages (from pyautogen) (2.5.0) Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.10/dist-packages (from openai<1->pyautogen) (2.32.3) Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai<1->pyautogen) (4.66.6) Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from openai<1->pyautogen) (3.11.2) Requirement already satisfied: NumPy>=1.17 in /usr/local/lib/python3.10/dist-packages (from flaml->pyautogen) (1.26.4) Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<1->pyautogen) (3.4.0) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<1->pyautogen) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<1->pyautogen) (2.2.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai<1->pyautogen) (2024.8.30) Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (2.4.3) Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (1.3.1) Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (24.2.0) Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (1.5.0) Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (6.1.0) Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (0.2.0) Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (1.17.2) Requirement already satisfied: async-timeout<6.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai<1->pyautogen) (4.0.3) Requirement already satisfied: typing-extensions>=4.1.0 in /usr/local/lib/python3.10/dist-packages (from multidict<7.0,>=4.5->aiohttp->openai<1->pyautogen) (4.12.2)
Add OpenAI API key as env variable
[6]
Import data
This example utilizes "Food and Nutrition handbook by MINISTRY OF AGRICULTURE, ANIMAL INDUSTRY AND FISHERIES" PDF which is imported from following code.
[8]
--2024-12-04 18:08:56-- https://pdf.usaid.gov/pdf_docs/PA00TBCT.pdf Resolving pdf.usaid.gov (pdf.usaid.gov)... 23.54.221.184, 2600:1408:c400:1687::1923, 2600:1408:c400:1697::1923 Connecting to pdf.usaid.gov (pdf.usaid.gov)|23.54.221.184|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6419525 (6.1M) [application/pdf] Saving to: ‘food.pdf’ food.pdf 100%[===================>] 6.12M --.-KB/s in 0.08s 2024-12-04 18:08:56 (74.1 MB/s) - ‘food.pdf’ saved [6419525/6419525]
create file name with OAI_CONFIG_LIT.json
[12]
create OAI_CONFIG_LIST.json file in pwd & upload in it
[ { "model": "gpt-4", "api_key": "sk-yourapikey" } ]
[13]
[14]
[15]
[16]
<ipython-input-16-70ff1d5b784c>:3: LangChainDeprecationWarning: The class `OpenAI` was deprecated in LangChain 0.0.10 and will be removed in 1.0. An updated version of the class exists in the :class:`~langchain-openai package and should be used instead. To use it run `pip install -U :class:`~langchain-openai` and import as `from :class:`~langchain_openai import OpenAI``. OpenAI( <ipython-input-16-70ff1d5b784c>:7: LangChainDeprecationWarning: Please see the migration guide at: https://python.langchain.com/docs/versions/migrating_memory/ memory=ConversationBufferMemory(memory_key="chat_history", return_messages=True),
Start Querying
Now we are ready to start query and get answers
[17]
[18]
<ipython-input-17-e9499bd82ace>:3: LangChainDeprecationWarning: The method `Chain.__call__` was deprecated in langchain 0.1.0 and will be removed in 1.0. Use :meth:`~invoke` instead.
response = qa({"question": question})
' Good food is food that provides the recommended amounts of nutrients for the body to perform all its physiological activities. It is important to eat the right types of food, in the right amounts, at the right time, and prepared in the correct way and place. Good food is essential for physical and cognitive development, and it can improve overall health and quality of life.'
Autogen
Now lets try to go same using Autogen
[21]
Set up AutoGen user agent and assistant agent with function calling enabled.
[20]
[22]
[23]
user_proxy (to assistant):
what is good food?
--------------------------------------------------------------------------------
assistant (to user_proxy):
***** Suggested function Call: answer_food_question *****
Arguments:
{
"question": "what is good food?"
}
*********************************************************
--------------------------------------------------------------------------------
>>>>>>>> EXECUTING FUNCTION answer_food_question...
user_proxy (to assistant):
***** Response from calling function "answer_food_question" *****
Good food is any type of liquid, semi-solid, or solid substance that contains nutrients and nourishes the body when consumed. It is important because it provides the body with essential nutrients, promotes good health, and helps prevent illnesses and health problems. Good food should be eaten in appropriate amounts, at the right time, and prepared correctly to ensure a balanced diet. It should also include a variety of foods from each food group to meet the body's different needs.
*****************************************************************
--------------------------------------------------------------------------------
assistant (to user_proxy):
Good food is any type of liquid, semi-solid, or solid substance that contains nutrients and nourishes the body when consumed. It is important because it provides the body with essential nutrients, promotes good health, and helps prevent illnesses and health problems. Good food should be eaten in appropriate amounts, at the right time, and prepared correctly to ensure a balanced diet. It should also include a variety of foods from each food group to meet the body's different needs.
TERMINATE
--------------------------------------------------------------------------------
user_proxy (to assistant):
TERMINATE
--------------------------------------------------------------------------------
[24]
user_proxy (to assistant): please explain me essential minerals, sources, functions and symptoms of deficiency? -------------------------------------------------------------------------------- assistant (to user_proxy): Sure, here are some essential minerals, their sources, functions, and symptoms of deficiency: 1. Calcium: - Sources: Dairy products, leafy green vegetables, fish with edible bones (like sardines and salmon), fortified foods. - Functions: Necessary for bone health, muscle function, nerve transmission, blood clotting. - Deficiency Symptoms: Osteoporosis, muscle cramps, dental problems, brittle nails. 2. Iron: - Sources: Red meat, poultry, eggs, fruits, green vegetables, fortified bread. - Functions: Essential for the production of red blood cells, carries oxygen in the blood. - Deficiency Symptoms: Anemia, fatigue, weakness, pale skin, shortness of breath. 3. Potassium: - Sources: Bananas, oranges, cantaloupes, raisins, nuts, fish, and whole grains. - Functions: Helps maintain fluid balance, helps control the electrical activity of the heart, helps with muscle contractions. - Deficiency Symptoms: Weakness, fatigue, muscle cramps, constipation. 4. Magnesium: - Sources: Green leafy vegetables, nuts, seeds, dry beans, whole grains, and low-fat dairy products. - Functions: Involved in over 300 biochemical reactions in the body, including nerve and muscle function, bone health, and regulating blood sugar levels. - Deficiency Symptoms: Loss of appetite, nausea, fatigue, and weakness. In severe cases, numbness, muscle cramps, seizures, personality changes, abnormal heart rhythms. 5. Zinc: - Sources: Meat, shellfish, legumes, seeds, nuts, dairy, eggs, whole grains. - Functions: Supports the immune system, necessary for the sense of taste and smell, needed for wound healing. - Deficiency Symptoms: Loss of appetite, impaired immune function, hair loss, diarrhea, delayed sexual maturation, impotence. Remember, it's always best to get your minerals from a balanced diet rather than supplements unless advised by a healthcare provider. Overdosing on minerals can be harmful. Always consult with a healthcare provider for personalized advice. -------------------------------------------------------------------------------- user_proxy (to assistant): TERMINATE --------------------------------------------------------------------------------
[25]
user_proxy (to assistant):
which food Keeps eyes healthy ?
--------------------------------------------------------------------------------
assistant (to user_proxy):
***** Suggested function Call: answer_food_question *****
Arguments:
{
"question": "which food Keeps eyes healthy ?"
}
*********************************************************
--------------------------------------------------------------------------------
>>>>>>>> EXECUTING FUNCTION answer_food_question...
user_proxy (to assistant):
***** Response from calling function "answer_food_question" *****
Foods rich in Vitamin A, such as liver, egg yolk, dairy products, and dark green and deep yellow fruits and vegetables, can help maintain healthy eyes.
*****************************************************************
--------------------------------------------------------------------------------
assistant (to user_proxy):
Foods that are rich in Vitamin A can help maintain healthy eyes. These include:
1. Liver
2. Egg yolk
3. Dairy products
4. Dark green fruits and vegetables
5. Deep yellow fruits and vegetables
These foods are beneficial for eye health because Vitamin A plays a crucial role in vision by maintaining a clear cornea, which is the outside covering of your eye.
TERMINATE
--------------------------------------------------------------------------------
user_proxy (to assistant):
TERMINATE
--------------------------------------------------------------------------------