Notebooks
M
MongoDB Pragmatic LLM Application Introduction From RAG To Agents With MongoDB

Pragmatic LLM Application Introduction From RAG To Agents With MongoDB

agentsartificial-intelligencellmsmongodb-genai-showcasenotebooksgenerative-airag
alph-notebooks/mongodb-genai-showcase / Pragmatic_LLM_Application_Introduction_From_RAG_to_Agents_with_MongoDB.ipynb

Pragmatic LLM Application Development: From RAG Pipleines to AI Agents

Open In Colab

A practical guide that introduces two forms of LLM Applications: RAG (Retrieval-Augmented Generation) pipelines and AI Agents.

This guide is designed to take you on a journey that develops your understanding of LLM Applications, starting with implementations without abstraction frameworks, and later introducing the implementation of RAG pipelines, AI agents, and other LLM application components using frameworks and libraries that alleviate the implementation burden for AI Stack Engineers.

Key topics covered:

  1. Document Model and MongoDB Integration: Introduces the Document model and its integration with MongoDB within LLM applications.

  2. RAG Pipeline Fundamentals: Guides you through the key processes within a RAG pipeline, including data embedding, data ingestion, and handling user queries.

  3. MongoDB Vector Database Integration: Guides you through the development of a RAG pipeline connected to a MongoDB Vector Database and utilizing OpenAI's models.

  4. MongoDB Aggregation Pipelines: Introduces MongoDB Aggregation pipelines and stages for efficient data retrieval implementation within pipelines.

  5. LLM Abstraction Frameworks: Showcases the development of RAG pipelines using widely-used LLM abstraction frameworks such as LangChain, LlamaIndex, and HayStack.

  6. Data Handling in LLM Applications: Presents methods for handling data in LLM applications using tools such as Pydantic and Pandas.

  7. AI Agent Implementation: Introduces the implementation of AI Agents using libraries such as LangChain and LlamaIndex.

  8. LLM Application Optimization: Introduces techniques for optimizing LLM Applications, such as prompt compression using the LLMLingua library.

Who is this for:

  • AI Engineers: Professionals responsible for developing generative AI applications will find practical guidance on implementing such systems.
  • AI Stack Engineers: Individuals working with AI Stack tools and libraries will gain insights into the implementation approaches employed by widely adopted libraries, enhancing their understanding and proficiency.
  • Software Engineers: For those seeking a straightforward introduction to LLM Applications, this guide provides a focused and concise exploration of the subject matter, without unnecessary verbosity or fluff.

Part 1: Vanilla RAG Application

Install Libaries

[ ]

Set Up OpenAI and MongoDB environment variables

[ ]
[ ]
  Cell In[1], line 3
    openai.api_key = os.environ.get("OPENAI_API_KEY
                                                   ^
SyntaxError: EOL while scanning string literal

1.1 Synthetic Data Creation

[ ]
[ ]
[ ]
Synthetic employee data has been saved to synthetic_data_employees.csv
[ ]

1.2 Embedding Data For Vector Search

[ ]
Here's what an employee string looks like: /n John Doe, Male, born on 1990-06-26. Job: Software Engineer in IT. Skills: Node.js, Flask, Docker, JavaScript. Reviews: Rated 3.7 on 2022-10-23: Outstanding performance and dedication. Rated 4.9 on 2021-07-24: Needs improvement in time management.. Location: Works at Singapore Office, Remote: True. Notes: Actively involved in company hackathons and innovation challenges.
[ ]
[ ]
Embeddings generated for employees
[ ]

1.3 Data Ingestion into MongoDB Database

Steps to creating a MongoDB Database

[ ]

To be able to connect your notebook to MongoDB Atlas, you need to your IP Access List

[ ]
[ ]
[ ]
[ ]
[ ]
Connection to MongoDB successful
[ ]
[ ]
DeleteResult({'n': 10, 'electionId': ObjectId('7fffffff0000000000000027'), 'opTime': {'ts': Timestamp(1718207302, 10), 't': 39}, 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1718207302, 10), 'signature': {'hash': b"\x8a\x85$\xbf\xed'\xc5\xf8\xe6\x1eJ5@w8\xf6\x82\xf3\x16u", 'keyId': 7320226449804230662}}, 'operationTime': Timestamp(1718207302, 10)}, acknowledged=True)
[ ]
Data ingestion into MongoDB completed

1.4 Vector Index Creation

  • Create an Atlas Vector Search Index

  • If you are following this notebook ensure that you are creating a vector search index for the right database(demo_company_employees) and collection(employees_records)

Below is the vector search index definition for this notebook

	{
  "fields": [
    {
      "numDimensions": 1536,
      "path": "embedding",
      "similarity": "cosine",
      "type": "vector"
    }
  ]
}
  • Give your vector search index the name "vector_index" if you are following this notebook

1.5 RAG with MongoDB

[ ]

1.6 Handling User Query

[ ]
[ ]
Uncompressed Prompt:

Answer this user query: Who is the CEO? with the following context: 
Response: Please provide the name of your company or any additional context that will help me identify the current CEO.

1.7 Handling User Query (With Prompt Compression)

[ ]
[ ]
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(

config.json:   0%|          | 0.00/875 [00:00<?, ?B/s]
tokenizer_config.json:   0%|          | 0.00/1.19k [00:00<?, ?B/s]
vocab.txt:   0%|          | 0.00/996k [00:00<?, ?B/s]
tokenizer.json:   0%|          | 0.00/2.92M [00:00<?, ?B/s]
special_tokens_map.json:   0%|          | 0.00/125 [00:00<?, ?B/s]
model.safetensors:   0%|          | 0.00/709M [00:00<?, ?B/s]
[ ]
[ ]
Token indices sequence length is longer than the specified maximum sequence length for this model (1434 > 512). Running this sequence through the model will result in indexing errors

------
{'compressed_prompt': 'Employee ID E123465 Olivia Martinez Female Birth 1952 - 01 - 05 959 Main Street Springfield IL 62704. martinez. 555 - 675 - 3033 CEO 2015 09 16 53606 USD Sydney. Performance Reviews 2020 - 10 - 09. 5 time management. Benefits Health Insurance Silver Plan Retirement Plan 401K PTO 28 Emergency Michael Doe - 555 - 465 - 9759 Received Employee of Month award 2022. Employee ID E123463 Chris Lee Male 1960 - 03 - 22 523 Main Street Springfield 62704. 928 5679 DevOps Engineer Department Operations 2001 07 28 227846 USD Toronto Docker AWS 2022 12 28. 5. 10 19 3. 6 expectations last project.Benefits Health Insurance - Gold Plan Retirement Plan 401K 26 days Emergency Contact Michael Smith Relationship Parent 555 - 613 - 9745 Completed leadership training 2021. Employee ID E123464 Sophia Garcia Birth 1998 - 06 - 13 374 Main Street Springfield IL 62704. 2568 CTO Executive 2011 06 17 128311 USD Nearest Office San Francisco. Performance 2023 - 11 - 02 3. 5. 2020 11 04. 4 time management. Gold Plan 401K 29 days Emily Doe Relationship Spouse 2565 Senior Software Engineer 2020. Employee ID E123456 John Doe Birth 1990 06 - 26 650 Main Street Springfield 62704. 2182 Software Engineer 68688 USD Office Singapore.Flask Docker JavaScript Performance Reviews 2022 - 10 - 23 3. 7 performance dedication. 07 24 4. 9 time management. Health Insurance Silver Plan 401K 28 days Emergency Contact Jane Smith - 555 - 765 5544. Employee ID E123461 Robert Johnson 1955 - 06 - 17 462 Main Street Springfield IL 62704 robert. johnson. 449 3367 Designer Design 2006 10 - 08 235758 USD Singapore SQL Kubernetes AWS Python Performance 2021 11 06 4. 6 expectations. 2020 02 - 22 4. 5. Health Insurance Bronze Plan 401K PTO 29 days Jane Smith 555 687 6856 leadership training 2021. answer.?', 'compressed_prompt_list': ['Employee ID E123465 Olivia Martinez Female Birth 1952 - 01 - 05 959 Main Street Springfield IL 62704. martinez. 555 - 675 - 3033 CEO 2015 09 16 53606 USD Sydney. Performance Reviews 2020 - 10 - 09. 5 time management. Benefits Health Insurance Silver Plan Retirement Plan 401K PTO 28 Emergency Michael Doe - 555 - 465 - 9759 Received Employee of Month award 2022. Employee ID E123463 Chris Lee Male 1960 - 03 - 22 523 Main Street Springfield 62704. 928 5679 DevOps Engineer Department Operations 2001 07 28 227846 USD Toronto Docker AWS 2022 12 28. 5. 10 19 3. 6 expectations last project.Benefits Health Insurance - Gold Plan Retirement Plan 401K 26 days Emergency Contact Michael Smith Relationship Parent 555 - 613 - 9745 Completed leadership training 2021. Employee ID E123464 Sophia Garcia Birth 1998 - 06 - 13 374 Main Street Springfield IL 62704. 2568 CTO Executive 2011 06 17 128311 USD Nearest Office San Francisco. Performance 2023 - 11 - 02 3. 5. 2020 11 04. 4 time management. Gold Plan 401K 29 days Emily Doe Relationship Spouse 2565 Senior Software Engineer 2020. Employee ID E123456 John Doe Birth 1990 06 - 26 650 Main Street Springfield 62704. 2182 Software Engineer 68688 USD Office Singapore.Flask Docker JavaScript Performance Reviews 2022 - 10 - 23 3. 7 performance dedication. 07 24 4. 9 time management. Health Insurance Silver Plan 401K 28 days Emergency Contact Jane Smith - 555 - 765 5544. Employee ID E123461 Robert Johnson 1955 - 06 - 17 462 Main Street Springfield IL 62704 robert. johnson. 449 3367 Designer Design 2006 10 - 08 235758 USD Singapore SQL Kubernetes AWS Python Performance 2021 11 06 4. 6 expectations. 2020 02 - 22 4. 5. Health Insurance Bronze Plan 401K PTO 29 days Jane Smith 555 687 6856 leadership training 2021. answer.?'], 'origin_tokens': 1344, 'compressed_tokens': 527, 'ratio': '2.6x', 'rate': '39.2%', 'saving': ', Saving $0.0 in GPT-4.'}
-------
Compressed Prompt:

('Answer this user query: Who is the CEO? with the following context:\n'
 "{'compressed_prompt': 'Employee ID E123465 Olivia Martinez Female Birth 1952 "
 '- 01 - 05 959 Main Street Springfield IL 62704. martinez. 555 - 675 - 3033 '
 'CEO 2015 09 16 53606 USD Sydney. Performance Reviews 2020 - 10 - 09. 5 time '
 'management. Benefits Health Insurance Silver Plan Retirement Plan 401K PTO '
 '28 Emergency Michael Doe - 555 - 465 - 9759 Received Employee of Month award '
 '2022. Employee ID E123463 Chris Lee Male 1960 - 03 - 22 523 Main Street '
 'Springfield 62704. 928 5679 DevOps Engineer Department Operations 2001 07 28 '
 '227846 USD Toronto Docker AWS 2022 12 28. 5. 10 19 3. 6 expectations last '
 'project.Benefits Health Insurance - Gold Plan Retirement Plan 401K 26 days '
 'Emergency Contact Michael Smith Relationship Parent 555 - 613 - 9745 '
 'Completed leadership training 2021. Employee ID E123464 Sophia Garcia Birth '
 '1998 - 06 - 13 374 Main Street Springfield IL 62704. 2568 CTO Executive 2011 '
 '06 17 128311 USD Nearest Office San Francisco. Performance 2023 - 11 - 02 3. '
 '5. 2020 11 04. 4 time management. Gold Plan 401K 29 days Emily Doe '
 'Relationship Spouse 2565 Senior Software Engineer 2020. Employee ID E123456 '
 'John Doe Birth 1990 06 - 26 650 Main Street Springfield 62704. 2182 Software '
 'Engineer 68688 USD Office Singapore.Flask Docker JavaScript Performance '
 'Reviews 2022 - 10 - 23 3. 7 performance dedication. 07 24 4. 9 time '
 'management. Health Insurance Silver Plan 401K 28 days Emergency Contact Jane '
 'Smith - 555 - 765 5544. Employee ID E123461 Robert Johnson 1955 - 06 - 17 '
 '462 Main Street Springfield IL 62704 robert. johnson. 449 3367 Designer '
 'Design 2006 10 - 08 235758 USD Singapore SQL Kubernetes AWS Python '
 'Performance 2021 11 06 4. 6 expectations. 2020 02 - 22 4. 5. Health '
 'Insurance Bronze Plan 401K PTO 29 days Jane Smith 555 687 6856 leadership '
 "training 2021. answer.?', 'compressed_prompt_list': ['Employee ID E123465 "
 'Olivia Martinez Female Birth 1952 - 01 - 05 959 Main Street Springfield IL '
 '62704. martinez. 555 - 675 - 3033 CEO 2015 09 16 53606 USD Sydney. '
 'Performance Reviews 2020 - 10 - 09. 5 time management. Benefits Health '
 'Insurance Silver Plan Retirement Plan 401K PTO 28 Emergency Michael Doe - '
 '555 - 465 - 9759 Received Employee of Month award 2022. Employee ID E123463 '
 'Chris Lee Male 1960 - 03 - 22 523 Main Street Springfield 62704. 928 5679 '
 'DevOps Engineer Department Operations 2001 07 28 227846 USD Toronto Docker '
 'AWS 2022 12 28. 5. 10 19 3. 6 expectations last project.Benefits Health '
 'Insurance - Gold Plan Retirement Plan 401K 26 days Emergency Contact Michael '
 'Smith Relationship Parent 555 - 613 - 9745 Completed leadership training '
 '2021. Employee ID E123464 Sophia Garcia Birth 1998 - 06 - 13 374 Main Street '
 'Springfield IL 62704. 2568 CTO Executive 2011 06 17 128311 USD Nearest '
 'Office San Francisco. Performance 2023 - 11 - 02 3. 5. 2020 11 04. 4 time '
 'management. Gold Plan 401K 29 days Emily Doe Relationship Spouse 2565 Senior '
 'Software Engineer 2020. Employee ID E123456 John Doe Birth 1990 06 - 26 650 '
 'Main Street Springfield 62704. 2182 Software Engineer 68688 USD Office '
 'Singapore.Flask Docker JavaScript Performance Reviews 2022 - 10 - 23 3. 7 '
 'performance dedication. 07 24 4. 9 time management. Health Insurance Silver '
 'Plan 401K 28 days Emergency Contact Jane Smith - 555 - 765 5544. Employee ID '
 'E123461 Robert Johnson 1955 - 06 - 17 462 Main Street Springfield IL 62704 '
 'robert. johnson. 449 3367 Designer Design 2006 10 - 08 235758 USD Singapore '
 'SQL Kubernetes AWS Python Performance 2021 11 06 4. 6 expectations. 2020 02 '
 '- 22 4. 5. Health Insurance Bronze Plan 401K PTO 29 days Jane Smith 555 687 '
 "6856 leadership training 2021. answer.?'], 'origin_tokens': 1344, "
 "'compressed_tokens': 527, 'ratio': '2.6x', 'rate': '39.2%', 'saving': ', "
 "Saving $0.0 in GPT-4.'}")
Response: The CEO of the company is Olivia Martinez.

Part 2: RAG Application: HR Use Case (POLM AI Stack)

RAG with Langchain and MongoDB

[ ]
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 974.0/974.0 kB 11.7 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 65.9 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 314.7/314.7 kB 30.6 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.2/125.2 kB 15.0 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.2/49.2 kB 5.6 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 6.8 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.7/142.7 kB 14.5 MB/s eta 0:00:00
[ ]
[ ]
[ ]
[ ]
[ ]
[ ]
Question: Who is the CEO??
Answer: Olivia Martinez is the CEO.

Prompt Compression with LangChain and LLMLingua

[ ]
[ ]
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(

config.json:   0%|          | 0.00/665 [00:00<?, ?B/s]
tokenizer_config.json:   0%|          | 0.00/26.0 [00:00<?, ?B/s]
vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]
merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]
tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]
model.safetensors:   0%|          | 0.00/548M [00:00<?, ?B/s]
generation_config.json:   0%|          | 0.00/124 [00:00<?, ?B/s]
[ ]
[Document(page_content='Olivia Martinez, Female, born on 1952-01-05. Job: CEO in Executive. Skills: Docker, React, Flask, Node.js. Reviews: Rated 4.5 on 2020-10-09: Needs improvement in time management. Rated 3.2 on 2021-01-08: Consistently meets performance standards.. Location: Works at Sydney Office, Remote: False. Notes: Received Employee of the Month award in 2022.', metadata={'_id': {'$oid': '6669c346ce0888213014cce4'}, 'employee_id': 'E123465', 'first_name': 'Olivia', 'last_name': 'Martinez', 'gender': 'Female', 'date_of_birth': '1952-01-05', 'address': {'street': '959 Main Street', 'city': 'Springfield', 'state': 'IL', 'postal_code': '62704', 'country': 'USA'}, 'contact_details': {'email': 'olivia.martinez@example.com', 'phone_number': '+1-555-675-3033'}, 'job_details': {'job_title': 'CEO', 'department': 'Executive', 'hire_date': '2015-09-16', 'employment_type': 'Full-Time', 'salary': 53606, 'currency': 'USD'}, 'work_location': {'nearest_office': 'Sydney Office', 'is_remote': False}, 'reporting_manager': None, 'skills': ['Docker', 'React', 'Flask', 'Node.js'], 'performance_reviews': [{'review_date': '2020-10-09', 'rating': 4.5, 'comments': 'Needs improvement in time management.'}, {'review_date': '2021-01-08', 'rating': 3.2, 'comments': 'Consistently meets performance standards.'}], 'benefits': {'health_insurance': 'Silver Plan', 'retirement_plan': '401K', 'paid_time_off': 28}, 'emergency_contact': {'name': 'Michael Doe', 'relationship': 'Friend', 'phone_number': '+1-555-465-9759'}, 'notes': 'Received Employee of the Month award in 2022.'}), Document(page_content='<#ref#> Johnson Female, born 1955-06-17. Job UX in Design. Skills: SQL, Kubernetes, AWS Python. Reviews Rated 4.6 on 2021-11-06: Exceeded expectations in the last project Rated 4.5 on 2020-0222: Exceeded expectations in the last.. Location: at Singapore Office, Remote: True. Notes: Completed leadership in 2021', metadata={'_id': {'$oid': '6669c346ce0888213014cce0'}, 'employee_id': 'E123461', 'first_name': 'Robert', 'last_name': 'Johnson', 'gender': 'Female', 'date_of_birth': '1955-06-17', 'address': {'street': '462 Main Street', 'city': 'Springfield', 'state': 'IL', 'postal_code': '62704', 'country': 'USA'}, 'contact_details': {'email': 'robert.johnson@example.com', 'phone_number': '+1-555-449-3367'}, 'job_details': {'job_title': 'UX Designer', 'department': 'Design', 'hire_date': '2006-10-08', 'employment_type': 'Full-Time', 'salary': 235758, 'currency': 'USD'}, 'work_location': {'nearest_office': 'Singapore Office', 'is_remote': True}, 'reporting_manager': 'M987658', 'skills': ['SQL', 'Kubernetes', 'AWS', 'Python'], 'performance_reviews': [{'review_date': '2021-11-06', 'rating': 4.6, 'comments': 'Exceeded expectations in the last project.'}, {'review_date': '2020-02-22', 'rating': 4.5, 'comments': 'Exceeded expectations in the last project.'}], 'benefits': {'health_insurance': 'Bronze Plan', 'retirement_plan': '401K', 'paid_time_off': 29}, 'emergency_contact': {'name': 'Jane Smith', 'relationship': 'Spouse', 'phone_number': '+1-555-687-6856'}, 'notes': 'Completed leadership training in 2021.'}), Document(page_content='##>, born 19980613 Executive.: JavaScriptjs Docker, AWS Reviews Rated. 2023-11-: Exceeded expectations in last project Rated. 202011-04 improvement management Location at San Office Remote: Notes:oted to Engineer <#ref#'), Document(page_content='ref> Job in. SQL AWS,. on--: Exceeded expectations in the last project.. Location: Works at Toronto Office, Remote: True. Notes: Completed leadership training in 2021.', metadata={'_id': {'$oid': '6669c346ce0888213014cce2'}, 'employee_id': 'E123463', 'first_name': 'Chris', 'last_name': 'Lee', 'gender': 'Male', 'date_of_birth': '1960-03-22', 'address': {'street': '523 Main Street', 'city': 'Springfield', 'state': 'IL', 'postal_code': '62704', 'country': 'USA'}, 'contact_details': {'email': 'chris.lee@example.com', 'phone_number': '+1-555-928-5679'}, 'job_details': {'job_title': 'DevOps Engineer', 'department': 'Operations', 'hire_date': '2001-07-28', 'employment_type': 'Full-Time', 'salary': 227846, 'currency': 'USD'}, 'work_location': {'nearest_office': 'Toronto Office', 'is_remote': True}, 'reporting_manager': 'M987660', 'skills': ['Docker', 'SQL', 'AWS', 'Kubernetes'], 'performance_reviews': [{'review_date': '2022-12-28', 'rating': 4.5, 'comments': 'Consistently meets performance standards.'}, {'review_date': '2020-10-19', 'rating': 3.6, 'comments': 'Exceeded expectations in the last project.'}], 'benefits': {'health_insurance': 'Gold Plan', 'retirement_plan': '401K', 'paid_time_off': 26}, 'emergency_contact': {'name': 'Michael Smith', 'relationship': 'Parent', 'phone_number': '+1-555-613-9745'}, 'notes': 'Completed leadership training in 2021.'})]
[ ]
[ ]
{'query': 'Who is the CEO?', 'result': 'Olivia Martinez is the CEO.'}

RAG with LlamaIndex and MongoDB (Coming Soon)

RAG with HayStack and MongoDB (Coming Soon)

Part 3: AI Agent Application: HR Use Case (POLM AI Stack)

AI Agents with langChain and MongoDB (Coming Soon)

AI Agents with LlamaIndex and MongoDB (Coming Soon)

AI Agents with HayStack and MongoDB (Coming Soon)

[ ]