Video 3 Langchain Copilot With NIM HF FAISS Deployed Locally

Creating a RAG application with LLM, Embedding model, and Vector DataBase hosted locally

This notebook showcases how to create a Retrieval Augmented Generation (RAG) application where the LLM model, embedding model, and Vector DataBase (VDB) are deployed locally.

We will be using NVIDIA NIM microservices to locally host Llama3-8B-instruct model. The microservice will be connected using LangChain NVIDIA AI Endpoints package.

For creating embeddings from your proprietary documents, we will be using embedding model hosted on huggingface which will again be connected using LangChain's huggingface plugin.

Lastly for storing the embeddings, we will be using the Facebook AI Similarity Search (FAISS) plugin available in LangChain.

This notebook is divided into two parts:

1. In the first part we will showcase how to create embeddings from your documents and store them in a VDB.
2. In the second part we will orchestrate the RAG application using Langchain framework and create a Gradio-based simple UI to interact with this application.

API Key generation:

Before we get started, generate the API keys to use model from NVIDIA NIM microservice and download the embedding model from HuggingFace.

To generate 'NVIDIA_API_KEY' for NVIDIA NIM microserice:

1. Create a free account with NVIDIA.
2. Click on your model of choice.
3. Under Input select the Python tab, and click Get API Key and then click Generate Key.
4. Copy and save the generated key as NVIDIA_API_KEY. From there, you should have access to the endpoints.

To generate 'ACCESS_TOKENS' for Huggingface:

1. Log in to Hugging Face Hub
2. Click your profile icon in the top right corner
3. Click Settings from the drop-down list
4. Click Access Tokens in the left-hand navigation panel
5. Click New token
6. Enter a name for your token and select a role
7. Click Generate token
8. Click Copy to copy the token to your clipboard

Getting Started!

Install all the prerequisite libraries to orchestrate the chat application with LangChain

Create Vector Data Base (VDB) using your Proprietary Documents (PART 1)

In this section, we will create vector embeddings from your documents and store them in a vector database. We will be following the below-listed steps

1. Download the embedding model from Huggingface using LangChain plugin
2. Parse all the PDF documents in a folder and break them into text chunks.
3. Pass the chunks to the embedding model to create embeddings.
4. Save the generated embeddings into a FAISS vector database.

Import all the required libraries

Embedding model and global variables

We provide the name of the embedding model and the location of where the VDB needs to be stored. We also initialize few global variables

Download embedding model

In this function, we download the embedding model from Huggingface using the Huggingface plugin in LangChain. In this function we also check for the availability of GPU/s and pass that as an argument in the encode and model arguments.

Document processing

In this function, we load the document and use the text splitter to split the document into chunks. We can use the chunk_size and chunk_overlap as parameters to specify the number of tokens per chunk and how much overlap we need between each chunk.

Loading and processing

In this function, we parse through the directory that has all the pdf documents, breaks them into chunks using the process_pdf function. We also call the download_embedding_model function to download the embedding model.

The document chunks and embedding model are passed to the FAISS plugin to create a Vector Database Base (VDB).

Load Vector DataBase (VDB)

In this function we load the VDB and pass it to the GPU.

Test Vector DataBase (VDB)

We use this function to test the embeddings generated and stored in the VDB.

Create Interactive Chat application (PART 2)

In this section, we will orchestrate the chat application using LangChain.

1. Before we move ahead, deploy Llama3-8b-instruct model using the [NVIDIA NIM] (https://build.nvidia.com/meta/llama3-70b). Select the docker tab and follow the instructions to deploy it locally.
2. We will connect with the VDB to retrieve the relevant document chunks based on the user query
3. Using the prompt, retrieved document chunk, the LLMs model will generate a response for the user
4. Create a simple Gradio-based UI to interact with this chat application

Import all the required libraries

Prompt template

In this fuction we use one of the default prompt template provided by langchain and use it for context and question asked by the user.

Response to user query

In this function, we connect the LLM model using the ChatNVIDIA plugin. We also connect the VDB using the retriever object. The LLM model, VDB retriever, and prompt template are passed to generate response to the user query.

UI to interact with chat application

In this block, we create a simple UI using Gradio to interact with the application. The select folder link provides functionality to select the folder where your documents are located. the load document loads the document files in the selected folder and generates embeddings for those documents.

Once processed, you can ask questions about your document in the Enter your question tab. The generated response will be displayed in the window while maintaining the history of past queries.