Question Answering with Amazon Sagemaker, Chroma and Haystack

Notebook by Sara Zanzottera and Bilge Yucel

Amazon Sagemaker is a comprehensive, fully managed machine learning service that allows data scientists and developers to build, train, and deploy ML models efficiently. You can choose from various foundation models to find the one best suited for your use case.

In this notebook, we'll go through the process of creating a generative question answering application using the newly added Amazon Sagemaker integration with Haystack and Chroma to store our documents efficiently. The demo will illustrate the step-by-step development of a QA application using some Wikipedia pages about NASA's Mars missions 🚀

Setup the Development Environment

Install dependencies

Deploy a model on Sagemaker

To use Amazon Sagemaker's models, you first need to deploy them. In this example we'll be using Falcon 7B Instruct BF16, so make sure to deploy such model on your account before proceeding.

For help you can check out:

• Amazon Sagemaker Jumpstart documentation.
• This notebook on how to deploy Falcon models programmatically with a notebook
• This blogpost about deploying models on Sagemaker for Haystack 1.x

API Keys

To use Amazon Sagemaker, you need to set a few environment variables: AWS ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and often to indicate the region by setting AWS_REGION. Once logged into your account, locate these keys under the IAM user's "Security Credentials" section. For detailed guidance, refer to the documentation on Managing access keys for IAM users.

Load data from Wikipedia

We are going to download the Wikipedia pages related to NASA's martian rovers using the python library wikipedia.

These pages are converted into Haystack Documents.

Building the Indexing Pipeline

Our indexing pipeline will preprocess the provided Wikipedia pages by cleaning and splitting it into chunks before storing them in ChromaDocumentStore.

Let’s run the pipeline below and index our file to our document store:

Run the pipeline with the files you want to index (note that this step may take some time):

Building the Query Pipeline

Let’s create another pipeline to query our application. In this pipeline, we’ll use ChromaQueryTextRetriever to retrieve relevant information from the ChromaDocumentStore and a Falcon 7B Instruct BF16 model to generate answers with SagemakerGenerator.

Next, we'll create a prompt for our task using the Retrieval-Augmented Generation (RAG) approach with PromptBuilder. This prompt will help generate answers by considering the provided context. Finally, we'll connect these three components to complete the pipeline.

Ask your question and learn about the Amazon Sagemaker service using Amazon Sagemaker models!