Use Ragas to evaluate RAG pipeline

Ragas is an open source project for evaluating RAG components. Paper, Code, Docs, Intro blog.

Please note that RAGAS can use a large amount of OpenAI api token consumption.

Read through this notebook carefully and pay attention to the number of questions and metrics you want to evaluate.

1. Prepare Ragas environment and ground truth data

2. Start Ragas Evaluation with custom Evaluation LLM

The default OpenAI model used by Ragas is gpt-3.5-turbo-16k.

Note that a large amount of OpenAI api token is consumed. Every time you ask a question and every evaluation, you will ask the OpenAI service. Please pay attention to your token consumption.