Advance Prompt Customization for Anthropic

Notebook by Bilge Yucel (LI & X (Twitter))

In this example, we'll create a RAG application using prompting techniques in Anthropic's Prompt Engineering Guide. This application will use Anthropic Claude 3 models and Haystack to extract relevant quotes from given documents and generate an answer based on extracted quotes.

📚 Useful Sources:

• Docs: AnthropicChatGenerator
• Integration: Anthropic

Setup the Development Environment

Install antropic-haystack package and other required packages with pip:

You need an ANTHROPIC_API_KEY to work with Claude models. Get your API key here

Download the Dataset

We'll use the bilgeyucel/seven-wonders dataset on Hugging Face

Create Embeddings and Index into the DocumentStore

Build the Prompt

Claude models are familiar with prompts that contain XML tags, as they were exposed to such prompts during training. By wrapping key parts of the prompt (such as instructions, examples, or input data) in XML tags, we can help Claude better understand the context and generate more accurate outputs.

To formulate a prompt that first extracts quotes from relevant documents and then refers to these quotes to generate the answer, follow these steps in your prompt:

1. Place retrieved documents between <documents> </documents> tags.
2. Render each document between <document> </document> tags, including a document index for Claude to reference later.
3. Instruct Claude to extract quotes within <quotes> </quotes> tags, including document index information.
4. Ensure that Claude generates the answer between <answer> </answer> tags.

The rendered prompt should look like this when it's fed with relevant documents and the question:

	Here is a document, in <document></document> XML tags:

<documents>
<document index="1">
  (the text content of the first document)
</document>
<document index="2">
  (the text content of the second document)
</document>
<document index="3">
  (the text content of the third document)
</document>
...
</documents>

First, extract, word-for-word, any quotes relevant to the question, enclose the full list of quotes in <quotes></quotes> XML tags with the corresponding document index and use these quotes to an answer to the question.

If there are no quotes in this document that seem relevant to this question, please say "I can't find any relevant quotes".

Then, answer the question in <answer></answer> tags. Do not include or reference quoted content verbatim in the answer. Ensure that your answer is accurate and doesn't contain any information not directly supported by the quotes. Make references to quotes relevant to each section of the answer solely by adding their bracketed numbers at the end of relevant sentences

Here is the question: (user question)

And in return, Claude's response should look like this:

	<quotes>
<quote index="1">Large numbers of people came to Ephesus in March and in the beginning of May to attend the main Artemis Procession.</quote>
<quote index="2">The Temple of Artemis or Artemision (Greek: Ἀρτεμίσιον; Turkish: Artemis Tapınağı), also known as the Temple of Diana, was a Greek temple dedicated to an ancient, local form of the goddess Artemis (identified with Diana, a Roman goddess).</quote>
</quotes>

<answer>
According to the documents, people visited the Temple of Artemis in Ephesus primarily for religious reasons to attend festivals and processions dedicated to the goddess Artemis (also known as Diana). The temple was dedicated to an ancient local form of the goddess and held a main Artemis Procession in March and early May that drew large crowds. [1] The temple was considered an important religious site for the worship of Artemis in the Greek world. [2] People likely traveled there to make offerings, participate in rituals, and celebrate the goddess during major festivals and ceremonies held at the sacred site.
</answer>

Initialize RAG Pipeline Components

Initialize components required for a RAG pipeline:

• SentenceTransformersTextEmbedder: to create embeddings using for the query using sentence-transformers models
• InMemoryEmbeddingRetriever: to retrieve relevant documents
• ChatPromptBuilder: to construct prompts by processing ChatMessage objects
• AnthropicChatGenerator: to use chat completion API of Anthropic Claude models, we'll use claude-3-sonnet-20240229 for this example

Build the RAG Pipeline

Learn how to build a pipeline in Creating Pipelines.

Test Different Questions

Extract the Quotes

Extract the Answer