Notebooks
C
Cohere
RAG With Connectors

RAG With Connectors

llmunotebookscohere-developer-experience

RAG with Connectors

This notebook shows how to build a RAG-powered chatbot with Cohere's Chat endpoint using connectors. The chatbot can extract relevant information from external documents and produce verifiable, inline citations in its responses.

Read the accompanying article here.

Connectors are ways of connecting to data sources. These data sources could be internal documents, document databases, the broader internet, or any other source of context which can inform the replies generated by the model.

We'll use the web search connector, a Cohere-managed connector that you can use without additional setup.

The diagram below provides an overview of what we’ll build.

Workflow

Setup

[ ]
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.2/117.2 kB 1.9 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.6/75.6 kB 3.7 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.8/77.8 kB 4.6 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 4.8 MB/s eta 0:00:00
[1]
[3]

Create a chatbot

In connector mode, most of the implementation is taken care of by the endpoint, including deciding whether to retrieve information, generating queries, retrieving documents, chunking and reranking documents (post-retrieval), and generating the response. This greatly simplifies our code.

The Chatbot class below handles the interaction between the user and chatbot. We define the connector for the chatbot to use with the attribute self.connectors. In this notebook, we will use Cohere's “web-search” connector, which runs searches against a browser in safe mode.

The run() method contains the logic for getting the user message, displaying the chatbot response with citations, along with a way for the user to end the conversation.

Then, the chatbot responds to the user message. We call co.chat() and supply a connectors parameter to make the chatbot component use connector mode. All of the remaining implementation is taken care of by the endpoint, up to generating the response.

We also pass the conversation_id parameter, which retains the interactions between the user and the chatbot in the same conversation thread. We enable the stream parameter so we can stream the chatbot response.

We then print the chatbot's response. In the case that the external information was used to generate a response, we also display documents and in-line citations.

[7]

Run the chatbot

We can now run the chatbot. For this, we create the instance of Chatbot using Cohere's managed web-search connector. Then we run the chatbot by invoking the run() method.

The format of each citation is:

  • start: The starting point of a span where one or more documents are referenced
  • end: The ending point of a span where one or more documents are referenced
  • text: The text representing this span
  • document_ids: The IDs of the documents being referenced (doc_0 being the ID of the first document passed to the documents creating parameter in the endpoint call, and so on)
[8]
User: What is Cohere's LLM University

Chatbot:
Cohere's LLM University (LLMU) is a set of comprehensive learning resources for anyone interested in natural language processing (NLP), from beginners to advanced learners. The curriculum covers everything from the basics of LLMs to the most advanced topics, including generative AI. The course is designed to give learners a solid foundation in NLP and help them develop their own applications.

CITATIONS:
start=24 end=30 text='(LLMU)' document_ids=['web-search_0', 'web-search_1']
start=36 end=75 text='set of comprehensive learning resources' document_ids=['web-search_1']
start=101 end=134 text='natural language processing (NLP)' document_ids=['web-search_0', 'web-search_1']
start=141 end=172 text='beginners to advanced learners.' document_ids=['web-search_0', 'web-search_1']
start=177 end=187 text='curriculum' document_ids=['web-search_0', 'web-search_1']
start=215 end=229 text='basics of LLMs' document_ids=['web-search_0', 'web-search_1']
start=237 end=283 text='most advanced topics, including generative AI.' document_ids=['web-search_1']
start=326 end=349 text='solid foundation in NLP' document_ids=['web-search_0', 'web-search_1']
start=364 end=395 text='develop their own applications.' document_ids=['web-search_0', 'web-search_1']

DOCUMENTS:
{'id': 'web-search_0', 'snippet': 'Guides and ConceptsAPI ReferenceRelease NotesApplication ExamplesLLMU\n\nCoralDashboardDocumentationPlaygroundCommunityLog In\n\nCoralDashboardDocumentationPlaygroundCommunityLog In\n\nWelcome to LLM University!\n\nWelcome to LLM University by Cohere!\n\nWe’re so happy that you’ve chosen to learn Natural Language Processing and Large Language Models with us.\n\nOur comprehensive curriculum aims to give you a ...', 'title': 'LLM University (LLMU) | Cohere', 'url': 'https://docs.cohere.com/docs/llmu'}
{'id': 'web-search_1', 'snippet': "Introducing LLM University — Your Go-To Learning Resource for NLP🎓\n\nDiscover our comprehensive NLP curriculum at LLM University. From the fundamentals of LLMs all the way to the most advanced topics, including generative AI\n\nWe're excited to announce the launch of LLM University (LLMU), a set of comprehensive learning resources for anyone interested in natural language processing (NLP), from begin...", 'title': 'Introducing LLM University — Your Go-To Learning Resource for NLP🎓', 'url': 'https://txt.cohere.com/llm-university/'}

----------------------------------------------------------------------------------------------------

Ending chat.