Building a custom component for RAG pipelines with Haystack

by Tuana Celik: Twitter, LinkedIn

📚 Check out the Customizing RAG Pipelines to Summarize Latest Hacker News Posts with Haystack article for a detailed run through of this example.

Install dependencies

Create a Custom Haystack Component

This HackernewsNewestFetcher ferches the last_k newest posts on Hacker News and returns the contents as a List of Haystack Document objects

Create a Haystack 2.0 RAG Pipeline

This pipeline uses the components available in the Haystack 2.0 preview package at time of writing (22 September 2023) as well as the custom component we've created above.

The end result is a RAG pipeline designed to provide a list of summaries for each of the last_k posts on Hacker News, followes by the source URL.