RAG From Scratch
agentsllmsvector-databaselancedbgptopenaiAImultimodal-aitutorialsRAG-from-Scratchmachine-learningembeddingsfine-tuningdeep-learninggpt-4-visionllama-indexragmultimodallangchainlancedb-recipes
Export
Retrieval-Augmented Generation(RAG) from Scratch with FTS, Vector Search and Hybrid Search
In this notebook, we will build a Retrieval-Augmented Generation(RAG) pipeline from scratch without using any popular libraries such as Langchain or Llamaindex.
RAG is a technique that retrieves related documents to the user's question, combines them with LLM-base prompt, and sends them to LLMs like GPT to produce more factually accurate generation.
Lets Split RAG Pipeline into 5 parts:
- Data loading
- Chunking and Embedding
- Ingest data in Vector Store
- FTS, Vector Search and Hybrid search Retrieval & Prompt preparation
- Answer Generation using results of FTS, Vector Search and Hybrid search all of them
Here is an image illustrating the RAG process
Install Dependencies
[17]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.3/37.3 MB 23.0 MB/s eta 0:00:00
Set OPENAI API KEY as env variable
[40]
Data Loading
[3]
--2025-04-18 09:46:48-- https://raw.githubusercontent.com/lancedb/vectordb-recipes/main/tutorials/RAG-from-Scratch/lease.txt Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 5248 (5.1K) [text/plain] Saving to: ‘lease.txt’ lease.txt 100%[===================>] 5.12K --.-KB/s in 0s 2025-04-18 09:46:48 (50.6 MB/s) - ‘lease.txt’ saved [5248/5248]
Data Chunking and Embedding
[4]
[nltk_data] Downloading package punkt to /root/nltk_data... [nltk_data] Unzipping tokenizers/punkt.zip.
[5]
Number of Chunks: 11
Vector Store
[21]
Retriever & Prompt preparation
Query Type: FTS
[22]
[23]
[' as defined in Section 2 of this Article II, and shall terminate on May 31, 2020 ("the Termination Date"); provided, however, that at the option of Tenant, Tenant may renew this Lease for five additional successive one- year terms at a Monthly Rent of $100,000 per month, provided that notice of such renewal is given in writing no less than 120 days prior to the Termination Date or the expiration of any one-year renewal term. Tenant may at any time cancel this Lease and terminate all of its obligations hereunder by the payment of $300,000, plus all other amounts then due',
, ' 2 elmonteleaseforfiling.htm MATERIAL CONTRACT\nCOMMERCIAL LEASE AGREEMENT\n\n\n\nTHIS LEASE AGREEMENT is made and entered into on December 1, 2013, by and between Temple CB, LLC, whose address is 4350 Temple City Boulevard, El Monte, California 91731 (hereinafter referred to as "Landlord"), and Okra Energy, Inc., whose address is 4350 Temple City Boulevard, El Monte, California 91731 (hereinafter referred to as "Tenant").\n\n\n\nARTICLE I - GRANT OF LEASE\n\n\n\nLandlord, in consideration of the rents to be paid and the covenants',
, ' consideration of the rents to be paid and the covenants and agreements to be performed and observed by the Tenant, does hereby lease to the Tenant and the Tenant does hereby lease and take from the Landlord the property described in Exhibit "A" attached hereto and by reference made a part hereof (the "Leased Premises"), together with, as part of the parcel, all improvements located thereon.\n\n\n\nARTICLE II - LEASE TERM\n\n\n\nSection l. Term of Lease. The term of this Lease shall begin on the Commencement Date, as defined in Section 2 of this Article II,',
, ' of $300,000, plus all other amounts then due under this Lease.\n\n\n\nSection 2. Commencement Date. The "Commencement Date" shall mean December 1, 2013.\n\n\n\nARTICLE III - EXTENSIONS\n\n\n\nThe parties hereto may elect to extend this Agreement upon such terms and conditions as may be agreed upon in writing and signed by the parties at the time of any such extension.\n\n\n\nARTICLE IV - DETERMINATION OF RENT\n\n\n\nSection 1. Monthly Rent: The Tenant agrees to pay the Landlord and the Landlord agrees to accept, during the term hereof, at',
, ' released from all liability for the return of such security to the Tenant.\n\n\n\nARTICLE VI - TAXES\n\n\n\nSection l. Personal Property Taxes. The Tenant shall be liable for all taxes levied against any leasehold interest of the Tenant or personal property and trade fixtures owned or placed by the Tenant in the Leased Premises.\n\n\n\nSection 2. Real Estate Taxes. During the continuance of this lease Landlord shall deliver to Tenant a copy of any real estate taxes and assessments against the Leased Property. From and after the Commencement Date, the Tenant shall pay to Landlord not'] Query Type: Vector Search
[49]
[50]
[' as defined in Section 2 of this Article II, and shall terminate on May 31, 2020 ("the Termination Date"); provided, however, that at the option of Tenant, Tenant may renew this Lease for five additional successive one- year terms at a Monthly Rent of $100,000 per month, provided that notice of such renewal is given in writing no less than 120 days prior to the Termination Date or the expiration of any one-year renewal term. Tenant may at any time cancel this Lease and terminate all of its obligations hereunder by the payment of $300,000, plus all other amounts then due',
, ' of $300,000, plus all other amounts then due under this Lease.\n\n\n\nSection 2. Commencement Date. The "Commencement Date" shall mean December 1, 2013.\n\n\n\nARTICLE III - EXTENSIONS\n\n\n\nThe parties hereto may elect to extend this Agreement upon such terms and conditions as may be agreed upon in writing and signed by the parties at the time of any such extension.\n\n\n\nARTICLE IV - DETERMINATION OF RENT\n\n\n\nSection 1. Monthly Rent: The Tenant agrees to pay the Landlord and the Landlord agrees to accept, during the term hereof, at',
, ' consideration of the rents to be paid and the covenants and agreements to be performed and observed by the Tenant, does hereby lease to the Tenant and the Tenant does hereby lease and take from the Landlord the property described in Exhibit "A" attached hereto and by reference made a part hereof (the "Leased Premises"), together with, as part of the parcel, all improvements located thereon.\n\n\n\nARTICLE II - LEASE TERM\n\n\n\nSection l. Term of Lease. The term of this Lease shall begin on the Commencement Date, as defined in Section 2 of this Article II,',
, " Commencement Date, the Tenant shall pay to Landlord not later than twenty-one (21) days after the day on which the same may become initially due, all real estate taxes and assessments applicable to the Leased Premises, together with any interest and penalties lawfully imposed thereon as a result of Tenant's late payment thereof, which shall be levied upon the Leased Premises during the term of this Lease.\n\n\n\nSection 3. Contest of Taxes. The Tenant, at its own cost and expense, may, if it shall in good faith so desire, contest by appropriate proceedings the amount of",
, ' agrees to accept, during the term hereof, at such place as the Landlord shall from time to time direct by notice to the Tenant, monthly rent of $40,000.\n\n\nSection 2. Late Fee. A late fee in the amount of 5% of the Monthly Rent shall be assessed if payment is not postmarked or received by Landlord on or before the tenth day of each month.\n\n\n\nARTICLE V - SECURITY DEPOSIT\n\n\n\nThe Tenant has deposited with the Landlord the sum of Twenty Thousand Dollars ($20,000.00) as security for the full and faithful performance by the Tenant',
, ' 2 elmonteleaseforfiling.htm MATERIAL CONTRACT\nCOMMERCIAL LEASE AGREEMENT\n\n\n\nTHIS LEASE AGREEMENT is made and entered into on December 1, 2013, by and between Temple CB, LLC, whose address is 4350 Temple City Boulevard, El Monte, California 91731 (hereinafter referred to as "Landlord"), and Okra Energy, Inc., whose address is 4350 Temple City Boulevard, El Monte, California 91731 (hereinafter referred to as "Tenant").\n\n\n\nARTICLE I - GRANT OF LEASE\n\n\n\nLandlord, in consideration of the rents to be paid and the covenants',
, ' proceedings, the Tenant shall have the right to contest the amount of any such tax and the Tenant shall have the right to withhold payment of any such tax, if the statute under which the Tenant is contesting such tax so permits.\n\n\n\nSection 4. Payment of Ordinary Assessments. The Tenant shall pay all assessments, ordinary and extraordinary, attributable to or against the Leased Premises not later than twenty-one (21) days after the day on which the same became initially due. The Tenant may take the benefit of any law allowing assessments to be paid in installments and in such event the',
, ' released from all liability for the return of such security to the Tenant.\n\n\n\nARTICLE VI - TAXES\n\n\n\nSection l. Personal Property Taxes. The Tenant shall be liable for all taxes levied against any leasehold interest of the Tenant or personal property and trade fixtures owned or placed by the Tenant in the Leased Premises.\n\n\n\nSection 2. Real Estate Taxes. During the continuance of this lease Landlord shall deliver to Tenant a copy of any real estate taxes and assessments against the Leased Property. From and after the Commencement Date, the Tenant shall pay to Landlord not',
, " so desire, contest by appropriate proceedings the amount of any personal or real property tax. The Tenant may, if it shall so desire, endeavor at any time or times, by appropriate proceedings, to obtain a reduction in the assessed valuation of the Leased Premises for tax purposes. In any such event, if the Landlord agrees, at the request of the Tenant, to join with the Tenant at Tenant's expense in said proceedings and the Landlord agrees to sign and deliver such papers and instruments as may be necessary to prosecute such proceedings, the Tenant shall have the right to contest",
, ' security for the full and faithful performance by the Tenant of all the terms of this lease required to be performed by the Tenant. Such sum shall be returned to the Tenant after the expiration of this lease, provided the Tenant has fully and faithfully carried out all of its terms. In the event of a bona fide sale of the property of which the leased premises are a part, the Landlord shall have the right to transfer the security to the purchaser to be held under the terms of this lease, and the Landlord shall be released from all liability for the return of such security'] Perform inbuilt Hybrid Search
They have some off the shelf functionalities and a way to implement the custom Re-Ranking and Filtering Function here Implement Custom Rerankers
[26]
[' released from all liability for the return of such security to the Tenant.\n\n\n\nARTICLE VI - TAXES\n\n\n\nSection l. Personal Property Taxes. The Tenant shall be liable for all taxes levied against any leasehold interest of the Tenant or personal property and trade fixtures owned or placed by the Tenant in the Leased Premises.\n\n\n\nSection 2. Real Estate Taxes. During the continuance of this lease Landlord shall deliver to Tenant a copy of any real estate taxes and assessments against the Leased Property. From and after the Commencement Date, the Tenant shall pay to Landlord not', , ' consideration of the rents to be paid and the covenants and agreements to be performed and observed by the Tenant, does hereby lease to the Tenant and the Tenant does hereby lease and take from the Landlord the property described in Exhibit "A" attached hereto and by reference made a part hereof (the "Leased Premises"), together with, as part of the parcel, all improvements located thereon.\n\n\n\nARTICLE II - LEASE TERM\n\n\n\nSection l. Term of Lease. The term of this Lease shall begin on the Commencement Date, as defined in Section 2 of this Article II,', , ' 2 elmonteleaseforfiling.htm MATERIAL CONTRACT\nCOMMERCIAL LEASE AGREEMENT\n\n\n\nTHIS LEASE AGREEMENT is made and entered into on December 1, 2013, by and between Temple CB, LLC, whose address is 4350 Temple City Boulevard, El Monte, California 91731 (hereinafter referred to as "Landlord"), and Okra Energy, Inc., whose address is 4350 Temple City Boulevard, El Monte, California 91731 (hereinafter referred to as "Tenant").\n\n\n\nARTICLE I - GRANT OF LEASE\n\n\n\nLandlord, in consideration of the rents to be paid and the covenants', , ' agrees to accept, during the term hereof, at such place as the Landlord shall from time to time direct by notice to the Tenant, monthly rent of $40,000.\n\n\nSection 2. Late Fee. A late fee in the amount of 5% of the Monthly Rent shall be assessed if payment is not postmarked or received by Landlord on or before the tenth day of each month.\n\n\n\nARTICLE V - SECURITY DEPOSIT\n\n\n\nThe Tenant has deposited with the Landlord the sum of Twenty Thousand Dollars ($20,000.00) as security for the full and faithful performance by the Tenant', , " Commencement Date, the Tenant shall pay to Landlord not later than twenty-one (21) days after the day on which the same may become initially due, all real estate taxes and assessments applicable to the Leased Premises, together with any interest and penalties lawfully imposed thereon as a result of Tenant's late payment thereof, which shall be levied upon the Leased Premises during the term of this Lease.\n\n\n\nSection 3. Contest of Taxes. The Tenant, at its own cost and expense, may, if it shall in good faith so desire, contest by appropriate proceedings the amount of"]
[27]
Answer Generation
Using FTS result
[46]
The issue date of the lease is December 1, 2013 [2].
Using Vector Search result
[51]
The issue date of the lease is December 1, 2013 [6].
Using Hybrid Search result
[48]
The issue date of the lease is December 1, 2013 [3].