Contextual Chunk Embedding
advanced_techniquesagentsartificial-intelligencellmsmongodb-genai-showcasenotebooksgenerative-airag
Export
Contextualized chunk embeddings: Combining local detail with global context
This notebook shows you how to implement and evaluate Voyage AI's voyage-context-3 contextualized chunk embedding model.
Step 1: Install required libraries
- datasets: Python library to get access to datasets available on Hugging Face Hub
- pdfplumber: Python library to parse and analyze PDFs
- langchain-text-splitters: Text chunking utilities in LangChain
- tiktoken: Token counting and encoding library
- voyageai: Python library to interact with Voyage AI's APIs
[ ]
Step 2: Setup prerequisites
Follow the steps here to obtain a Voyage AI API key.
[29]
[ ]
Step 3: Download the dataset
[31]
[32]
[33]
40
[34]
'Exhibit 10.2\nExecution Version\nINTELLECTUAL PROPERTY AGREEMENT\nThis INTELLECTUAL PROPERTY AGREEMENT (this “Agreement”), dated as of December 31, 2018 (the “Effective Date”) is entered into by and\nbetween Armstrong Flooring, Inc., a Delaware corporation (“Seller”) and AFI Licensing LLC, a Delaware limited liability company (“Licensing” and\ntogether with Seller, “Arizona”) and AHF Holding, Inc. (formerly known as Tarzan HoldCo, Inc.), a Delaware corporation (“Buyer”) and Armstrong\nHardwood Flooring Company, a Tennessee corporation (the “Company” and together with Buyer the “Buyer Entities”) (each of Arizona on the one hand\nand the Buyer Entities on the other hand, a “Party” and collectively, the “Parties”).\nWHEREAS, Seller and Buyer have entered into that certain Stock Purchase Agreement, dated November 14, 2018 (the “Stock Purchase\nAgreement”); WHEREAS, pursuant to the Stock Purchase Agreement, Seller has agreed to sell and transfer, and Buyer has agreed to purchase and\nacquire, all of Seller’s right, title and interest in and to Armstrong Wood Products, Inc., a Delaware corporation (“AWP”) and its Subsidiaries, the\nCompany and HomerWood Hardwood Flooring Company, a Delaware corporation (“HHFC,” and together with the Company, the “Company\nSubsidiaries” and together with AWP, the “Company Entities” and each a “Company Entity”) by way of a purchase by Buyer and sale by Seller of the\nShares, all upon the terms and condition set forth therein;\nWHEREAS, Arizona owns certain Copyrights, Know-How, Patents and Trademarks which may be used in the Company Field, and in connection\nwith the transactions contemplated by the Stock Purchase Agreement the Company desires to acquire all of Arizona’s right, title and interest in and to\nsuch Intellectual Property used exclusively in the Company Field, and obtain a license from Arizona to use other such Intellectual Property on the terms\nand subject to the conditions set forth herein;\nWHEREAS, Seller is signatory to the Trademark License Agreement pursuant to which Seller obtains a license to the Arizona Licensed\nTrademarks;\nWHEREAS, the Company desires to obtain a sublicense to use the Arizona Licensed Trademarks in the Company Field;\nWHEREAS, Arizona has obtained consent from all counterparties to the Trademark License Agreement to grant to the Company the sublicenses\nto the Arizona Licensed Trademarks included in this Agreement; and\nWHEREAS, the Company Entities own certain Copyrights and Know-How which may be used in the Arizona Field, and in connection with the\ntransactions contemplated by the Stock Purchase Agreement, Arizona desires to obtain a license from the Company Entities to use such Intellectual\nProperty on the terms and subject to the conditions set forth herein.\nNOW, THEREFORE, in consideration of the foregoing and the mutual agreements, provisions and covenants contained in this Agreement, and\nfor other good and valuable consideration, the receipt and sufficiency of which are hereby acknowledged, the Parties hereby agree as follows:\nSource: ARMSTRONG FLOORING, INC., 8-K, 1/7/2019'
Step 4: Chunk the PDF content
[35]
[36]
[37]
[38]
{'chunk': 'Exhibit 10.2\nExecution Version\nINTELLECTUAL PROPERTY AGREEMENT\nThis INTELLECTUAL PROPERTY AGREEMENT (this “Agreement”), dated as of December 31, 2018 (the “Effective Date”) is entered into by and\nbetween Armstrong Flooring, Inc., a Delaware corporation (“Seller”) and AFI Licensing LLC, a Delaware limited liability company (“Licensing” and\ntogether with Seller, “Arizona”) and AHF Holding, Inc. (formerly known as Tarzan HoldCo, Inc.), a Delaware corporation (“Buyer”) and Armstrong\nHardwood Flooring Company, a Tennessee corporation (the “Company” and together with Buyer the “Buyer Entities”) (each of Arizona on the one hand\nand the Buyer Entities on the other hand, a “Party” and collectively, the “Parties”).',
, 'chunk_id': 0,
, 'doc_id': 0} Step 5: Embed the chunks
[39]
[40]
[41]
Step 6: Evaluation
[42]
[43]
[44]
Standard Embeddings
[45]
[46]
Which state’s law governs the agreement between Armstrong Flooring and AHF Holding?: 1 In the Armstrong-AHF agreement, how many days' notice is required to remedy a breach?: None Where will disputes be resolved under the agreement between Armstrong Flooring and AHF Holding?: None What happens if either party materially breaches the Armstrong-AHF Intellectual agreement?: 1 Under the Armstrong Flooring-AHF Holding agreement, what is the minimum logo size?: 2 When does Playa Hotels & Resorts' right of first offer expire?: 5 Which state’s law governs the agreement between Hyatt Franchising Latin America and Playa Hotels & Resorts B.V.?: 3 What countries can Hyatt Franchising Latin America and Playa develop Hyatt All-Inclusive Resorts in?: 2 How many years of hotel experience must arbitrators have under the Hyatt-Playa agreement?: 1 Where will arbitration take place for disputes under the Hyatt-Playa agreement?: 2 When was the Quaker/Gulf Houghton agreement effective?: 1 Which state’s law governs the Quaker/Gulf Houghton agreement?: 2 What is the geographic scope of the Quaker/Gulf Houghton agreement?: None What percentage of publicly traded securities can Gulf Houghton sellers own as passive investors?: 1 How long must before Gulf Houghton sellers can hire former employees?: 2 Mean recall: 80.00% Mean reciprocal rank: 53.56%
Contextualized Embeddings
[47]
[48]
[49]
[50]
Which state’s law governs the agreement between Armstrong Flooring and AHF Holding?: 1 In the Armstrong-AHF agreement, how many days' notice is required to remedy a breach?: None Where will disputes be resolved under the agreement between Armstrong Flooring and AHF Holding?: 2 What happens if either party materially breaches the Armstrong-AHF Intellectual agreement?: 1 Under the Armstrong Flooring-AHF Holding agreement, what is the minimum logo size?: 2 When does Playa Hotels & Resorts' right of first offer expire?: 5 Which state’s law governs the agreement between Hyatt Franchising Latin America and Playa Hotels & Resorts B.V.?: 1 What countries can Hyatt Franchising Latin America and Playa develop Hyatt All-Inclusive Resorts in?: 3 How many years of hotel experience must arbitrators have under the Hyatt-Playa agreement?: 1 Where will arbitration take place for disputes under the Hyatt-Playa agreement?: 1 When was the Quaker/Gulf Houghton agreement effective?: 1 Which state’s law governs the Quaker/Gulf Houghton agreement?: 1 What is the geographic scope of the Quaker/Gulf Houghton agreement?: 2 What percentage of publicly traded securities can Gulf Houghton sellers own as passive investors?: 1 How long must before Gulf Houghton sellers can hire former employees?: 1 Mean recall: 93.33% Mean reciprocal rank: 73.56%