Example FinancialReport RAG
llamaAIvllmmachine-learningend-to-end-use-casesContextual-Chunking-RAGllama2LLMllama-cookbookPythonfinetuningpytorchlangchain
Export
Contextual keywords generation for Financial report document
In this Example we will use a Financial report document taken from https://github.com/patronus-ai/financebench/tree/main/pdfs
Steps:
- Parse financial report document (pdf file) using LlamaParse.
- Split document into chunks. Here we use basic tokens-based chunking with constant chunk_size, but you can use any other method for chunking.
- Generate contextual keywords for each chunk.
- Create questions related to randomly selected chunks since a predefined test set is unavailable.
- Evaluate the method by retrieving the top five most relevant chunks based on cosine similarity between chunk and question embeddings, checking if the correct chunk is included.
- To compare results with raw content (without keywords), create a new index in the "Create index" section and modify the document structure by replacing Document(text='#'+x['keywords']+'\n'+x['content'], .. ) with Document(text=x['content'], .. ).
[ ]
[ ]
[12]
[9]
--- Test: What is the fiscal year end date for Adobe Systems Incorporated's annual report filed with the SEC in 2015? idx: 0
{'id': '7'}
{'id': '0'}
{'id': '0'}
{'id': '4'}
{'id': '2'}
--- Test: What is the Commission File Number for Adobe Systems Incorporated? idx: 0
{'id': '1'}
{'id': '0'}
{'id': '10'}
{'id': '468'}
{'id': '397'}
--- Test: In which state is Adobe Systems Incorporated incorporated or organized? idx: 0
{'id': '10'}
{'id': '280'}
{'id': '9'}
{'id': '1'}
{'id': '8'}
--- Test: How is Adobe's business organized? idx: 13
{'id': '10'}
{'id': '280'}
{'id': '10'}
{'id': '9'}
{'id': '5'}
--- Test: What are Adobe's two strategic growth opportunities? idx: 13
{'id': '0'}
{'id': '3'}
{'id': '13'}
{'id': '1'}
{'id': '6'}
--- Test: What are the three reportable segments of Adobe's business? idx: 13
{'id': '7'}
{'id': '10'}
{'id': '184'}
{'id': '13'}
{'id': '9'}
--- Test: What is Adobe Digital Publishing Solution used for? idx: 16
{'id': '3'}
{'id': '9'}
{'id': '3'}
{'id': '10'}
{'id': '16'}
--- Test: What are some examples of mobile apps offered by Adobe's Digital Media business? idx: 16
{'id': '5'}
{'id': '3'}
{'id': '16'}
{'id': '0'}
{'id': '9'}
--- Test: What is the main purpose of Adobe's Document Cloud business? idx: 16
{'id': '4'}
{'id': '6'}
{'id': '5'}
{'id': '3'}
{'id': '0'}
--- Test: What is Adobe's strategy to accelerate the adoption of Creative Cloud? idx: 18
{'id': '8'}
{'id': '10'}
{'id': '18'}
{'id': '3'}
{'id': '174'}
--- Test: How does Adobe's subscription model benefit new customers? idx: 18
{'id': '9'}
{'id': '0'}
{'id': '6'}
{'id': '5'}
{'id': '18'}
--- Test: What is driving the increase in Adobe's recurring revenue over the next several years? idx: 18
{'id': '2'}
{'id': '6'}
{'id': '4'}
{'id': '18'}
{'id': '1'}
--- Test: What is Adobe Stock and how is it integrated with Creative Cloud apps? idx: 29
{'id': '29'}
{'id': '3'}
{'id': '10'}
{'id': '8'}
{'id': '4'}
--- Test: How do Adobe's mobile apps extend the capabilities of Creative Cloud desktop tools? idx: 29
{'id': '6'}
{'id': '29'}
{'id': '4'}
{'id': '8'}
{'id': '5'}
--- Test: What licensing options are available for Creative Cloud, and how can individuals and teams subscribe to the service? idx: 29
{'id': '6'}
{'id': '174'}
{'id': '10'}
{'id': '18'}
{'id': '8'}
--- Test: Who are the main competitors of Adobe in the electronic signatures market? idx: 48
{'id': '48'}
{'id': '3'}
{'id': '5'}
{'id': '7'}
{'id': '6'}
--- Test: What companies compete with Adobe Marketing Cloud in the digital marketing space? idx: 48
{'id': '7'}
{'id': '48'}
{'id': '1'}
{'id': '3'}
{'id': '6'}
--- Test: How do Adobe's competitors deliver their solutions to customers in the digital marketing market? idx: 48
{'id': '48'}
{'id': '7'}
{'id': '1'}
{'id': '6'}
{'id': '1'}
--- Test: What was Michael Dillon's role at Sun Microsystems from April 2006 to January 2010? idx: 67
{'id': '67'}
{'id': '3'}
{'id': '2'}
{'id': '0'}
{'id': '77'}
--- Test: Which company did Michael Dillon join in August 2012 as Senior Vice President, General Counsel and Corporate Secretary? idx: 67
{'id': '67'}
{'id': '2'}
{'id': '3'}
{'id': '0'}
{'id': '77'}
--- Test: What was Michael Dillon's position at ONI Systems Corp from October 1999 until June 2002? idx: 67
{'id': '2'}
{'id': '3'}
{'id': '67'}
{'id': '0'}
{'id': '77'}
--- Test: What positions did Bryan Lamkin hold at Adobe from 1992 to 2006? idx: 72
{'id': '5'}
{'id': '4'}
{'id': '72'}
{'id': '3'}
{'id': '77'}
--- Test: What company did Bryan Lamkin co-found and serve as CEO of from June 2010 to May 2011? idx: 72
{'id': '4'}
{'id': '5'}
{'id': '72'}
{'id': '3'}
{'id': '9'}
--- Test: Who is the Senior Vice President and Chief Marketing Officer at Adobe? idx: 72
{'id': '77'}
{'id': '8'}
{'id': '9'}
{'id': '2'}
{'id': '72'}
--- Test: What was Mr. Parasnis' role at Oracle from January 2012 to November 2013? idx: 77
{'id': '10'}
{'id': '77'}
{'id': '9'}
{'id': '3'}
{'id': '67'}
--- Test: Who was the General Manager of Microsoft Azure AppFabric at Microsoft from April 2009 to December 2011? idx: 77
{'id': '10'}
{'id': '77'}
{'id': '3'}
{'id': '67'}
{'id': '9'}
--- Test: When did Adobe acquire Omniture, Inc.? idx: 77
{'id': '77'}
{'id': '2'}
{'id': '1'}
{'id': '7'}
{'id': '457'}
--- Test: What are the potential risks associated with collecting and managing customer data? idx: 102
{'id': '2'}
{'id': '0'}
{'id': '102'}
{'id': '0'}
{'id': '6'}
--- Test: How can hardware or software failures affect the accuracy of collected data? idx: 102
{'id': '102'}
{'id': '4'}
{'id': '2'}
{'id': '7'}
{'id': '0'}
--- Test: What are the potential consequences of a security breach or malware attack on data collection systems? idx: 102
{'id': '2'}
{'id': '102'}
{'id': '0'}
{'id': '7'}
{'id': '9'}
--- Test: What are the potential risks associated with integrating acquired companies or technologies? idx: 103
{'id': '103'}
{'id': '6'}
{'id': '5'}
{'id': '106'}
{'id': '10'}
--- Test: How might the disruption of integrating an acquisition affect a company's business and employees? idx: 103
{'id': '103'}
{'id': '5'}
{'id': '6'}
{'id': '106'}
{'id': '4'}
--- Test: What are some challenges that a company may face when entering new markets through an acquisition? idx: 103
{'id': '103'}
{'id': '6'}
{'id': '5'}
{'id': '7'}
{'id': '6'}
--- Test: What are the potential consequences of not completing an announced acquisition transaction or integrating an acquired business successfully? idx: 106
{'id': '103'}
{'id': '5'}
{'id': '6'}
{'id': '106'}
{'id': '4'}
--- Test: Why may the company incur substantial costs related to intellectual property rights? idx: 106
{'id': '8'}
{'id': '106'}
{'id': '2'}
{'id': '5'}
{'id': '3'}
--- Test: How can intellectual property disputes and litigation impact the company's business operations? idx: 106
{'id': '106'}
{'id': '8'}
{'id': '2'}
{'id': '6'}
{'id': '0'}
--- Test: What are some of the global business risks faced by Adobe? idx: 127
{'id': '127'}
{'id': '5'}
{'id': '3'}
{'id': '3'}
{'id': '2'}
--- Test: How might delays in obtaining export licenses affect Adobe's revenue? idx: 127
{'id': '1'}
{'id': '127'}
{'id': '6'}
{'id': '289'}
{'id': '0'}
--- Test: What percentage of Adobe's employees are located outside the United States? idx: 127
{'id': '127'}
{'id': '0'}
{'id': '192'}
{'id': '5'}
{'id': '9'}
--- Test: What are the potential risks associated with Adobe's investment portfolio? idx: 137
{'id': '137'}
{'id': '3'}
{'id': '3'}
{'id': '4'}
{'id': '9'}
--- Test: Does Adobe own or lease most of its principal properties used during fiscal 2015? idx: 137
{'id': '7'}
{'id': '10'}
{'id': '137'}
{'id': '2'}
{'id': '6'}
--- Test: Where are the two locations where Adobe owns both the building and the land? idx: 137
{'id': '6'}
{'id': '7'}
{'id': '7'}
{'id': '10'}
{'id': '8'}
--- Test: What type of relief are the plaintiffs seeking in the class action lawsuit? idx: 145
{'id': '145'}
{'id': '3'}
{'id': '1'}
{'id': '2'}
{'id': '3'}
--- Test: Which companies are named as defendants in the antitrust litigation case? idx: 145
{'id': '3'}
{'id': '145'}
{'id': '3'}
{'id': '1'}
{'id': '5'}
--- Test: What is the time period specified by the court for employees who worked at Apple to be included in the certified class? idx: 145
{'id': '145'}
{'id': '3'}
{'id': '1'}
{'id': '5'}
{'id': '4'}
--- Test: On which market is Adobe's common stock traded? idx: 149
{'id': '149'}
{'id': '1'}
{'id': '2'}
{'id': '29'}
{'id': '397'}
--- Test: What was the highest sales price per share of Adobe's common stock in Fiscal 2015? idx: 149
{'id': '149'}
{'id': '399'}
{'id': '192'}
{'id': '3'}
{'id': '393'}
--- Test: What is the symbol under which Adobe's common stock is traded on the NASDAQ Global Select Market? idx: 149
{'id': '149'}
{'id': '1'}
{'id': '2'}
{'id': '5'}
{'id': '29'}
--- Test: How much authority was granted by the Board of Directors for the new stock repurchase program in January 2015? idx: 152
{'id': '152'}
{'id': '240'}
{'id': '397'}
{'id': '8'}
{'id': '5'}
--- Test: What was the prepayment amount provided to a large financial institution as part of the stock repurchase program in September 2015? idx: 152
{'id': '8'}
{'id': '152'}
{'id': '6'}
{'id': '5'}
{'id': '8'}
--- Test: When is the stock repurchase program approved by the Board of Directors set to end? idx: 152
{'id': '152'}
{'id': '397'}
{'id': '10'}
{'id': '240'}
{'id': '8'}
--- Test: What were the two companies that Adobe integrated into its reportable segments in fiscal 2013? idx: 158
{'id': '184'}
{'id': '7'}
{'id': '10'}
{'id': '8'}
{'id': '0'}
--- Test: Why did Adobe not present pro forma information for its acquisitions in fiscal years 2015, 2014, and 2013? idx: 158
{'id': '158'}
{'id': '3'}
{'id': '6'}
{'id': '7'}
{'id': '2'}
--- Test: What factors does Adobe use as a basis for its assumptions, judgments, and estimates when preparing its Consolidated Financial Statements? idx: 158
{'id': '7'}
{'id': '5'}
{'id': '8'}
{'id': '8'}
{'id': '6'}
--- Test: What is Adobe Creative Cloud and what kind of licenses does it offer? idx: 174
{'id': '174'}
{'id': '8'}
{'id': '29'}
{'id': '6'}
{'id': '4'}
--- Test: How does Adobe Creative Cloud deliver value to its users compared to perpetual licenses? idx: 174
{'id': '174'}
{'id': '18'}
{'id': '10'}
{'id': '4'}
{'id': '0'}
--- Test: What are the different ways that customers can acquire Adobe Creative Cloud capabilities? idx: 174
{'id': '3'}
{'id': '10'}
{'id': '8'}
{'id': '8'}
{'id': '16'}
--- Test: What are the reportable segments of Adobe's business? idx: 184
{'id': '7'}
{'id': '10'}
{'id': '184'}
{'id': '9'}
{'id': '8'}
--- Test: How much subscription revenue did Adobe's Digital Media segment generate in fiscal 2015? idx: 184
{'id': '184'}
{'id': '0'}
{'id': '3'}
{'id': '2'}
{'id': '1'}
--- Test: What was the percentage change in subscription revenue for Adobe's Digital Marketing segment from fiscal 2013 to fiscal 2014? idx: 184
{'id': '191'}
{'id': '184'}
{'id': '0'}
{'id': '2'}
{'id': '3'}
--- Test: What was the percentage growth of Adobe Marketing Cloud revenue in fiscal 2014 compared to fiscal 2013? idx: 191
{'id': '191'}
{'id': '2'}
{'id': '3'}
{'id': '7'}
{'id': '0'}
--- Test: Which product contributed to the increase in Digital Marketing revenue in fiscal 2014? idx: 191
{'id': '191'}
{'id': '2'}
{'id': '4'}
{'id': '1'}
{'id': '5'}
--- Test: Why did revenue from Print and Publishing decrease in fiscal 2014 compared to fiscal 2013? idx: 191
{'id': '3'}
{'id': '8'}
{'id': '7'}
{'id': '191'}
{'id': '184'}
--- Test: What percentage of Adobe's total revenue came from the Americas in fiscal year 2015? idx: 192
{'id': '192'}
{'id': '0'}
{'id': '6'}
{'id': '0'}
{'id': '7'}
--- Test: How much revenue did Adobe generate in the APAC region in fiscal year 2013? idx: 192
{'id': '192'}
{'id': '0'}
{'id': '7'}
{'id': '10'}
{'id': '8'}
--- Test: What was the percentage increase in Adobe's total revenue from fiscal year 2014 to fiscal year 2015? idx: 192
{'id': '6'}
{'id': '1'}
{'id': '191'}
{'id': '0'}
{'id': '3'}
--- Test: What was the total revenue impact of foreign currency fluctuations on Adobe in fiscal year 2015? idx: 196
{'id': '8'}
{'id': '196'}
{'id': '6'}
{'id': '192'}
{'id': '7'}
--- Test: How much did Adobe's hedging program offset the revenue decline in EMEA in fiscal year 2015? idx: 196
{'id': '8'}
{'id': '196'}
{'id': '192'}
{'id': '3'}
{'id': '0'}
--- Test: Which currencies strengthened against the U.S. Dollar in fiscal year 2015, causing a decrease in Adobe's revenue? idx: 196
{'id': '8'}
{'id': '196'}
{'id': '6'}
{'id': '10'}
{'id': '7'}
--- Test: What is the significance of unbilled deferred revenue backlog in Adobe's financial statements? idx: 198
{'id': '198'}
{'id': '1'}
{'id': '7'}
{'id': '5'}
{'id': '6'}
--- Test: How much was Adobe's unbilled deferred revenue backlog as of November 27, 2015? idx: 198
{'id': '198'}
{'id': '1'}
{'id': '7'}
{'id': '1'}
{'id': '3'}
--- Test: What percentage of Adobe's unbilled deferred revenue backlog was not expected to be billed during fiscal 2016? idx: 198
{'id': '198'}
{'id': '1'}
{'id': '7'}
{'id': '0'}
{'id': '192'}
--- Test: What were the main reasons for the increase in Adobe's cost of subscription revenue from Fiscal 2013 to Fiscal 2014? idx: 203
{'id': '191'}
{'id': '203'}
{'id': '2'}
{'id': '3'}
{'id': '7'}
--- Test: How did Adobe's acquisition of Neolane in Fiscal 2013 impact the company's compensation and related benefits costs? idx: 203
{'id': '0'}
{'id': '203'}
{'id': '392'}
{'id': '158'}
{'id': '10'}
--- Test: What factors contributed to the increase in Adobe's royalty costs in Fiscal 2014? idx: 203
{'id': '203'}
{'id': '7'}
{'id': '8'}
{'id': '4'}
{'id': '6'}
--- Test: What was the main reason for the increase in amortization expense during fiscal 2015? idx: 214
{'id': '214'}
{'id': '10'}
{'id': '8'}
{'id': '7'}
{'id': '6'}
--- Test: How did the amortization expense change during fiscal 2014 compared to fiscal 2013? idx: 214
{'id': '214'}
{'id': '8'}
{'id': '8'}
{'id': '8'}
{'id': '3'}
--- Test: What was the percentage change in interest and other income (expense), net from fiscal 2014 to fiscal 2013? idx: 214
{'id': '7'}
{'id': '10'}
{'id': '8'}
{'id': '214'}
{'id': '6'}
--- Test: What is the estimated range of potential decreases in underlying unrecognized tax benefits? idx: 222
{'id': '1'}
{'id': '2'}
{'id': '7'}
{'id': '222'}
{'id': '361'}
--- Test: How much cash and cash equivalents did the company have as of November 27, 2015? idx: 222
{'id': '3'}
{'id': '2'}
{'id': '6'}
{'id': '4'}
{'id': '3'}
--- Test: What is the purpose of reading the data in conjunction with the Consolidated Statements of Cash Flows? idx: 222
{'id': '3'}
{'id': '7'}
{'id': '4'}
{'id': '8'}
{'id': '3'}
--- Test: What were the primary reasons for the net cash used for investing activities in fiscal 2014? idx: 228
{'id': '7'}
{'id': '228'}
{'id': '4'}
{'id': '5'}
{'id': '6'}
--- Test: How much was spent on acquisitions of Neolane and Behance in fiscal 2013? idx: 228
{'id': '309'}
{'id': '2'}
{'id': '229'}
{'id': '158'}
{'id': '228'}
--- Test: Where were the construction projects located that involved purchases of property and equipment in fiscal 2013? idx: 228
{'id': '228'}
{'id': '7'}
{'id': '1'}
{'id': '6'}
{'id': '4'}
--- Test: How much did Adobe issue in senior notes in January 2015? idx: 229
{'id': '0'}
{'id': '3'}
{'id': '2'}
{'id': '192'}
{'id': '2'}
--- Test: What was the purpose of using $600 million of the proceeds from the 2025 Notes offering? idx: 229
{'id': '10'}
{'id': '0'}
{'id': '4'}
{'id': '0'}
{'id': '229'}
--- Test: What is the interest rate of the senior notes due February 1, 2025? idx: 229
{'id': '0'}
{'id': '8'}
{'id': '4'}
{'id': '1'}
{'id': '8'}
--- Test: What was the average price of shares repurchased under the plan in Fiscal 2013? idx: 240
{'id': '240'}
{'id': '7'}
{'id': '399'}
{'id': '383'}
{'id': '6'}
--- Test: How many shares were repurchased under the structured repurchases plan approved in April 2012 for Fiscal 2014? idx: 240
{'id': '240'}
{'id': '7'}
{'id': '6'}
{'id': '5'}
{'id': '152'}
--- Test: When was the board approval date for the structured repurchases plan that resulted in 4,849 shares being repurchased? idx: 240
{'id': '240'}
{'id': '152'}
{'id': '7'}
{'id': '8'}
{'id': '6'}
--- Test: What is the estimated increase in the fair value of financial hedging instruments if the U.S. Dollar increases by 10%? idx: 248
{'id': '248'}
{'id': '2'}
{'id': '8'}
{'id': '1'}
{'id': '9'}
--- Test: What model is used to measure the hypothetical market value of option contracts in the sensitivity analysis? idx: 248
{'id': '248'}
{'id': '8'}
{'id': '4'}
{'id': '5'}
{'id': '10'}
--- Test: Why doesn't the company use foreign exchange contracts to hedge local currency denominated operating expenses in certain countries? idx: 248
{'id': '248'}
{'id': '4'}
{'id': '3'}
{'id': '7'}
{'id': '4'}
--- Test: What was Adobe Systems Incorporated's net income for the given period? idx: 264
{'id': '9'}
{'id': '1'}
{'id': '4'}
{'id': '3'}
{'id': '264'}
--- Test: What was the basic net income per share for Adobe Systems Incorporated? idx: 264
{'id': '264'}
{'id': '3'}
{'id': '5'}
{'id': '4'}
{'id': '1'}
--- Test: How many shares were used to compute the diluted net income per share for Adobe Systems Incorporated? idx: 264
{'id': '5'}
{'id': '1'}
{'id': '264'}
{'id': '6'}
{'id': '3'}
--- Test: What was the net increase/decrease from derivatives designated as hedging instruments in the reported period? idx: 266
{'id': '9'}
{'id': '7'}
{'id': '8'}
{'id': '0'}
{'id': '4'}
--- Test: How much was the total comprehensive income, net of taxes, for Adobe Systems Incorporated? idx: 266
{'id': '1'}
{'id': '10'}
{'id': '393'}
{'id': '264'}
{'id': '9'}
--- Test: What was the amount of unrealized gains/losses on derivative instruments for Adobe Systems Incorporated? idx: 266
{'id': '9'}
{'id': '9'}
{'id': '266'}
{'id': '1'}
{'id': '385'}
--- Test: What is the total stockholders' equity of Adobe Systems Incorporated as of November 30, 2012? idx: 267
{'id': '267'}
{'id': '393'}
{'id': '6'}
{'id': '5'}
{'id': '385'}
--- Test: How much net income did Adobe Systems Incorporated report in the period ending November 30, 2012? idx: 267
{'id': '1'}
{'id': '393'}
{'id': '9'}
{'id': '4'}
{'id': '10'}
--- Test: What is the amount of treasury stock held by Adobe Systems Incorporated as of November 30, 2012? idx: 267
{'id': '6'}
{'id': '8'}
{'id': '5'}
{'id': '2'}
{'id': '7'}
--- Test: What was the net increase (decrease) in cash and cash equivalents for Adobe Systems Incorporated in November 2015? idx: 278
{'id': '278'}
{'id': '9'}
{'id': '0'}
{'id': '6'}
{'id': '7'}
--- Test: How much cash and cash equivalents did Adobe Systems Incorporated have at the beginning of the year in November 2014? idx: 278
{'id': '4'}
{'id': '0'}
{'id': '4'}
{'id': '9'}
{'id': '278'}
--- Test: What was the effect of foreign currency exchange rates on Adobe Systems Incorporated's cash flows in November 2013? idx: 278
{'id': '278'}
{'id': '8'}
{'id': '196'}
{'id': '10'}
{'id': '2'}
--- Test: When was Adobe Systems Incorporated founded? idx: 280
{'id': '280'}
{'id': '10'}
{'id': '9'}
{'id': '468'}
{'id': '1'}
--- Test: How does Adobe Systems Incorporated market and license its products and services? idx: 280
{'id': '9'}
{'id': '280'}
{'id': '9'}
{'id': '9'}
{'id': '0'}
--- Test: What models does Adobe Systems Incorporated use to offer its products, besides traditional licensing? idx: 280
{'id': '9'}
{'id': '9'}
{'id': '280'}
{'id': '8'}
{'id': '1'}
--- Test: How does Adobe Systems Incorporated recognize revenue from OEM licensing? idx: 289
{'id': '289'}
{'id': '0'}
{'id': '1'}
{'id': '7'}
{'id': '4'}
--- Test: What factors does Adobe consider when estimating future returns and rebates for its desktop application product revenue? idx: 289
{'id': '289'}
{'id': '10'}
{'id': '5'}
{'id': '4'}
{'id': '6'}
--- Test: When does Adobe recognize royalty revenue from OEM customers? idx: 289
{'id': '289'}
{'id': '1'}
{'id': '7'}
{'id': '0'}
{'id': '9'}
--- Test: How much was the total final purchase price for Neolane? idx: 309
{'id': '309'}
{'id': '0'}
{'id': '229'}
{'id': '158'}
{'id': '3'}
--- Test: What was the allocation of the purchase price for Neolane towards goodwill? idx: 309
{'id': '309'}
{'id': '0'}
{'id': '3'}
{'id': '4'}
{'id': '2'}
--- Test: What was the approximate total final purchase price for Behance? idx: 309
{'id': '2'}
{'id': '309'}
{'id': '383'}
{'id': '7'}
{'id': '399'}
--- Test: What is the total value of short-term investments in municipal securities? idx: 315
{'id': '5'}
{'id': '315'}
{'id': '4'}
{'id': '3'}
{'id': '6'}
--- Test: How much are the total unrealized gains on short-term investments? idx: 315
{'id': '10'}
{'id': '5'}
{'id': '6'}
{'id': '315'}
{'id': '316'}
--- Test: What is the total value of cash, cash equivalents, and short-term investments? idx: 315
{'id': '6'}
{'id': '3'}
{'id': '4'}
{'id': '3'}
{'id': '3'}
--- Test: What was the fair value of corporate bonds and commercial paper as of November 27, 2015? idx: 316
{'id': '3'}
{'id': '2'}
{'id': '10'}
{'id': '315'}
{'id': '7'}
--- Test: How much were the gross unrealized losses for municipal securities in 2014? idx: 316
{'id': '316'}
{'id': '10'}
{'id': '9'}
{'id': '5'}
{'id': '315'}
--- Test: What was the fair value of asset-backed securities as of November 28, 2014? idx: 316
{'id': '2'}
{'id': '324'}
{'id': '8'}
{'id': '315'}
{'id': '316'}
--- Test: What is the total value of Adobe Systems Incorporated's debt and marketable equity securities classified as short-term investments? idx: 319
{'id': '319'}
{'id': '5'}
{'id': '137'}
{'id': '0'}
{'id': '1'}
--- Test: How does Adobe Systems Incorporated evaluate whether a security has experienced an other-than-temporary decline in fair value? idx: 319
{'id': '319'}
{'id': '7'}
{'id': '0'}
{'id': '8'}
{'id': '5'}
--- Test: What is the value of Adobe Systems Incorporated's debt and marketable equity securities that are due within one year? idx: 319
{'id': '319'}
{'id': '2'}
{'id': '0'}
{'id': '2'}
{'id': '5'}
--- Test: What is the total value of Adobe Systems Incorporated's liabilities as of November 28, 2014? idx: 324
{'id': '0'}
{'id': '393'}
{'id': '7'}
{'id': '385'}
{'id': '3'}
--- Test: How much of Adobe Systems Incorporated's cash equivalents are invested in money market mutual funds as of November 28, 2014? idx: 324
{'id': '6'}
{'id': '5'}
{'id': '324'}
{'id': '4'}
{'id': '278'}
--- Test: What method is used to determine the fair value of Adobe Systems Incorporated's financial assets and liabilities, as reported on November 28, 2014? idx: 324
{'id': '324'}
{'id': '319'}
{'id': '6'}
{'id': '2'}
{'id': '5'}
--- Test: What is the estimated range of potential decreases in underlying unrecognized tax benefits for Adobe Systems Incorporated? idx: 361
{'id': '1'}
{'id': '7'}
{'id': '6'}
{'id': '5'}
{'id': '8'}
--- Test: Why did Adobe Systems Incorporated initiate a restructuring plan in the fourth quarter of fiscal 2014? idx: 361
{'id': '362'}
{'id': '361'}
{'id': '364'}
{'id': '0'}
{'id': '3'}
--- Test: How many positions were reduced as part of Adobe's Fiscal 2014 Restructuring Plan? idx: 361
{'id': '362'}
{'id': '3'}
{'id': '0'}
{'id': '361'}
{'id': '364'}
--- Test: How much did Adobe Systems Incorporated spend on restructuring charges related to its fiscal 2014 restructuring plan? idx: 362
{'id': '362'}
{'id': '364'}
{'id': '0'}
{'id': '0'}
{'id': '361'}
--- Test: What actions did Adobe Systems Incorporated take as part of its fiscal 2014 restructuring plan? idx: 362
{'id': '362'}
{'id': '361'}
{'id': '0'}
{'id': '3'}
{'id': '364'}
--- Test: As of November 27, 2015, what was the status of Adobe Systems Incorporated's Other Restructuring Plans? idx: 362
{'id': '0'}
{'id': '362'}
{'id': '364'}
{'id': '361'}
{'id': '3'}
--- Test: How much accrued restructuring charges did Adobe Systems have as of November 27, 2015? idx: 364
{'id': '362'}
{'id': '0'}
{'id': '364'}
{'id': '9'}
{'id': '0'}
--- Test: When does Adobe Systems expect to pay off the majority of its facilities-related liabilities? idx: 364
{'id': '7'}
{'id': '4'}
{'id': '364'}
{'id': '0'}
{'id': '3'}
--- Test: In what year did Adobe adopt its Employee Investment Plan, a retirement savings plan for its U.S. employees? idx: 364
{'id': '2'}
{'id': '3'}
{'id': '6'}
{'id': '0'}
{'id': '4'}
--- Test: What is the expected life of employee stock purchase rights in fiscal years 2015, 2014, and 2013? idx: 376
{'id': '376'}
{'id': '383'}
{'id': '240'}
{'id': '8'}
{'id': '5'}
--- Test: What is the range of volatility used to value employee stock purchase rights in fiscal year 2015? idx: 376
{'id': '376'}
{'id': '383'}
{'id': '1'}
{'id': '8'}
{'id': '4'}
--- Test: What was the beginning outstanding balance of restricted stock units in fiscal year 2015? idx: 376
{'id': '3'}
{'id': '240'}
{'id': '4'}
{'id': '10'}
{'id': '376'}
--- Test: How many shares were granted to participants in the 2012 Performance Share Program? idx: 380
{'id': '380'}
{'id': '7'}
{'id': '10'}
{'id': '0'}
{'id': '5'}
--- Test: What was the actual performance achievement of participants in the 2012 Performance Share Program as a percentage of target? idx: 380
{'id': '380'}
{'id': '9'}
{'id': '10'}
{'id': '8'}
{'id': '0'}
--- Test: How many shares vested in the first quarter of fiscal 2013 under the 2012 Performance Share Program? idx: 380
{'id': '380'}
{'id': '7'}
{'id': '10'}
{'id': '8'}
{'id': '240'}
--- Test: What was the average price at which employees purchased shares under the ESPP during fiscal 2015? idx: 383
{'id': '383'}
{'id': '376'}
{'id': '7'}
{'id': '399'}
{'id': '240'}
--- Test: How many shares were purchased by employees under the ESPP during fiscal 2013? idx: 383
{'id': '383'}
{'id': '376'}
{'id': '6'}
{'id': '8'}
{'id': '10'}
--- Test: What is the intrinsic value of shares purchased under the ESPP, and how is it calculated? idx: 383
{'id': '8'}
{'id': '376'}
{'id': '383'}
{'id': '6'}
{'id': '0'}
--- Test: What was the weighted average fair value of options granted by Adobe Systems Incorporated during fiscal 2013? idx: 385
{'id': '385'}
{'id': '2'}
{'id': '6'}
{'id': '386'}
{'id': '4'}
--- Test: How much was the total intrinsic value of options exercised by Adobe Systems Incorporated during fiscal 2015? idx: 385
{'id': '385'}
{'id': '2'}
{'id': '386'}
{'id': '7'}
{'id': '6'}
--- Test: What is the method used by Adobe Systems Incorporated to calculate the intrinsic value of options exercised? idx: 385
{'id': '385'}
{'id': '2'}
{'id': '386'}
{'id': '6'}
{'id': '4'}
--- Test: What was the weighted average exercise price of Adobe Systems Incorporated's stock options outstanding in fiscal year 2015? idx: 386
{'id': '386'}
{'id': '385'}
{'id': '10'}
{'id': '2'}
{'id': '6'}
--- Test: How many shares of stock options were outstanding at Adobe Systems Incorporated as of November 27, 2015? idx: 386
{'id': '386'}
{'id': '385'}
{'id': '2'}
{'id': '8'}
{'id': '397'}
--- Test: What was the aggregate intrinsic value of Adobe Systems Incorporated's stock options outstanding in fiscal year 2014? idx: 386
{'id': '385'}
{'id': '386'}
{'id': '2'}
{'id': '8'}
{'id': '5'}
--- Test: What was the total stock-based compensation cost for Adobe Systems Incorporated in fiscal year 2015? idx: 392
{'id': '392'}
{'id': '393'}
{'id': '6'}
{'id': '4'}
{'id': '8'}
--- Test: How much did Adobe Systems Incorporated spend on Sales and Marketing in fiscal year 2013? idx: 392
{'id': '0'}
{'id': '191'}
{'id': '10'}
{'id': '7'}
{'id': '392'}
--- Test: What was the cost of Research and Development for Adobe Systems Incorporated in fiscal year 2014? idx: 392
{'id': '10'}
{'id': '203'}
{'id': '7'}
{'id': '0'}
{'id': '10'}
--- Test: What were the total stock-based compensation costs for Adobe Systems Incorporated in fiscal year 2015? idx: 393
{'id': '393'}
{'id': '392'}
{'id': '6'}
{'id': '4'}
{'id': '10'}
--- Test: How much tax benefit did Adobe Systems Incorporated record in fiscal year 2014? idx: 393
{'id': '0'}
{'id': '393'}
{'id': '4'}
{'id': '7'}
{'id': '1'}
--- Test: What was the total accumulated other comprehensive income (loss) for Adobe Systems Incorporated in fiscal year 2015? idx: 393
{'id': '10'}
{'id': '0'}
{'id': '1'}
{'id': '393'}
{'id': '7'}
--- Test: What is the total amount of authority granted to Adobe Systems Incorporated for stock repurchase through the end of fiscal 2017? idx: 397
{'id': '397'}
{'id': '399'}
{'id': '2'}
{'id': '0'}
{'id': '152'}
--- Test: How much did Adobe Systems Incorporated provide in prepayments for structured stock repurchase agreements during fiscal 2015? idx: 397
{'id': '397'}
{'id': '399'}
{'id': '2'}
{'id': '400'}
{'id': '8'}
--- Test: Under which stock repurchase program was the $425.0 million prepayment made during fiscal 2015? idx: 397
{'id': '8'}
{'id': '6'}
{'id': '152'}
{'id': '5'}
{'id': '397'}
--- Test: How many shares did Adobe Systems Incorporated repurchase during fiscal 2013? idx: 399
{'id': '399'}
{'id': '397'}
{'id': '10'}
{'id': '385'}
{'id': '6'}
--- Test: What was the average price per share of Adobe's stock repurchases during fiscal 2015? idx: 399
{'id': '399'}
{'id': '240'}
{'id': '192'}
{'id': '7'}
{'id': '397'}
--- Test: How were the prepayments for stock repurchases classified on Adobe's Consolidated Balance Sheets? idx: 399
{'id': '400'}
{'id': '397'}
{'id': '2'}
{'id': '8'}
{'id': '399'}
--- Test: How much prepayment did Adobe Systems Incorporated provide to a large financial institution as part of its stock repurchase program? idx: 400
{'id': '397'}
{'id': '400'}
{'id': '2'}
{'id': '399'}
{'id': '10'}
--- Test: What is the remaining amount under Adobe's current stock repurchase authority after completing the $150 million stock repurchase agreement? idx: 400
{'id': '397'}
{'id': '400'}
{'id': '399'}
{'id': '2'}
{'id': '6'}
--- Test: How is diluted net income per share calculated by Adobe Systems Incorporated? idx: 400
{'id': '264'}
{'id': '5'}
{'id': '1'}
{'id': '6'}
{'id': '3'}
--- Test: What is the interest rate on the 2015 Notes and the 2020 Notes? idx: 416
{'id': '416'}
{'id': '0'}
{'id': '0'}
{'id': '8'}
{'id': '8'}
--- Test: When are the interest payments due on the Notes? idx: 416
{'id': '8'}
{'id': '0'}
{'id': '0'}
{'id': '416'}
{'id': '1'}
--- Test: What is the purpose of the interest rate swaps entered into by the company in June 2014? idx: 416
{'id': '1'}
{'id': '8'}
{'id': '416'}
{'id': '2'}
{'id': '1'}
--- Test: What is the name of the plan that governs the Restricted Stock Unit Award Agreements mentioned in the document? idx: 453
{'id': '9'}
{'id': '10'}
{'id': '453'}
{'id': '3'}
{'id': '5'}
--- Test: What is the filing date of the Form 8-K that includes the Director Initial Grant Restricted Stock Unit Award Agreement? idx: 453
{'id': '453'}
{'id': '9'}
{'id': '1'}
{'id': '3'}
{'id': '10'}
--- Test: What is the SEC File Number associated with the exhibits listed in the document? idx: 453
{'id': '8'}
{'id': '4'}
{'id': '0'}
{'id': '7'}
{'id': '453'}
--- Test: What is the name of the equity incentive plan adopted by Omniture, Inc. in 1999? idx: 457
{'id': '457'}
{'id': '7'}
{'id': '8'}
{'id': '10'}
{'id': '1'}
--- Test: What is the SEC file number associated with the Omniture 1999 Equity Incentive Plan? idx: 457
{'id': '457'}
{'id': '7'}
{'id': '8'}
{'id': '453'}
{'id': '460'}
--- Test: On what date was the Form S-1 filing made for the Omniture 1999 Plan's Stock Option Agreement? idx: 457
{'id': '457'}
{'id': '7'}
{'id': '9'}
{'id': '3'}
{'id': '10'}
--- Test: What is the name of the company that has an Employee Stock Option Plan mentioned in the document? idx: 460
{'id': '6'}
{'id': '460'}
{'id': '2'}
{'id': '5'}
{'id': '7'}
--- Test: What is the form number of the document that describes the 2012 Director Compensation? idx: 460
{'id': '0'}
{'id': '3'}
{'id': '2'}
{'id': '4'}
{'id': '2'}
--- Test: On what date was the 2012 Executive Annual Incentive Plan filed, according to the document? idx: 460
{'id': '0'}
{'id': '10'}
{'id': '2'}
{'id': '460'}
{'id': '380'}
--- Test: How did Adobe Systems' stock performance compare to the S&P 500 Index over a five-year period? idx: 468
{'id': '5'}
{'id': '468'}
{'id': '399'}
{'id': '5'}
{'id': '392'}
--- Test: What was the cumulative total return of the S&P 500 Software & Services Index in 2015? idx: 468
{'id': '5'}
{'id': '468'}
{'id': '8'}
{'id': '2'}
{'id': '393'}
--- Test: In which year did Adobe Systems' stock price reach approximately $194.85? idx: 468
{'id': '399'}
{'id': '468'}
{'id': '2'}
{'id': '149'}
{'id': '385'}
Test correct, all: 156 180
[ ]