Reading Charts Graphs Powerpoints
Working with Charts, Graphs, and Slide Decks
Claude is highly capable of working with charts, graphs, and broader slide decks. Depending on your use case, there are a number of tips and tricks that you may want to take advantage of. This recipe will show you common patterns for using Claude with these materials.
Charts and Graphs
For the most part, using claude with charts and graphs is simple. Let's walk through how to ingest them and pass them to Claude, as well as some common tips to improve your results.
Ingestion and calling the Claude API
The best way to pass Claude charts and graphs is to take advantage of its vision capabilities and the PDF support feature. That is, give Claude a PDF document of the chart or graph, along with a text question about it.
At the moment, only claude-sonnet-4-6 supports the PDF feature. Since the feature is still in beta, you will need to provide it with the pdfs-2024-09-25 beta header.
Let's see how we can pass this document to the model alongside a simple question.
This is a page from Carvana's 2021 Annual Report showing four key metrics: retail units sold, total revenue, total markets at year end, and car vending machines, all displaying significant growth from 2014 to 2021.
That's pretty good! Now let's ask it some more useful questions.
----------Question 1---------- According to the graph showing Total Revenue ($M), Carvana's revenue in 2020 was $5,587 million (or approximately $5.59 billion). ----------Question 2---------- According to the "TOTAL MARKETS AT YEAR END" graph, Carvana started with 4 markets in 2014 and grew to 311 markets by 2021. Therefore, Carvana added 307 markets since 2014 (311 - 4 = 307 additional markets). ----------Question 3---------- Let me calculate this for you: 2016 Revenue: $365 million 2016 Retail Units Sold: 18,761 units $365 million ÷ 18,761 units = $19,455 per unit (rounded to nearest dollar) So in 2016, Carvana's revenue per retail unit sold was approximately $19,455.
As you can see, Claude is capable of answering fairly detailed questions about charts and graphs. However, there are some tips and tricks that will help you get the most out of it.
- Sometimes Claude's arithmetic capabilities get in the way. You'll notice that if you sample the third question above it will occasionally output an incorrect final answer because it messes up the arithmetic. Consider providing Claude with a calculator tool to ensure it doesn't make these types of mistakes.
- With super complicated charts and graphs, we can ask Claude to "First describe every data point you see in the document" as a way to elicit similar improvements to what we seen in traditional Chain of Thought.
- Claude occasionally struggles with charts that depend on lots of colors to convey information, such as grouped bar charts with many groups. Asking Claude to first identify the colors in your graph using HEX codes can boost its accuracy.
Slide Decks
Now that we know Claude is a charts and graphs wizard, it is only logical that we extend it to the true home of charts and graphs - slide decks!
Slides represent a critical source of information for many domains, including financial services. While you can use packages like PyPDF to extract text from slide decks, their chart/graph heavy nature often makes this a poor choice as models will struggle to access the information they actually need.
The PDF support feature can be a great replacement as a result. It uses both extracted text and vision in order when processing PDF documents. In this section we will go over how to use PDF documents in Claude to review slide decks, and how to deal with some common pitfalls of this approach.
The best way to get a typical slide deck into claude is to download it as a PDF and provide it directly to Claude.
According to the financial results shown in the presentation, Twilio's year-over-year revenue growth for fiscal year 2023 was 9%. This can be found in the "Total Company Results Highlights" section, which shows FY 2023 revenue growth of 9%.
This approach is a great way to get started, and for some use cases offers the best performance. However, there are some limitations.
- You can only include a total of 100 pages across all provided documents in a request (we intend to increase this limit over time).
- If you are using slide content as part of RAG, introducing multimodal PDFs into your embeddings can cause problems
Luckily, we can take advantage of Claude's vision capabilities to get a much higher quality representation of the slide deck in text form than normal pdf text extraction allows.
We find the best way to do this is to ask Claude to sequentially narrate the deck from start to finish, passing it the current slide and its prior narration. Let's see how.
Now that we have a text-based narration (it's far from perfect but it's pretty good), we have the ability to use this deck with any text-only workflow. Including vector search!
As a final sanity check, let's ask a few questions of our narration-only setup!
----------Question 1---------- Let me calculate this: Segment revenue in Q4 2023: $75 million Total revenue in Q4 2023: $1,076 million $75M ÷ $1,076M = 0.0697 or approximately 7% Therefore, the Segment business line represented approximately 7% of Twilio's total Q4 2023 revenue. ----------Question 2---------- Decreasing. The transcript shows Q4 2023 revenue growth was 5% year-over-year, while for the full year 2023 revenue growth was 9% year-over-year, indicating a slowing growth rate. Additionally, the Q1 2024 guidance projects even lower growth of 2-3% year-over-year, confirming the declining trend. ----------Question 3---------- Let me help calculate the acquisition revenue for 2023. From the transcript, we can see: - Total revenue for 2023: $4,154 million - Organic revenue for 2023: $4,146 million Therefore, acquisition revenue would be: Total Revenue - Organic Revenue = $4,154M - $4,146M = $8 million So the acquisition revenue for the year ended December 31, 2023 was $8 million. This can be verified by the fact that the difference between total revenue growth (9%) and organic revenue growth (10%) also suggests a small contribution from acquisitions.
Looks good! With these techniques at your side, you are ready to start applying models to chart and graph heavy content like slide decks.