ScrapeGraphAI Scrapegraph Langgraph

Scrapegraph Langgraph

scrapegraph-pysdk-nodejscookbookscrapingwired-newsjson-schemasdk-pythonweb-scraping-pythonweb-scrapingsdk-jsapiPythonscrapegraphweb-crawler

alph-notebooks/scrapegraph-py / scrapegraph_langgraph.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

🕷️ Extract Wired Science News with `langchain-scrapegraph` and `langgraph`

Presentazione senza titolo.pptx (8).png

🔧 Install `dependencies`

[1]

🔑 Import `ScrapeGraph` and `OpenAI` API keys

You can find the Scrapegraph API key here

[2]

Scrapegraph API key:
··········
OpenAI API key:
··········

📝 Defining an `Output Schema` for Webpage Content Extraction

If you already know what you want to extract from a webpage, you can define an output schema using Pydantic. This schema acts as a "blueprint" that tells the AI how to structure the response.

Pydantic Schema Quick Guide

Types of Schemas

Simple Schema
Use this when you want to extract straightforward information, such as a single piece of content.

	from pydantic import BaseModel, Field

# Simple schema for a single webpage
class PageInfoSchema(BaseModel):
    title: str = Field(description="The title of the webpage")
    description: str = Field(description="The description of the webpage")

# Example Output JSON after AI extraction
{
    "title": "ScrapeGraphAI: The Best Content Extraction Tool",
    "description": "ScrapeGraphAI provides powerful tools for structured content extraction from websites."
}

Complex Schema (Nested)
If you need to extract structured information with multiple related items (like a list of repositories), you can nest schemas.

	from pydantic import BaseModel, Field
from typing import List

# Define a schema for a single repository
class RepositorySchema(BaseModel):
    name: str = Field(description="Name of the repository (e.g., 'owner/repo')")
    description: str = Field(description="Description of the repository")
    stars: int = Field(description="Star count of the repository")
    forks: int = Field(description="Fork count of the repository")
    today_stars: int = Field(description="Stars gained today")
    language: str = Field(description="Programming language used")

# Define a schema for a list of repositories
class ListRepositoriesSchema(BaseModel):
    repositories: List[RepositorySchema] = Field(description="List of GitHub trending repositories")

# Example Output JSON after AI extraction
{
    "repositories": [
        {
            "name": "google-gemini/cookbook",
            "description": "Examples and guides for using the Gemini API",
            "stars": 8036,
            "forks": 1001,
            "today_stars": 649,
            "language": "Jupyter Notebook"
        },
        {
            "name": "TEN-framework/TEN-Agent",
            "description": "TEN Agent is a conversational AI powered by TEN, integrating Gemini 2.0 Multimodal Live API, OpenAI Realtime API, RTC, and more.",
            "stars": 3224,
            "forks": 311,
            "today_stars": 361,
            "language": "Python"
        }
    ]
}

Key Takeaways

Simple Schema: Perfect for small, straightforward extractions.
Complex Schema: Use nesting to extract lists or structured data, like "a list of repositories."

Both approaches give the AI a clear structure to follow, ensuring that the extracted content matches exactly what you need.

[71]

🚀 Initialize `langchain-scrapegraph` tools and `langgraph` prebuilt agent and run the `extraction`

Here we use SmartScraperTool to extract structured data using AI from a webpage.

If you already have an HTML file, you can upload it and use LocalScraperTool instead.

You can find more info in the official langchain documentation

[72]

We then initialize the llm model we want to use in the agent

[12]

Here we use create_react_agent to quickly use one of the prebuilt agents from langgraph.prebuilt module

You can find more info in the official langgraph documentation

[75]

Let's visualize the graph

[42]

Run the graph and stream the agent reasoning.

We are going to ask the agent to extract the content from a specific webpage.

[86]

================================ Human Message =================================

Go to https://www.wired.com/category/science/ and extract all the news
================================== Ai Message ==================================
Tool Calls:
  SmartScraper (call_jIkP03hdmoO83XT1V7PsBI6p)
 Call ID: call_jIkP03hdmoO83XT1V7PsBI6p
  Args:
    user_prompt: Extract all the news articles from the Science category on Wired. Include the title, publication date, author, and a brief summary of each article.
    website_url: https://www.wired.com/category/science/
================================= Tool Message =================================
Name: SmartScraper

{"news": [{"title": "December Wildfires Are Now a Thing", "link": "https://www.wired.com/story/december-wildfires-are-now-a-thing/", "author": "Kylie Mohr"}, {"title": "How to Manage Food Anxiety Over the Holidays", "link": "https://www.wired.com/story/how-to-cope-with-food-anxiety-during-the-festive-season/", "author": "Alison Fixsen"}, {"title": "A Spacecraft Is About to Fly Into the Sun’s Atmosphere for the First Time", "link": "https://www.wired.com/story/parker-solar-probe-atmosphere/", "author": "Eric Berger, Ars Technica"}, {"title": "To Improve Your Gut Microbiome, Spend More Time in Nature", "link": "https://www.wired.com/story/to-improve-your-gut-microbiome-spend-more-time-in-nature-kathy-willis/", "author": "Kathy Willis"}, {"title": "This Tropical Virus Is Spreading Out of the Amazon to the US and Europe", "link": "https://www.wired.com/story/this-tropical-virus-is-spreading-out-of-the-amazon-to-the-us-and-europe/", "author": "Geraldine Castro"}, {"title": "CDC Confirms First US Case of Severe Bird Flu", "link": "https://www.wired.com/story/cdc-confirms-first-us-case-of-severe-bird-flu/", "author": "Emily Mullin"}, {"title": "The Study That Called Out Black Plastic Utensils Had a Major Math Error", "link": "https://www.wired.com/story/black-plastic-utensils-study-math-error-correction/", "author": "Beth Mole, Ars Technica"}, {"title": "How Christmas Trees Could Become a Source of Low-Carbon Protein", "link": "https://www.wired.com/story/how-christmas-trees-could-become-a-source-of-low-carbon-protein/", "author": "Alexa Phillips"}, {"title": "Creating a Global Package to Solve the Problem of Plastics", "link": "https://www.wired.com/story/global-plastics-treaty-united-nations/", "author": "Susan Solomon"}, {"title": "These 3 Things Are Standing in the Way of a Global Plastics Treaty", "link": "https://www.wired.com/story/these-3-things-are-standing-in-the-way-of-a-global-plastics-treaty/", "author": "Steve Fletcher and Samuel Winton"}, {"title": "Environmental Sensing Is Here, Tracking Everything from Forest Fires to Threatened Species", "link": "https://www.wired.com/story/environmental-sensing-is-here-tracking-everything-from-forest-fires-to-threatened-species/", "author": "Sabrina Weiss"}, {"title": "Generative AI and Climate Change Are on a Collision Course", "link": "https://www.wired.com/story/true-cost-generative-ai-data-centers-energy/", "author": "Sasha Luccioni"}, {"title": "Climate Change Is Destroying Monarch Butterflies’ Winter Habitat", "link": "https://www.wired.com/story/global-warming-threatens-the-monarch-butterfly-sanctuary-but-this-scientist-prepares-a-new-home-for-them/", "author": "Andrea J. Arratibel"}, {"title": "More Humanitarian Organizations Will Harness AI’s Potential", "link": "https://www.wired.com/story/humanitarian-organizations-artificial-intelligence/", "author": "David Miliband"}, {"title": "Chocolate Has a Sustainability Problem. Science Thinks It's Found the Answer", "link": "https://www.wired.com/story/chocolate-has-a-sustainability-problem-science-thinks-its-found-the-answer/", "author": "Eve Thomas"}, {"title": "We’ve Never Been Closer to Finding Life Outside Our Solar System", "link": "https://www.wired.com/story/james-webb-space-telescope-signs-of-life/", "author": "Lisa Kaltenegger"}, {"title": "The End Is Near for NASA’s Voyager Probes", "link": "https://www.wired.com/story/the-end-is-near-for-nasas-voyager-probes/", "author": "Luca Nardi"}, {"title": "Why Can’t You Switch Seats in an Empty Airplane?", "link": "https://www.wired.com/story/why-cant-you-switch-seats-in-an-empty-airplane/", "author": "Rhett Allain"}, {"title": "The Simple Math Behind Public Key Cryptography", "link": "https://www.wired.com/story/how-public-key-cryptography-really-works-using-only-simple-math/", "author": "John Pavlus"}, {"title": "Everyone Is Capable of Mathematical Thinking--Yes, Even You", "link": "https://www.wired.com/story/everyone-is-capable-of-mathematical-thinking-yes-even-you/", "author": "Kelsey Houston-Edwards"}, {"title": "The Physics of the Macy’s Thanksgiving Day Parade Balloons", "link": "https://www.wired.com/story/the-physics-of-the-macys-thanksgiving-day-parade-balloons/", "author": "Rhett Allain"}, {"title": "A Third Person Has Received a Transplant of a Genetically Engineered Pig Kidney", "link": "https://www.wired.com/story/a-third-person-has-received-a-transplant-of-a-genetically-engineered-pig-kidney/", "author": "Emily Mullin"}, {"title": "Muscle Implants Could Allow Mind-Controlled Prosthetics--No Brain Surgery Required", "link": "https://www.wired.com/story/amputees-could-control-prosthetics-with-just-their-thoughts-no-brain-surgery-required-phantom-neuro/", "author": "Emily Mullin"}, {"title": "Combining AI and Crispr Will Be Transformational", "link": "https://www.wired.com/story/combining-ai-and-crispr-will-be-transformational/", "author": "Jennifer Doudna"}, {"title": "Neuralink Plans to Test Whether Its Brain Implant Can Control a Robotic Arm", "link": "https://www.wired.com/story/neuralink-robotic-arm-controlled-by-mind/", "author": "Emily Mullin"}, {"title": "Eight Scientists, a Billion Dollars, and the Moonshot Agency Trying to Make Britain Great Again", "link": "https://www.wired.com/story/aria-moonshot-darpa-uk-britain-great-again/", "author": "Matt Reynolds"}, {"title": "The Atlas Robot Is Dead. Long Live the Atlas Robot", "link": "https://www.wired.com/story/the-atlas-robot-is-dead-long-live-the-atlas-robot/", "author": "NA"}, {"title": "The Atlas Robot Is Dead. Long Live the Atlas Robot", "link": "https://www.wired.com/story/the-atlas-robot-is-dead-long-live-the-atlas-robot/", "author": "Carlton Reid"}, {"title": "Meet the Next Generation of Doctors--and Their Surgical Robots", "link": "https://www.wired.com/story/next-generation-doctors-surgical-robots/", "author": "Neha Mukherjee"}, {"title": "AI Is Building Highly Effective Antibodies That Humans Can’t Even Imagine", "link": "https://www.wired.com/story/labgenius-antibody-factory-machine-learning/", "author": "Amit Katwala"}, {"title": "An Uncertain Future Requires Uncertain Prediction Skills", "link": "https://www.wired.com/story/embrace-uncertainty-forecasting-prediction-skills/", "author": "David Spiegelhalter"}, {"title": "These Rats Learned to Drive--and They Love It", "link": "https://www.wired.com/story/these-rats-learned-to-drive-and-they-love-it/", "author": "Kelly Lambert"}, {"title": "Scientists Are Unlocking the Secrets of Your ‗Little Brain’", "link": "https://www.wired.com/story/cerebellum-brain-movement-feelings/", "author": "R Douglas Fields"}, {"title": "Meet the Designer Behind Neuralink’s Surgical Robot", "link": "https://www.wired.com/story/designer-behind-neuralinks-surgical-robot-afshin-mehin/", "author": "Emily Mullin"}, {"title": "Antibodies Could Soon Help Slow the Aging Process", "link": "https://www.wired.com/story/antibodies-could-soon-help-slow-the-aging-process/", "author": "Andrew Steele"}, {"title": "Good at Reading? Your Brain May Be Structured Differently", "link": "https://www.wired.com/story/good-at-reading-your-brain-may-be-structured-differently/", "author": "Mikael Roll"}, {"title": "Mega-Farms Are Driving the Threat of Bird Flu", "link": "https://www.wired.com/story/mega-farms-are-driving-the-threat-of-bird-flu/", "author": "Georgina Gustin"}, {"title": "RFK Plans to Take on Big Pharma. It’s Easier Said Than Done", "link": "https://www.wired.com/story/rfks-plan-to-take-on-big-pharma/", "author": "Emily Mullin"}, {"title": "Designer Babies Are Teenagers Now--and Some of Them Need Therapy Because of It", "link": "https://www.wired.com/story/your-next-job-designer-baby-therapist/", "author": "Emi Nietfeld"}, {"title": "US Meat, Milk Prices Should Spike if Donald Trump Carries Out Mass Deportation Schemes", "link": "https://www.wired.com/story/us-meat-milk-prices-should-spike-if-donald-trump-carries-out-mass-deportation-schemes/", "author": "Matt Reynolds"}, {"title": "An Augmented Reality Program Can Help Patients Overcome Parkinson’s Symptoms", "link": "https://www.wired.com/story/lining-up-tech-to-help-banish-tremors-strolll-parkinsons/", "author": "Grace Browne"}, {"title": "Meet the Plant Hacker Creating Flowers Never Seen (or Smelled) Before", "link": "https://www.wired.com/story/meet-the-plant-hacker-creating-flowers-never-seen-or-smelled-before/", "author": "Matt Reynolds"}, {"title": "A Mysterious Respiratory Disease Has the Democratic Republic of the Congo on High Alert", "link": "https://www.wired.com/story/drc-mysterious-respiratory-disease-children-who-africa/", "author": "Marta Musso"}, {"title": "Skip the Sea Kelp Supplements", "link": "https://www.wired.com/story/pass-on-sea-kelp-supplements/", "author": "Boutayna Chokrane"}, {"title": "Why Soccer Players Are Training in the Dark", "link": "https://www.wired.com/story/why-soccer-players-are-training-in-the-dark-okkulo-football-sunderland-leeds-united-neuroscience/", "author": "RM Clark"}, {"title": "A Parasite That Eats Cattle Alive Is Creeping North Toward the US", "link": "https://www.wired.com/story/a-parasite-that-eats-cattle-alive-is-creeping-north-toward-the-us/", "author": "Geraldine Castro"}, {"title": "Lasers Are Making It Easier to Find Buried Land Mines", "link": "https://www.wired.com/story/this-laser-system-can-locate-landmines-with-high-accuracy/", "author": "Ritsuko Kawai"}, {"title": "Mark Cuban’s War on Drug Prices: ‖How Much Fucking Money Do I Need?‗", "link": "https://www.wired.com/story/big-interview-mark-cuban-2024/", "author": "Marah Eakin"}, {"title": "Can Artificial Rain, Drones, or Satellites Clean Toxic Air?", "link": "https://www.wired.com/story/artificial-rain-drones-and-satellites-can-tech-clean-indias-toxic-air/", "author": "Arunima Kar"}, {"title": "These Stem Cell Treatments Are Worth Millions. Donors Get Paid $200", "link": "https://www.wired.com/story/stem-cells-cost-rich-16500-donors-get-paid-200-cellcolabs-sweden/", "author": "Matt Reynolds"}, {"title": "The Mystery of How Supermassive Black Holes Merge", "link": "https://www.wired.com/story/how-do-merging-supermassive-black-holes-pass-the-final-parsec/", "author": "Jonathan O’Callaghan"}, {"title": "The $60 Billion Potential Hiding in Your Discarded Gadgets", "link": "https://www.wired.com/story/a-dollar60-billion-a-year-climate-solution-is-sitting-in-our-junk-drawers/", "author": "Vince Beiser"}, {"title": "Tune In to the Healing Powers of a Decent Playlist", "link": "https://www.wired.com/story/music-therapy-health-care/", "author": "Daniel Levitin"}, {"title": "Returning the Amazon Rainforest to Its True Caretakers", "link": "https://www.wired.com/story/amazon-rainforest-indigenous-peoples-justice-stewardship/", "author": "Nemonte Nenquimo and Mitch Anderson"}]}

[87]

Print the response

[90]

Science News:
{
  "news": [
    {
      "title": "December Wildfires Are Now a Thing",
      "link": "https://www.wired.com/story/december-wildfires-are-now-a-thing/",
      "author": "Kylie Mohr"
    },
    {
      "title": "How to Manage Food Anxiety Over the Holidays",
      "link": "https://www.wired.com/story/how-to-cope-with-food-anxiety-during-the-festive-season/",
      "author": "Alison Fixsen"
    },
    {
      "title": "A Spacecraft Is About to Fly Into the Sun\u2019s Atmosphere for the First Time",
      "link": "https://www.wired.com/story/parker-solar-probe-atmosphere/",
      "author": "Eric Berger, Ars Technica"
    },
    {
      "title": "To Improve Your Gut Microbiome, Spend More Time in Nature",
      "link": "https://www.wired.com/story/to-improve-your-gut-microbiome-spend-more-time-in-nature-kathy-willis/",
      "author": "Kathy Willis"
    },
    {
      "title": "This Tropical Virus Is Spreading Out of the Amazon to the US and Europe",
      "link": "https://www.wired.com/story/this-tropical-virus-is-spreading-out-of-the-amazon-to-the-us-and-europe/",
      "author": "Geraldine Castro"
    },
    {
      "title": "CDC Confirms First US Case of Severe Bird Flu",
      "link": "https://www.wired.com/story/cdc-confirms-first-us-case-of-severe-bird-flu/",
      "author": "Emily Mullin"
    },
    {
      "title": "The Study That Called Out Black Plastic Utensils Had a Major Math Error",
      "link": "https://www.wired.com/story/black-plastic-utensils-study-math-error-correction/",
      "author": "Beth Mole, Ars Technica"
    },
    {
      "title": "How Christmas Trees Could Become a Source of Low-Carbon Protein",
      "link": "https://www.wired.com/story/how-christmas-trees-could-become-a-source-of-low-carbon-protein/",
      "author": "Alexa Phillips"
    },
    {
      "title": "Creating a Global Package to Solve the Problem of Plastics",
      "link": "https://www.wired.com/story/global-plastics-treaty-united-nations/",
      "author": "Susan Solomon"
    },
    {
      "title": "These 3 Things Are Standing in the Way of a Global Plastics Treaty",
      "link": "https://www.wired.com/story/these-3-things-are-standing-in-the-way-of-a-global-plastics-treaty/",
      "author": "Steve Fletcher and Samuel Winton"
    },
    {
      "title": "Environmental Sensing Is Here, Tracking Everything from Forest Fires to Threatened Species",
      "link": "https://www.wired.com/story/environmental-sensing-is-here-tracking-everything-from-forest-fires-to-threatened-species/",
      "author": "Sabrina Weiss"
    },
    {
      "title": "Generative AI and Climate Change Are on a Collision Course",
      "link": "https://www.wired.com/story/true-cost-generative-ai-data-centers-energy/",
      "author": "Sasha Luccioni"
    },
    {
      "title": "Climate Change Is Destroying Monarch Butterflies\u2019 Winter Habitat",
      "link": "https://www.wired.com/story/global-warming-threatens-the-monarch-butterfly-sanctuary-but-this-scientist-prepares-a-new-home-for-them/",
      "author": "Andrea J. Arratibel"
    },
    {
      "title": "More Humanitarian Organizations Will Harness AI\u2019s Potential",
      "link": "https://www.wired.com/story/humanitarian-organizations-artificial-intelligence/",
      "author": "David Miliband"
    },
    {
      "title": "Chocolate Has a Sustainability Problem. Science Thinks It's Found the Answer",
      "link": "https://www.wired.com/story/chocolate-has-a-sustainability-problem-science-thinks-its-found-the-answer/",
      "author": "Eve Thomas"
    },
    {
      "title": "We\u2019ve Never Been Closer to Finding Life Outside Our Solar System",
      "link": "https://www.wired.com/story/james-webb-space-telescope-signs-of-life/",
      "author": "Lisa Kaltenegger"
    },
    {
      "title": "The End Is Near for NASA\u2019s Voyager Probes",
      "link": "https://www.wired.com/story/the-end-is-near-for-nasas-voyager-probes/",
      "author": "Luca Nardi"
    },
    {
      "title": "Why Can\u2019t You Switch Seats in an Empty Airplane?",
      "link": "https://www.wired.com/story/why-cant-you-switch-seats-in-an-empty-airplane/",
      "author": "Rhett Allain"
    },
    {
      "title": "The Simple Math Behind Public Key Cryptography",
      "link": "https://www.wired.com/story/how-public-key-cryptography-really-works-using-only-simple-math/",
      "author": "John Pavlus"
    },
    {
      "title": "Everyone Is Capable of Mathematical Thinking--Yes, Even You",
      "link": "https://www.wired.com/story/everyone-is-capable-of-mathematical-thinking-yes-even-you/",
      "author": "Kelsey Houston-Edwards"
    },
    {
      "title": "The Physics of the Macy\u2019s Thanksgiving Day Parade Balloons",
      "link": "https://www.wired.com/story/the-physics-of-the-macys-thanksgiving-day-parade-balloons/",
      "author": "Rhett Allain"
    },
    {
      "title": "A Third Person Has Received a Transplant of a Genetically Engineered Pig Kidney",
      "link": "https://www.wired.com/story/a-third-person-has-received-a-transplant-of-a-genetically-engineered-pig-kidney/",
      "author": "Emily Mullin"
    },
    {
      "title": "Muscle Implants Could Allow Mind-Controlled Prosthetics--No Brain Surgery Required",
      "link": "https://www.wired.com/story/amputees-could-control-prosthetics-with-just-their-thoughts-no-brain-surgery-required-phantom-neuro/",
      "author": "Emily Mullin"
    },
    {
      "title": "Combining AI and Crispr Will Be Transformational",
      "link": "https://www.wired.com/story/combining-ai-and-crispr-will-be-transformational/",
      "author": "Jennifer Doudna"
    },
    {
      "title": "Neuralink Plans to Test Whether Its Brain Implant Can Control a Robotic Arm",
      "link": "https://www.wired.com/story/neuralink-robotic-arm-controlled-by-mind/",
      "author": "Emily Mullin"
    },
    {
      "title": "Eight Scientists, a Billion Dollars, and the Moonshot Agency Trying to Make Britain Great Again",
      "link": "https://www.wired.com/story/aria-moonshot-darpa-uk-britain-great-again/",
      "author": "Matt Reynolds"
    },
    {
      "title": "The Atlas Robot Is Dead. Long Live the Atlas Robot",
      "link": "https://www.wired.com/story/the-atlas-robot-is-dead-long-live-the-atlas-robot/",
      "author": "NA"
    },
    {
      "title": "The Atlas Robot Is Dead. Long Live the Atlas Robot",
      "link": "https://www.wired.com/story/the-atlas-robot-is-dead-long-live-the-atlas-robot/",
      "author": "Carlton Reid"
    },
    {
      "title": "Meet the Next Generation of Doctors--and Their Surgical Robots",
      "link": "https://www.wired.com/story/next-generation-doctors-surgical-robots/",
      "author": "Neha Mukherjee"
    },
    {
      "title": "AI Is Building Highly Effective Antibodies That Humans Can\u2019t Even Imagine",
      "link": "https://www.wired.com/story/labgenius-antibody-factory-machine-learning/",
      "author": "Amit Katwala"
    },
    {
      "title": "An Uncertain Future Requires Uncertain Prediction Skills",
      "link": "https://www.wired.com/story/embrace-uncertainty-forecasting-prediction-skills/",
      "author": "David Spiegelhalter"
    },
    {
      "title": "These Rats Learned to Drive--and They Love It",
      "link": "https://www.wired.com/story/these-rats-learned-to-drive-and-they-love-it/",
      "author": "Kelly Lambert"
    },
    {
      "title": "Scientists Are Unlocking the Secrets of Your \u2017Little Brain\u2019",
      "link": "https://www.wired.com/story/cerebellum-brain-movement-feelings/",
      "author": "R Douglas Fields"
    },
    {
      "title": "Meet the Designer Behind Neuralink\u2019s Surgical Robot",
      "link": "https://www.wired.com/story/designer-behind-neuralinks-surgical-robot-afshin-mehin/",
      "author": "Emily Mullin"
    },
    {
      "title": "Antibodies Could Soon Help Slow the Aging Process",
      "link": "https://www.wired.com/story/antibodies-could-soon-help-slow-the-aging-process/",
      "author": "Andrew Steele"
    },
    {
      "title": "Good at Reading? Your Brain May Be Structured Differently",
      "link": "https://www.wired.com/story/good-at-reading-your-brain-may-be-structured-differently/",
      "author": "Mikael Roll"
    },
    {
      "title": "Mega-Farms Are Driving the Threat of Bird Flu",
      "link": "https://www.wired.com/story/mega-farms-are-driving-the-threat-of-bird-flu/",
      "author": "Georgina Gustin"
    },
    {
      "title": "RFK Plans to Take on Big Pharma. It\u2019s Easier Said Than Done",
      "link": "https://www.wired.com/story/rfks-plan-to-take-on-big-pharma/",
      "author": "Emily Mullin"
    },
    {
      "title": "Designer Babies Are Teenagers Now--and Some of Them Need Therapy Because of It",
      "link": "https://www.wired.com/story/your-next-job-designer-baby-therapist/",
      "author": "Emi Nietfeld"
    },
    {
      "title": "US Meat, Milk Prices Should Spike if Donald Trump Carries Out Mass Deportation Schemes",
      "link": "https://www.wired.com/story/us-meat-milk-prices-should-spike-if-donald-trump-carries-out-mass-deportation-schemes/",
      "author": "Matt Reynolds"
    },
    {
      "title": "An Augmented Reality Program Can Help Patients Overcome Parkinson\u2019s Symptoms",
      "link": "https://www.wired.com/story/lining-up-tech-to-help-banish-tremors-strolll-parkinsons/",
      "author": "Grace Browne"
    },
    {
      "title": "Meet the Plant Hacker Creating Flowers Never Seen (or Smelled) Before",
      "link": "https://www.wired.com/story/meet-the-plant-hacker-creating-flowers-never-seen-or-smelled-before/",
      "author": "Matt Reynolds"
    },
    {
      "title": "A Mysterious Respiratory Disease Has the Democratic Republic of the Congo on High Alert",
      "link": "https://www.wired.com/story/drc-mysterious-respiratory-disease-children-who-africa/",
      "author": "Marta Musso"
    },
    {
      "title": "Skip the Sea Kelp Supplements",
      "link": "https://www.wired.com/story/pass-on-sea-kelp-supplements/",
      "author": "Boutayna Chokrane"
    },
    {
      "title": "Why Soccer Players Are Training in the Dark",
      "link": "https://www.wired.com/story/why-soccer-players-are-training-in-the-dark-okkulo-football-sunderland-leeds-united-neuroscience/",
      "author": "RM Clark"
    },
    {
      "title": "A Parasite That Eats Cattle Alive Is Creeping North Toward the US",
      "link": "https://www.wired.com/story/a-parasite-that-eats-cattle-alive-is-creeping-north-toward-the-us/",
      "author": "Geraldine Castro"
    },
    {
      "title": "Lasers Are Making It Easier to Find Buried Land Mines",
      "link": "https://www.wired.com/story/this-laser-system-can-locate-landmines-with-high-accuracy/",
      "author": "Ritsuko Kawai"
    },
    {
      "title": "Mark Cuban\u2019s War on Drug Prices: \u2016How Much Fucking Money Do I Need?\u2017",
      "link": "https://www.wired.com/story/big-interview-mark-cuban-2024/",
      "author": "Marah Eakin"
    },
    {
      "title": "Can Artificial Rain, Drones, or Satellites Clean Toxic Air?",
      "link": "https://www.wired.com/story/artificial-rain-drones-and-satellites-can-tech-clean-indias-toxic-air/",
      "author": "Arunima Kar"
    },
    {
      "title": "These Stem Cell Treatments Are Worth Millions. Donors Get Paid $200",
      "link": "https://www.wired.com/story/stem-cells-cost-rich-16500-donors-get-paid-200-cellcolabs-sweden/",
      "author": "Matt Reynolds"
    },
    {
      "title": "The Mystery of How Supermassive Black Holes Merge",
      "link": "https://www.wired.com/story/how-do-merging-supermassive-black-holes-pass-the-final-parsec/",
      "author": "Jonathan O\u2019Callaghan"
    },
    {
      "title": "The $60 Billion Potential Hiding in Your Discarded Gadgets",
      "link": "https://www.wired.com/story/a-dollar60-billion-a-year-climate-solution-is-sitting-in-our-junk-drawers/",
      "author": "Vince Beiser"
    },
    {
      "title": "Tune In to the Healing Powers of a Decent Playlist",
      "link": "https://www.wired.com/story/music-therapy-health-care/",
      "author": "Daniel Levitin"
    },
    {
      "title": "Returning the Amazon Rainforest to Its True Caretakers",
      "link": "https://www.wired.com/story/amazon-rainforest-indigenous-peoples-justice-stewardship/",
      "author": "Nemonte Nenquimo and Mitch Anderson"
    }
  ]
}

💾 Save the output to a `CSV` file

Let's create a pandas dataframe and show the table with the extracted content

[91]

Save it to CSV

[92]

Data saved to wired_news.csv

🔗 Resources

🚀 Get your API Key: ScrapeGraphAI Dashboard
🐙 GitHub: ScrapeGraphAI GitHub
💼 LinkedIn: ScrapeGraphAI LinkedIn
🐦 Twitter: ScrapeGraphAI Twitter
💬 Discord: Join our Discord Community
🦜 Langchain: ScrapeGraph docs

Made with ❤️ by the ScrapeGraphAI Team

Scrapegraph Langgraph

🕷️ Extract Wired Science News with langchain-scrapegraph and langgraph

🔧 Install dependencies

🔑 Import ScrapeGraph and OpenAI API keys

📝 Defining an Output Schema for Webpage Content Extraction

🚀 Initialize langchain-scrapegraph tools and langgraph prebuilt agent and run the extraction

💾 Save the output to a CSV file