Puppeteer Web Scraping: A Node.js Guide With Runnable Code

Quick answer: Puppeteer is a Node.js library that drives a headless Chrome browser with code. For scraping, you install puppeteer, launch a browser, open a page, navigate with page.goto(url), wait for content with page.waitForSelector, then run page.evaluate to pull data out of the live DOM. It renders JavaScript-heavy pages that plain HTTP requests cannot, and it is great for screenshots and PDFs, but it is heavier and slower than a simple fetch.

Puppeteer is the Chrome team's own browser-automation library for Node. If you live in JavaScript and need to scrape pages that build themselves with client-side code, it is a natural fit: you write your scraper in the same language as the page you are scraping. This guide covers launch, navigation, waiting, extraction, and screenshots, then where Puppeteer sits next to Playwright and a fetch API.

Install Puppeteer

The default package downloads a compatible Chromium for you, so one install gets you a working browser.

npm install puppeteer

If you already have Chrome installed and want to skip the download, use puppeteer-core and point it at your existing binary. For most scrapers, plain puppeteer is the easy path.

Launch a Browser and Open a Page

Every Puppeteer script follows the same skeleton: launch, open a page, do work, close.

import puppeteer from "puppeteer";

const browser = await puppeteer.launch({ headless: "new" });
const page = await browser.newPage();

// ... work happens here ...

await browser.close();

Always close() the browser. A crashed script that skips it leaves Chromium processes running in the background.

Navigate to a Page

page.goto loads a URL. The waitUntil option controls when the promise resolves. For dynamic pages, networkidle2 (waits until the network is mostly quiet) is often more reliable than the default.

await page.goto("https://example.com", {
  waitUntil: "networkidle2",
  timeout: 30000,
});

Wait for Content

The most common scraping bug is reading the DOM before JavaScript has filled it in. Wait for a selector that only exists once the content you want has rendered.

await page.waitForSelector(".product", { timeout: 10000 });

Prefer waiting on a selector over a fixed delay. A hard-coded setTimeout is either too short and flaky or too long and wasteful.

Extract Data With page.evaluate

page.evaluate runs a function inside the page's own browser context, so you have the full DOM API. Whatever you return is serialized back to Node.

const products = await page.evaluate(() => {
  const cards = Array.from(document.querySelectorAll(".product"));
  return cards.map((card) => ({
    name: card.querySelector(".name")?.textContent.trim(),
    price: card.querySelector(".price")?.textContent.trim(),
  }));
});

The key mental model: code inside evaluate runs in the browser, not in Node. You cannot reach your Node variables in there unless you pass them as arguments, and you can only return data that serializes to JSON.

Take a Screenshot or PDF

Screenshots and PDFs are where Puppeteer really earns its keep. Two lines each.

await page.screenshot({ path: "page.png", fullPage: true });
await page.pdf({ path: "page.pdf", format: "A4" });

This is handy for archiving what a page looked like, or for debugging a scraper that returns empty data (the screenshot usually shows a cookie banner or a login wall you did not expect).

A Full Runnable Example

Here is a complete scraper: launch, navigate, wait, extract, and clean up safely.

import puppeteer from "puppeteer";

async function scrape(url) {
  const browser = await puppeteer.launch({ headless: "new" });
  try {
    const page = await browser.newPage();
    await page.goto(url, { waitUntil: "networkidle2", timeout: 30000 });
    await page.waitForSelector(".product", { timeout: 10000 });

    const products = await page.evaluate(() => {
      const cards = Array.from(document.querySelectorAll(".product"));
      return cards.map((card) => ({
        name: card.querySelector(".name")?.textContent.trim(),
        price: card.querySelector(".price")?.textContent.trim(),
      }));
    });

    return products;
  } finally {
    await browser.close();
  }
}

const rows = await scrape("https://example.com/products");
console.log(rows);

The try/finally makes sure the browser closes even when a selector times out. That one habit will keep your servers clean.

When to Use Puppeteer vs Playwright vs a Fetch API

Puppeteer is not always the right choice. Here is how I decide.

Tool	Best for	Watch out for
Puppeteer	Chrome-only scraping, screenshots, PDFs, Node projects	Chromium-focused; cross-browser support is limited
Playwright	Cross-browser (Chromium, Firefox, WebKit), auto-waiting, multi-language	Slightly larger API surface to learn
Fetch API	Reading rendered content without running a browser yourself	Not built for driving long multi-step UI flows

Puppeteer and Playwright are close cousins; Playwright grew out of the same team and added cross-browser support and cleaner auto-waiting. If you are picking between them for a new project, read our head-to-head in Playwright vs Puppeteer.

The bigger question is whether you need a browser at all. Running headless Chrome is heavy: real memory per instance, slower than an HTTP request, and it needs care to run reliably at scale. If your goal is simply to get the rendered content of pages, a hosted fetch API removes the browser entirely. Against link.sc, the same job is one request:

const resp = await fetch("https://api.link.sc/v1/fetch", {
  method: "POST",
  headers: {
    "x-api-key": "lsc_your_key_here",
    "content-type": "application/json",
  },
  body: JSON.stringify({ url: "https://example.com/products", format: "markdown" }),
});

const data = await resp.json();
console.log(data.content); // rendered page as clean markdown

Note the header is x-api-key, not Authorization: Bearer, and the rendered content comes back as { content: "..." }. No browser to launch, no selectors to wait on, no cleanup. See the link.sc docs for options like rendering JavaScript, waiting on a selector, or setting a country. Keep Puppeteer for the jobs that genuinely need browser control, such as multi-step forms, screenshots, and PDFs.

Scrape Responsibly

Because Puppeteer drives a real browser, it is easy to forget you are still a guest on someone else's server. Collect only public data, honor robots.txt and stated rate limits, and do not automate past logins or paywalls. Throttle your requests, avoid pointing a swarm of browsers at one small site, and cache what you have already fetched. Being a browser does not grant permission the site itself has not given. Polite scraping is what keeps your access alive.

The Bottom Line

Puppeteer web scraping is the right call when you are in Node, need Chrome to run JavaScript, or want screenshots and PDFs. Learn the launch-navigate-wait-evaluate loop and always close the browser in a finally. But it is heavy and Chromium-focused, so consider Playwright for cross-browser work and a fetch API when you only need rendered content. Match the tool to the job, and reach for the browser only when you truly need one.

---

Need rendered pages without running Chrome yourself? link.sc renders and cleans any URL in one call. Get started free with 500 credits a month.