Quick answer: A headless browser is a real web browser running without a visible window. It loads pages, executes JavaScript, builds the DOM, and fires network requests exactly like Chrome on your desktop, but it's controlled by code instead of a mouse. You use one when you need a page's fully rendered result (for scraping JavaScript-heavy sites, automated testing, screenshots, or PDFs) and no human is there to watch.
The name throws people off. "Headless" just means "no head," where the head is the graphical interface. Everything else about the browser is intact. Let me unpack what that actually means and, just as important, when you don't need one.
Headless vs headed: what's actually different
A headed browser paints pixels to a screen and waits for a human. A headless browser skips the painting-to-a-screen part and exposes a programmatic control channel instead. Under the hood it's the same engine: same HTML parser, same JavaScript engine, same networking stack.
| Headed browser | Headless browser | |
|---|---|---|
| Renders pages | Yes, to your screen | Yes, to an off-screen buffer |
| Runs JavaScript | Yes | Yes |
| Controlled by | Human input | Code (via a protocol like CDP or WebDriver) |
| GUI overhead | Full window, GPU compositing to display | None visible; can still screenshot |
| Typical home | Your laptop | A server or CI runner |
The practical consequence: a headless browser can run on a Linux server with no display attached, at scale, in parallel. That's the whole point.
How it works, briefly
When you launch headless Chrome through a library like Playwright, three things happen. Your script starts a real Chromium process with a flag telling it not to open a window. The library connects to it over the Chrome DevTools Protocol, a websocket channel the browser exposes for automation. Then your code issues commands: navigate here, wait for this selector, click that button, give me the HTML.
The browser does everything a normal page load involves: fetching resources, executing scripts, running the event loop, hydrating your React app. When you ask for the page content, you get the result after JavaScript has done its work. That's the crucial difference from a plain HTTP client like curl, which only ever sees the raw HTML the server sent.
The main tools
Three libraries dominate, and they're all good:
- Playwright (Microsoft): my default recommendation today. Controls Chromium, Firefox, and WebKit with one API, has excellent auto-waiting, and first-class support in Python, Node, Java, and .NET.
- Puppeteer (Google): the original modern headless library, Node-focused, Chrome-centric. Mature and widely deployed.
- Selenium: the veteran. Broadest language and browser support via the WebDriver standard, huge ecosystem, but a clunkier API and slower feedback loop than the other two.
A minimal Playwright example:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page()
page.goto("https://example.com/app")
page.wait_for_selector(".results")
print(page.content()) # HTML after JS has rendered
browser.close()
What people use headless browsers for
Scraping JavaScript-rendered sites. If a page arrives as an empty <div id="root"> and fills in via API calls, a plain HTTP fetch gets you nothing. A headless browser executes the app and hands you the finished DOM. Figuring out whether a given site needs this is a skill of its own; I wrote a triage guide in how to tell which scraping method a site needs.
Automated testing. End-to-end tests drive a real browser through signup flows, checkouts, and dashboards in CI, headlessly, because CI machines have no screens.
Screenshots and visual snapshots. Rendering a URL to a PNG for previews, monitoring, or visual regression testing.
PDF generation. Headless Chrome's print-to-PDF is one of the most reliable ways to turn HTML into a decent PDF, which is why half the invoice generators you've used are secretly running Chrome.
The costs nobody mentions in the tutorials
Headless browsers are heavy, and this matters the moment you scale past one page.
A plain HTTP request costs a few milliseconds of CPU and a few kilobytes of memory. A headless browser page load costs hundreds of megabytes of RAM per browser context, several seconds of wall time, and all the bandwidth of every image, font, and script on the page. In rough terms, expect one to two orders of magnitude more resource cost per page than plain HTTP.
There's operational drag too: browser binaries to install and patch, zombie processes to reap, crashes to handle, and version drift between the library and the browser. None of it is hard individually. All of it together is a part-time job.
One more thing: headless does not mean undetectable. Anti-bot systems fingerprint headless browsers specifically, and a default headless Chrome is fairly easy to spot. If your interest in headless browsers is scraping protected sites, read curl vs headless vs stealth browser for where plain headless stops working, and keep it ethical: public pages, respectful rates, robots.txt, and no sneaking past logins or paywalls.
When you don't need one
This is my favorite section, because the most common headless browser mistake is using one at all.
- The data is in the initial HTML. View source (not inspect element, actual view source). If your content is there, plain HTTP wins by 100x on cost.
- The site has a JSON API behind the page. Open the network tab. Many "JavaScript sites" fetch their data from a clean JSON endpoint you can request directly.
- The data is embedded in a script tag. Next.js and friends often ship the full page data as JSON inside
__NEXT_DATA__. No rendering required.
At link.sc we run a per-domain escalation ladder for exactly this reason: plain HTTP first, browser-fingerprint HTTP next, headless only when a domain demonstrably needs rendering, stealth plus residential as the last rung. You make one call and the system picks the cheapest rung that works:
curl https://link.sc/v1/fetch \
-H "Authorization: Bearer lsc_your_key" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/spa-page", "format": "markdown"}'
If you'd rather not run browsers at all, that's the entire pitch of the docs: rendered pages as clean markdown without owning the Chrome fleet.
Bottom line
A headless browser is just a normal browser minus the window, driven by code. It's the right tool when you need executed JavaScript: dynamic scraping, E2E testing, screenshots, PDFs. It's the wrong tool when the data was in the HTML all along. Check that first, and you'll skip a lot of unnecessary infrastructure.
Need rendered pages without running Chrome yourself? link.sc fetches any URL, escalating to headless only when the site requires it. Try it free.