Quick answer: Headless browser testing means running your end-to-end tests in a real browser with no visible window, which is how they run in CI. Use Playwright, rely on its auto-waiting instead of sleeps, run headless in CI and headed locally when debugging, and treat flaky tests as bugs in your waits, not as weather. The setup is about 20 lines of config.
I've written and babysat a lot of browser test suites, and the difference between a suite the team trusts and one they retry-until-green comes down to a handful of habits. This is the guide I wish someone had handed me.
What headless testing is (and how it differs from scraping)
Quick scoping note, since this blog talks about scraping a lot. Testing and scraping both drive headless browsers, but the posture is opposite. In testing, you own the site, you want deterministic behavior, and nothing is trying to detect you. In scraping, you don't own the site and the interesting problems are rendering someone else's JavaScript reliably (that side is covered in curl vs headless vs stealth browser, and it's the problem link.sc exists to solve). Everything below is the testing side: no proxies, no fingerprints, just making your own app's tests fast and trustworthy.
Headless matters for testing for one dominant reason: CI runners have no display. Your tests must run without a GUI, so headless is the default execution mode everywhere that matters.
A real Playwright setup
Playwright is my recommendation for new suites. Install and scaffold:
npm init playwright@latest
A sensible playwright.config.ts:
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './e2e',
fullyParallel: true,
retries: process.env.CI ? 2 : 0,
workers: process.env.CI ? 4 : undefined,
reporter: process.env.CI ? [['html'], ['github']] : 'list',
use: {
baseURL: 'http://localhost:3000',
trace: 'on-first-retry',
screenshot: 'only-on-failure',
},
projects: [
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
],
webServer: {
command: 'npm run start',
url: 'http://localhost:3000',
reuseExistingServer: !process.env.CI,
},
});
And a test that shows the style I push teams toward:
import { test, expect } from '@playwright/test';
test('user can search and see results', async ({ page }) => {
await page.goto('/');
await page.getByRole('searchbox', { name: 'Search' }).fill('widgets');
await page.getByRole('button', { name: 'Search' }).click();
await expect(page.getByRole('list', { name: 'Results' }))
.toContainText('Widget Pro');
});
Two things to notice. First, selectors target roles and accessible names, not CSS classes, so the tests survive a redesign. Second, there is no explicit waiting anywhere. That's the next section.
Flakiness is almost always a waiting bug
The classic flaky test looks like this:
// Don't do this
await page.click('#submit');
await page.waitForTimeout(3000);
expect(await page.textContent('.status')).toBe('Saved');
That waitForTimeout(3000) fails when the server takes 3.1 seconds and wastes 2.9 seconds when it takes 100ms. Both outcomes are bad. Playwright's web-first assertions poll until the condition holds or a timeout expires:
// Do this
await page.getByRole('button', { name: 'Submit' }).click();
await expect(page.getByText('Saved')).toBeVisible();
My rules for de-flaking, in priority order:
- Delete every
waitForTimeout. Each one is a race condition with a schedule. - Assert on outcomes, not intermediate states. Wait for the result the user would see, not for a spinner to disappear.
- Control your test data. Shared mutable state between parallel tests is the second-biggest flake source after bad waits. Give each test its own user or namespace.
- Mock the network you don't own. Third-party calls (payments, analytics, external APIs) get stubbed with
page.route(). Your E2E suite should test your app, not Stripe's uptime. If your app fetches external web content at runtime through something like the link.sc fetch API, stub that route too and assert on how your app handles the response shape. - Use trace on retry. Playwright's trace viewer replays the failure with DOM snapshots and network logs. It turns "works on my machine" into a five-minute diagnosis.
Retries in CI are a pressure valve, not a fix. If a test needed a retry, it gets a ticket.
CI setup that actually works
GitHub Actions example, which transfers to any CI:
name: e2e
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20 }
- run: npm ci
- run: npx playwright install --with-deps chromium
- run: npx playwright test
- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-report
path: playwright-report/
The details that save you pain: --with-deps installs the system libraries Chromium needs on a bare runner, uploading the report on failure gives you the trace viewer for red builds, and pinning your Playwright version in package.json keeps browser and library in sync. If installs are slow, Playwright publishes a Docker image with browsers baked in.
Budget resources honestly. Each browser worker wants roughly a CPU core and 1 to 2 GB of RAM. Four workers on a 2-core runner will thrash and, worse, create timing conditions you'll misdiagnose as test flakiness.
Headless vs headed: when to flip the switch
Run headless by default, everywhere. But headed mode earns its keep in specific moments:
| Situation | Mode |
|---|---|
| CI, always | Headless |
| Local bulk runs | Headless (faster, quieter) |
| Writing a new test | Headed, or npx playwright codegen |
| Debugging a failure | Headed with --debug, or replay the CI trace |
| A test passes headed but fails headless | Investigate viewport size and missing fonts first |
That last row is a real phenomenon. Headless defaults can differ in viewport, device scale factor, and available fonts, which shifts layout enough to hide a button behind an overlay. Pin your viewport in config and the gap mostly disappears. Modern Chrome's headless mode shares the full browser code path, so the old "headless renders differently" folklore is far less true than it was years ago.
Keep the pyramid in mind
E2E browser tests are the most expensive tests you own: slowest to run, costliest to maintain, most prone to environmental noise. They should sit on top of a much larger base of unit and integration tests. My rough guidance: E2E covers the critical user journeys (signup, login, core workflow, checkout), integration tests cover API behavior, and unit tests cover logic. If your E2E suite has 800 tests and takes 90 minutes, that's not thoroughness, that's misallocation.
One boundary worth stating: point your tests at environments you own or are authorized to test. Driving automated browsers against production systems you don't control isn't testing, and load-testing someone else's site without permission is just an attack with extra steps.
Bottom line
Headless browser testing is the same testing you'd do with a visible browser, minus the window, plus CI-friendliness. Choose Playwright, lean on auto-waiting, isolate test data, mock third parties, replay traces instead of guessing, and keep the suite small enough that people believe it when it fails. A trusted red build is the entire point.
Testing is one use of headless browsers; fetching the live web for your app is another. link.sc handles the second one, returning any URL as clean markdown. Start free.