← All posts

curl_cffi vs. Headless Browser vs. Stealth Browser: Which Should You Use?

ENGINEERING · JUL 6, 2026 · 5 MIN READ

curl vs headless vs stealth browser

If you're fetching pages from the web at any scale, you've probably hit the moment where a simple HTTP request stops working. The page loads fine in your browser but comes back empty, or blocked, or full of JavaScript that never runs. So you reach for a headless browser — and now your fetch that took 80ms takes 4 seconds and eats a gigabyte of RAM.

There are really three tiers of tooling here, and the honest answer to "which should I use?" is: the cheapest one that actually works for that specific site. The trick is knowing which is which without hand-tuning every domain. Let's walk through the tiers, the tradeoffs, and why picking wrong is expensive in both directions.

The Three Tiers

Tier 1 — Plain HTTP (curl_cffi). This is a raw HTTP client that sends a request and parses the response. No browser, no JavaScript engine, no rendering. curl_cffi is the standout here because it impersonates a real browser's TLS and JA3 fingerprint (more on that below). It's fast — tens of milliseconds — and cheap enough that you can run thousands concurrently on a modest box.

Tier 2 — Headless / stealth browser. A real browser engine (Chromium, Firefox) driving the page without a visible window. It runs JavaScript, executes SPA frameworks, waits for XHR calls, and gives you the fully-rendered DOM. A stealth browser goes further, patching the automation tells that bot detectors look for — navigator.webdriver, missing plugins, headless-specific quirks in canvas and WebGL. This is heavy: hundreds of megabytes of memory per instance and seconds per page.

Tier 3 — Stealth browser over Tor / residential exit. For the hardest targets — sites behind aggressive Cloudflare or DataDome challenges that fingerprint at every layer — you need a full stealth browser (something like Camoufox, a hardened Firefox) routed through Tor or residential IPs. This is the slowest and most resource-intensive option, but it clears walls the other two can't.

The cost gap between tiers is not subtle. Plain HTTP is roughly 10–100x cheaper and faster than driving a browser, because you're skipping the entire render pipeline. That's the whole reason to care about this: if you send every request through a browser "to be safe," you're paying browser prices for pages that a plain GET would have returned instantly.

Why requests Gets Blocked and curl_cffi Doesn't

This one surprises people. You write clean Python with requests, set a real-looking User-Agent, and still get a 403 or a challenge page. Meanwhile the same URL opens fine in Chrome.

The reason is TLS fingerprinting. When a client opens an HTTPS connection, the TLS handshake — the order of cipher suites, extensions, elliptic curves, and so on — forms a fingerprint (JA3/JA4). Real browsers have distinctive, well-known fingerprints. Python's requests (via OpenSSL) has a completely different one that screams "this is a script." Bot-detection services keep a catalog: they can flag you before you've sent a single HTTP header, no matter how perfect your User-Agent string is.

curl_cffi solves this by using curl compiled against a TLS backend that can mimic Chrome's or Firefox's exact handshake:

from curl_cffi import requests

r = requests.get(
    "https://example.com",
    impersonate="chrome",  # matches a real Chrome TLS/JA3 fingerprint
)
print(r.status_code, len(r.text))

That single impersonate flag gets you past a large class of blocks — with none of the cost of a browser. It's the reason Tier 1 is viable far more often than people assume.

When Each Tier Is Right

  • Use plain HTTP (Tier 1) when the content is in the initial HTML: news articles, blog posts, JSON APIs, most documentation, server-rendered pages. If curl with browser impersonation returns the content you want, stop here. This should be your default.
  • Use a headless/stealth browser (Tier 2) when the content only appears after JavaScript runs — React/Vue SPAs, infinite scroll, data loaded via XHR — or when the site does light bot detection that TLS impersonation alone doesn't clear.
  • Use a stealth browser over Tor (Tier 3) when you're facing serious anti-bot infrastructure: Cloudflare "checking your browser" interstitials, DataDome, PerimeterX. This is a last resort, not a starting point — it's slow, and overusing it wastes resources on sites that never needed it.

The failure mode in both directions is real. Start too low and you get empty pages and silent data loss. Start too high and you burn compute and money on requests that a GET would have handled. Neither is acceptable at scale.

Letting the Escalation Happen Automatically

The tedious part is that the right tier is per-domain, and it changes over time. Nobody wants to maintain a spreadsheet mapping example.com → tier 2. This is exactly what link.sc handles for you.

link.sc escalates cheapest-first. Every fetch starts with plain curl_cffi and browser-fingerprint impersonation. If that comes back blocked or under-rendered, it escalates to a stealth CloakBrowser with full JS rendering. If that still hits a wall, it falls back to a Camoufox browser over Tor. Crucially, it remembers what worked for each domain — so the next request to that site skips straight to the method that succeeds, instead of paying the escalation cost every time.

You send one request and get clean content back, without deciding the tier yourself:

curl -X POST https://api.link.sc/v1/fetch \
  -H "x-api-key: lsc_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

const res = await fetch("https://api.link.sc/v1/fetch", {
  method: "POST",
  headers: {
    "x-api-key": "lsc_YOUR_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ url: "https://example.com" }),
});
const { content } = await res.json();

You get the reliability of the heavy tiers with the cost profile of the light one, because most requests never leave Tier 1 — and the ones that do only escalate as far as they need to.

If you want to try the escalation-and-remember behavior without wiring up three tiers of infrastructure yourself, link.sc gives you 500 free credits a month. Sign up at link.sc/register, or read the docs to see the full fetch options. It's open core, so you can self-host the whole thing if you'd rather run it in-house.