Zyte (formerly Scrapinghub) earned a lot of goodwill by maintaining Scrapy, and the Zyte API is a genuinely capable product. But if you have ever opened your invoice after a month of crawling "tier 5" domains, you know why people go looking for alternatives.
The core complaint is almost never reliability. It is the pricing model: Zyte charges per request, and the per-request cost changes based on how difficult Zyte judges the target website to be. The same spider hitting two different domains can cost wildly different amounts, and the tier of a domain can change without warning. Budget forecasting turns into guesswork.
So let's look at what else is out there, what each option actually costs you, and how painful the migration is from a Scrapy codebase.
Why People Leave Zyte in the First Place
Three reasons come up over and over:
Unpredictable spend. Difficulty-based tiers mean a domain that cost $1.50 per thousand requests last quarter can silently move to a tier costing five times that. Your code did not change. Your bill did.
Lock-in through the plugin. The
scrapy-zyte-apiplugin is convenient, but it wires Zyte-specific request metadata (zyte_apikeys inRequest.meta) throughout your spiders. Ripping it out later means touching every spider.Paying browser prices for HTML jobs. A lot of Zyte requests get routed through browser rendering whether you needed it or not, and browser requests cost more. If most of your targets serve usable HTML to a well-configured HTTP client, you are overpaying.
If none of these bother you, staying put is a fine choice. Zyte's ban-handling is good. But if you nodded at any of the three, keep reading.
The Alternatives, Compared
| Service | Pricing model | Scrapy migration effort | JS rendering | Best for |
|---|---|---|---|---|
| link.sc | Flat per successful request | Low (plain HTTP API) | Automatic escalation | LLM pipelines, markdown output, predictable cost |
| ScraperAPI | Credits per request (multipliers for JS/geo) | Low | Optional, costs extra credits | General scraping at volume |
| ScrapingBee | Credits per request (multipliers) | Low | Optional, 5x credit cost | Small to mid-size projects |
| Bright Data | Per GB or per request depending on product | Medium | Yes | Enterprise scale, strict compliance needs |
| Apify | Compute units plus storage | Medium to high | Yes | Actor-based workflows, marketplace scrapers |
| Self-hosted (Scrapy + proxies) | Proxy bandwidth only | None, you already have it | scrapy-playwright | Teams with ops capacity and stable targets |
A few notes the table cannot capture.
link.sc
I work on link.sc, so weigh this section accordingly. The design goal is different from Zyte's: one flat price per successful fetch, no difficulty tiers, and responses come back as clean markdown or raw HTML depending on what you ask for. Anti-bot escalation (plain HTTP, then browser, then stealth) happens on our side and does not change what you pay.
For Scrapy users the integration is deliberately boring. There is no plugin to adopt. You call a REST endpoint:
import scrapy
from urllib.parse import quote
class ProductSpider(scrapy.Spider):
name = "products"
def start_requests(self):
for url in self.target_urls:
yield scrapy.Request(
f"https://api.link.sc/fetch?url={quote(url)}",
headers={"Authorization": f"Bearer {self.settings['LINKSC_KEY']}"},
callback=self.parse,
)
Because it is just a URL rewrite, backing out later is one line. That is intentional. The docs cover response formats, and if your downstream consumer is an LLM rather than an XPath parser, the markdown output saves you a cleaning step. We wrote more about that tradeoff in web scraping vs API.
ScraperAPI and ScrapingBee
These two are the closest like-for-like swaps. Both proxy your request through their infrastructure and return the page. Both use credit multipliers instead of domain tiers: JS rendering costs more credits, premium proxies cost more credits, but the multiplier is declared up front and you opt into it per request.
That is the meaningful difference from Zyte. You choose when to pay the premium instead of the vendor's difficulty classifier choosing for you. The downside: when a site does need the premium treatment and you did not enable it, you eat the failed requests and retry with the flag on.
Bright Data
Bright Data is the heavyweight. Enormous proxy pool, strong unblocking, serious compliance program. It is also the most complex to buy: separate products for proxies, the Web Unlocker, and the Scraping Browser, each priced differently. In my experience, teams below roughly seven figures of annual scraping spend end up paying for capability they never touch. If you are leaving Zyte because of billing complexity, be honest with yourself about whether this fixes that.
Apify
Apify is a different shape of product: you deploy scrapers ("actors") onto their platform and pay for compute. If your team lives in Scrapy, moving to Apify means adopting their runtime model, not just swapping a downloader. It shines when you want the marketplace, prebuilt scrapers for common sites, rather than running your own spiders.
Self-Hosting: The Option Zyte Hopes You Forget
Here is the thing about the Scrapy crowd specifically: you already have the hard part. Your spiders, your parsers, your item pipelines all work. What Zyte sells you is the fetching layer, and that layer can be rebuilt with residential proxies plus a TLS-fingerprint-aware client.
A stack of Scrapy, scrapy-playwright for the JavaScript-heavy minority of targets, and a residential proxy provider handles a surprising share of real-world targets. We ran the numbers on fingerprinting in our curl_cffi guide: a large fraction of "blocks" are IP reputation and TLS fingerprint checks, both solvable without a paid unblocker.
The honest caveat: self-hosting trades money for engineering time. Proxy pools degrade, Cloudflare changes tactics (see how to scrape Cloudflare-protected sites), and someone on your team owns that pager. If your target list is long-tail and stable, self-hosting wins. If you hit a rotating set of hostile domains, a managed service earns its fee.
How to Actually Migrate a Scrapy Project
Whatever you pick, do the migration in this order:
- Inventory your
zyte_apimeta usage.grep -r "zyte_api" spiders/tells you how deep the coupling goes. - Classify your targets. Run a sample through plain Scrapy with good headers. Anything that returns usable HTML does not need a paid unblocker at all.
- Route only the hard domains through the new provider. A custom downloader middleware that rewrites requests for a domain allowlist keeps costs down and keeps the swap reversible.
- Run both providers in parallel for a week. Compare success rates per domain before you cancel anything.
Step 2 is the one people skip, and it is the one that saves the most money. In most crawls I have seen, fewer than a third of domains actually need premium unblocking. Paying tier prices for the other two thirds is the real Zyte tax.
The Bottom Line
If you want the smallest possible change: ScraperAPI or ScrapingBee, with multipliers you control. If you want flat, predictable pricing and LLM-ready output: link.sc. If you are an enterprise with a compliance team: Bright Data. If you have ops capacity and stable targets: self-host and keep the money.
The only wrong move is staying on difficulty-tier pricing out of inertia while your invoice quietly climbs.
Get a flat-rate fetch API with clean markdown output: create a free link.sc account and run your first request in under a minute.