← All posts

How to Scrape Shopify Store Products Without Parsing HTML

GUIDES · JUL 19, 2026 · 5 MIN READ

Quick answer: append /products.json to almost any Shopify store's domain and you get the product catalog as structured JSON: titles, variants, prices, SKUs, stock flags, images. No HTML parsing, no CSS selectors, no headless browser. Paginate with ?limit=250&page=N until you get an empty array, and you have the whole catalog in a few dozen requests.

People build elaborate scrapers for Shopify stores (browser automation, selector maintenance, the works) without realizing the data is already sitting there in JSON form. Let's do this the easy way first, then handle the stores where the easy way is turned off.

The Endpoint Shopify Never Advertises

Every Shopify storefront ships with a set of public JSON endpoints that power storefront features. The big one:

https://examplestore.com/products.json

Open that in a browser on nearly any Shopify store and you'll see something like:

{
  "products": [
    {
      "id": 7654321098765,
      "title": "Trail Runner 2.0",
      "handle": "trail-runner-2",
      "vendor": "Example Co",
      "product_type": "Shoes",
      "tags": ["running", "outdoor"],
      "variants": [
        {
          "id": 43210987654321,
          "title": "US 9 / Blue",
          "sku": "TR2-9-BLU",
          "price": "129.00",
          "compare_at_price": "159.00",
          "available": true
        }
      ],
      "images": [{ "src": "https://cdn.shopify.com/..." }]
    }
  ]
}

That's everything you'd painfully extract from HTML, already structured. A few field notes worth knowing before you write code:

  • Prices are strings, in the store's base currency. Cast them before doing math.
  • available is a boolean, not an inventory count. Shopify doesn't expose quantities here.
  • compare_at_price is the "was" price. When it's set and higher than price, the item is on sale, which makes this endpoint great for discount monitoring.
  • body_html contains the product description as HTML. It's the one field where markup sneaks back in.
  • Only products published to the online store channel appear. Draft and hidden products don't.

Not sure a site runs Shopify? View source and look for cdn.shopify.com asset URLs, or just try the endpoint. A 200 with a products array answers the question.

Pulling the Whole Catalog

The endpoint returns 30 products by default and caps at 250 per page. Pagination is old-school page numbers, and the loop terminates when a page comes back empty:

import requests, time

base = "https://examplestore.com"
products = []
page = 1

while True:
    r = requests.get(
        f"{base}/products.json",
        params={"limit": 250, "page": page},
        headers={"User-Agent": "catalog-research/1.0"},
        timeout=30,
    )
    r.raise_for_status()
    batch = r.json()["products"]
    if not batch:
        break
    products.extend(batch)
    page += 1
    time.sleep(1)

print(f"{len(products)} products")

A 2,000-product store is eight requests. Compare that to rendering 2,000 product pages in a headless browser and you can see why this should always be your first move.

Since pricing lives on variants, not products, flatten before analysis:

rows = []
for p in products:
    for v in p["variants"]:
        rows.append({
            "product": p["title"],
            "vendor": p["vendor"],
            "sku": v["sku"],
            "variant": v["title"],
            "price": float(v["price"]),
            "on_sale": bool(v.get("compare_at_price"))
                and float(v["compare_at_price"]) > float(v["price"]),
            "available": v["available"],
        })

Two more URLs round out the toolkit. A single product by handle:

https://examplestore.com/products/trail-runner-2.json

And a specific collection, useful when you only care about one category:

https://examplestore.com/collections/sale/products.json

Be Polite or Meet HTTP 430

Shopify fronts every store, so aggressive clients get platform-level pushback. Hammer the endpoint and you'll start seeing HTTP 430, a nonstandard status Shopify uses for security rejections, or plain 429s. In my experience one request per second with a descriptive User-Agent runs all day without trouble; there's no prize for finishing a 12-request job in 300 milliseconds.

If you're doing recurring runs (say, daily price tracking), store each day's flattened rows with a timestamp and diff values, not pages. We walk through that exact pattern in monitoring competitor pricing changes, and it applies directly here, except Shopify hands you the structured data for free.

When /products.json Is Disabled or Blocked

Now the catch. The endpoint is on by default, but it's not guaranteed:

Situation What you'll see
Store edited theme/app settings to disable it 404 or empty products array
Password-protected or pre-launch store Redirect to a password page
Bot protection in front of the store 403, a challenge page, or HTTP 430 on your first request
Checkout-only headless setups Storefront exists but JSON endpoints don't behave normally

Datacenter IPs are the most common trigger for that third row: the code works from your laptop, then fails from your server. That's an IP reputation problem, not a code problem, and it's the same story we cover in scraping Cloudflare-protected sites.

The fix is to stop fetching from infrastructure that looks like a bot farm. The link.sc fetch API routes the request through infrastructure built to not get blocked, and since /products.json is just a URL, you can fetch it like any other:

curl -X POST https://api.link.sc/v1/fetch \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://examplestore.com/products.json?limit=250&page=1"}'

You get the same JSON back, minus the 403s.

For stores that have genuinely disabled the endpoint, you fall back to the product pages themselves, but you still don't have to write selectors. Pass a JSON schema and let extraction do the work:

curl -X POST https://api.link.sc/v1/fetch \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://examplestore.com/products/trail-runner-2",
    "format": "json",
    "schema": {
      "type": "object",
      "properties": {
        "title": {"type": "string"},
        "price": {"type": "number"},
        "in_stock": {"type": "boolean"}
      }
    }
  }'

Find the product URLs to feed it from the store's sitemap: Shopify publishes every product in sitemap_products_1.xml, linked from /sitemap.xml. So even in the worst case, the pipeline is "read sitemap, fetch pages, extract to schema," and your code never touches the DOM.

A Note on Playing Fair

The endpoint being public doesn't make everything you might do with the data fine. You're reading data the storefront already serves to every visitor, which puts you on solid ground for price comparison and market research, but terms of service and data usage still matter for what happens downstream. The nuances are in is web scraping legal, and the broader "should I even be scraping this" question gets a full treatment in web scraping vs. APIs.

The short version of this whole post: check for the JSON endpoint first, paginate politely, and reach for a fetch API only when a store makes you. Most of the time, Shopify already built your scraper for you.

Need Shopify data from stores that block your servers? Sign up for link.sc and get 500 free credits a month.