You want G2 reviews and ratings for a competitor or a category you sell into. Maybe you are building a win-loss narrative, tracking how a rival's satisfaction score drifts quarter over quarter, or mining "what users dislike" to sharpen your own positioning. You write a script, point it at a G2 product page, and get back a challenge page, a 403, or HTML with no reviews in it at all.
Before the mechanics: this is about collecting what G2 already shows the public, for internal analysis, in a way that respects the reviewers behind the text. It is not about defeating a login, republishing reviews as your own testimonials, or building a mirror of G2's corpus. Get that framing right first, because on a B2B review site the compliance question is the hard part and the scraping is comparatively easy.
Why G2 Is Not a Plain Fetch
G2 sits behind a modern anti-bot stack, and a naive request trips several defenses before it ever sees a review.
- TLS and HTTP/2 fingerprinting. Your client's handshake reveals it is Python or Go well before the server parses a header. Spoofing the User-Agent does nothing, because the User-Agent was never what flagged you.
- A JavaScript-rendered review feed. Review cards, star breakdowns, and pagination load through client-side calls. Fetch the raw HTML and you often get a shell with no review bodies.
- Managed challenge pages. G2 uses interstitial "checking your browser" gates that a bare HTTP client cannot solve.
- IP reputation scoring. Requests from AWS, GCP, or cheap VPS ranges carry low trust and get challenged fast.
The block usually lives a layer below the request you are staring at. I broke down that fingerprinting layer in how to scrape a Cloudflare-protected site, and the same pattern applies here.
Settle the ToS and PII Question First
This is the part most tutorials skip, and on a review site it is the part that can actually hurt you.
G2's terms prohibit automated collection. A ToS breach does not automatically make scraping public data illegal (US case law like hiQ v. LinkedIn leans toward public data being fair game), but it is a real breach that can get your accounts and IPs banned. I covered the nuance in is web scraping legal. "Public" and "permitted" are not the same word.
The sharper issue is personal data. G2 reviews carry reviewer names, job titles, company size, industry, and sometimes a LinkedIn-verified identity. That is regulated personal data under GDPR the moment you store it, and CCPA in California. A few rules I hold to:
- Extract the signal, not the person. Store the rating, the sentiment, the product, and the date. Do not build a table keyed on reviewer identity.
- Aggregate for competitive intelligence. "40 percent of recent detractors cite onboarding" is useful and low-risk. A named quote attached to a named individual is neither.
- Never scrape behind a gate. If G2 asks for an account to read more, that content is off the table.
- Do not republish review text verbatim as your own marketing. Summarize trends instead.
Get this right and the rest is engineering. Get it wrong and no amount of clever proxying saves you.
The Review Schema Worth Extracting
Competitive intelligence gets easier when you decide up front what a clean review record looks like, then extract only those fields. A tight schema keeps you out of the PII weeds and makes the data trivial to analyze.
| Field | Why it matters | PII risk |
|---|---|---|
| Overall rating (1 to 5) | Core satisfaction signal | None |
| Review date | Trend and recency weighting | None |
| Product and category | Segmentation | None |
| "What do you like" text | Strength themes | Low, avoid names |
| "What do you dislike" text | Weakness themes, your wedge | Low, avoid names |
| Verified purchase flag | Quality filter | None |
| Company size and industry | Segment fit | Low |
| Reviewer name and title | Rarely needed | High, usually skip |
Most competitive-intelligence work needs the first six rows and can drop the last one entirely. For the general pattern of turning a rendered page into structured fields, see what is data extraction.
The Escalation Ladder for G2
The reliable approach climbs only as high as the site forces you to, because higher rungs are slower and cost more.
| Rung | Method | Clears | Cost |
|---|---|---|---|
| 1 | Browser-TLS HTTP client (curl_cffi) |
Fingerprint checks | Cheap, fast |
| 2 | Stealth headless browser | JS rendering and interstitials | Memory and time per page |
| 3 | Residential IP egress | Reputation blocks | Slowest, priciest |
Rung 1 is a client that presents a genuine Chrome TLS and HTTP/2 fingerprint. It clears the fingerprint layer at near-normal request speed.
from curl_cffi import requests as cffi
r = cffi.get("https://www.g2.com/products/example/reviews", impersonate="chrome")
print(r.status_code)
For G2 you will usually need Rung 2, because the review feed renders client-side. That means a hardened headless browser that runs the JavaScript, waits for the review calls to resolve, and reads the rendered DOM. See scraping JavaScript-rendered websites for the mechanics.
Rung 3 is the one people miss. A stealth browser on a datacenter IP still looks like a datacenter. Reached through a clean residential exit, it looks like a person on a home connection, which is the combination G2's reputation scoring is not tuned to reject.
Rate Limiting and Politeness
Being reliable and being polite are the same discipline here.
- Crawl slowly. A handful of pages per minute, with jitter and backoff, not a burst.
- Cache everything. Reviews barely change between runs. Refetching a page you already have is waste and risk.
- Back off on a
429. Honor the limit and slow down rather than rotating IPs to punch through. The 429 error guide has the details. - Stop when the signal is clear. Repeated hard blocks mean the site does not want you there.
If your goal is tracking how a competitor's ratings move over time rather than a one-off pull, the cadence and diffing patterns in monitoring competitor pricing changes transfer directly to review scores.
Letting the Ladder Run Itself
Building all three rungs is a real project: TLS impersonation, a fleet of patched browsers, a residential proxy pool, challenge handling, retries, and per-domain tuning that drifts every time G2 ships a change.
link.sc is one API that runs that ladder for you. You pass a URL and get back clean markdown, JSON, or raw HTML, and the escalation happens internally, cheapest-first, learning which method a domain needs so it does not waste a browser render where a fast request would do.
import requests
resp = requests.post(
"https://api.link.sc/v1/fetch",
headers={"x-api-key": "lsc_YOUR_KEY"},
json={"url": "https://www.g2.com/products/example/reviews", "format": "markdown"},
)
print(resp.json()["content"])
One request, and the fingerprinting, JS rendering, residential egress, and parsing are handled. It is open core and self-hostable if you would rather run the stack yourself. You still own the ToS and PII judgment calls. The API only handles reliable access to what is already public.
Turn public review pages into clean competitive intelligence: grab a free API key with 500 credits a month and let the escalation ladder run itself.