Quick answer: To build an SEO audit tool, fetch each page's rendered HTML, extract the on-page signals that matter (title, meta description, heading structure, canonical tag, structured data, internal links, and broken links), score each signal against a rule set, and roll the results into a per-page report. A fetch API like link.sc gives you rendered HTML and clean markdown in one call, so your audit checks JavaScript-rendered pages the way a modern crawler sees them.
Most SEO audit tools are a crawler plus a checklist. The crawler is the hard part, JavaScript rendering, proxies, retries, and it is also the part you can offload. That leaves you free to write the checks, which is where the actual value lives. This post shows how to build a working auditor with a clear scoring model.
The Signals Worth Auditing
Not every SEO factor is on-page, and not every on-page factor is worth a rule. Focus on the ones that are concrete, measurable from the HTML, and frequently broken.
| Signal | What to check | Why it matters |
|---|---|---|
| Title tag | Present, 30 to 60 chars, unique | Primary relevance and click signal |
| Meta description | Present, 70 to 160 chars | Drives click-through from the SERP |
| H1 | Exactly one, non-empty | Clear page topic |
| Heading order | No skipped levels (H1 to H2 to H3) | Structure and accessibility |
| Canonical | Present, absolute, self or intended target | Prevents duplicate content splits |
| Structured data | Valid JSON-LD present | Rich result eligibility |
| Internal links | Reasonable count, descriptive anchors | Crawl paths and link equity |
| Broken links | No 404s in outbound links | Wasted crawl budget, bad UX |
Step 1: Fetch the Page as HTML and Markdown
You want two views of each page. HTML for tag-level checks (title, canonical, JSON-LD), and markdown for a clean read of the content and link structure. The link.sc fetch API returns either, and renders JavaScript so single-page-app content is actually present.
import requests
API = "https://api.link.sc/v1/fetch"
HEADERS = {"x-api-key": "lsc_your_key", "Content-Type": "application/json"}
def fetch(url, fmt):
r = requests.post(API, headers=HEADERS, json={
"url": url,
"format": fmt,
"render_js": True
})
r.raise_for_status()
return r.json()["content"]
Step 2: Extract the On-Page Signals
Parse the HTML with a real parser rather than regex for the structural bits. Here is the core extraction:
from bs4 import BeautifulSoup
import json
def extract_signals(html):
soup = BeautifulSoup(html, "html.parser")
title = soup.title.string.strip() if soup.title and soup.title.string else None
meta = soup.find("meta", attrs={"name": "description"})
meta_desc = meta["content"].strip() if meta and meta.get("content") else None
canonical = soup.find("link", attrs={"rel": "canonical"})
canonical_href = canonical["href"] if canonical and canonical.get("href") else None
headings = [(int(h.name[1]), h.get_text(strip=True))
for h in soup.find_all(["h1", "h2", "h3", "h4"])]
jsonld = []
for tag in soup.find_all("script", type="application/ld+json"):
try:
jsonld.append(json.loads(tag.string))
except (json.JSONDecodeError, TypeError):
pass
links = [a["href"] for a in soup.find_all("a", href=True)]
return {
"title": title,
"meta_description": meta_desc,
"canonical": canonical_href,
"headings": headings,
"jsonld": jsonld,
"links": links,
}
Step 3: Score Each Signal
A score is only useful if the rules are explicit. Give each check a weight, run it, and collect the failures with human-readable messages.
def score_page(sig, page_url):
issues, points, total = [], 0, 0
def check(ok, weight, msg):
nonlocal points, total
total += weight
if ok:
points += weight
else:
issues.append(msg)
t = sig["title"]
check(bool(t), 15, "Missing title tag")
check(t and 30 <= len(t) <= 60, 5,
f"Title length off (is {len(t) if t else 0}, want 30-60)")
m = sig["meta_description"]
check(bool(m), 10, "Missing meta description")
check(m and 70 <= len(m) <= 160, 5, "Meta description length off (want 70-160)")
h1s = [h for lvl, h in sig["headings"] if lvl == 1]
check(len(h1s) == 1, 10,
f"Expected exactly one H1, found {len(h1s)}")
check(bool(sig["canonical"]), 10, "Missing canonical tag")
check(len(sig["jsonld"]) > 0, 10, "No JSON-LD structured data")
internal = [l for l in sig["links"] if page_url.split("/")[2] in l]
check(len(internal) >= 3, 10,
f"Only {len(internal)} internal links")
return round(points / total * 100), issues
Tune the weights to your priorities. A title tag failing should hurt more than a slightly short meta description, and the weights above reflect that.
Step 4: Check for Broken Links
Broken outbound links waste crawl budget and signal neglect. Check them with lightweight requests so you are not fetching full bodies you do not need.
def find_broken(links, limit=25):
broken = []
for url in links[:limit]:
if not url.startswith("http"):
continue
try:
r = requests.head(url, allow_redirects=True, timeout=10)
if r.status_code >= 400:
broken.append((url, r.status_code))
except requests.RequestException:
broken.append((url, "unreachable"))
return broken
Cap the number you check per page so a link-heavy page does not blow up your run time, and cache results across pages since the same footer links repeat everywhere.
Step 5: Assemble the Report
Roll per-page results into one report. Sort by score ascending so the worst pages, the ones worth fixing first, sit at the top.
def audit(urls):
results = []
for url in urls:
html = fetch(url, "html")
sig = extract_signals(html)
score, issues = score_page(sig, url)
results.append({
"url": url, "score": score, "issues": issues,
"broken_links": find_broken(sig["links"]),
})
return sorted(results, key=lambda r: r["score"])
Output as JSON for a dashboard, or render a simple HTML table for a human. Either way, the deliverable is a ranked list of pages and exactly what is wrong with each.
Where On-Page Auditing Meets Keyword Research
An on-page audit tells you whether a page is technically sound. It does not tell you whether the page answers what searchers actually ask. Pair the auditor with SERP research so you close both gaps at once. Our guide to extracting People Also Ask data for SEO shows how to pull the questions a page should answer, which you can then check your headings against.
Why Offload the Crawl
You can crawl with plain requests, and for a static site it works. It breaks the moment a page renders content with JavaScript, blocks your IP, or sits behind a CDN that serves a challenge to non-browser clients. A rendering fetch API returns the page as a real browser sees it, which is the whole point of an audit: you want to grade what Google grades. link.sc offers 500 free credits a month at link.sc/pricing, enough to audit a small site end to end.
Start with the eight checks above, get a report you trust, then add rules as you find patterns worth catching. An audit tool is never finished, but it is useful the day it runs its first page.
Want to audit any site without building a crawler first? Grab a free link.sc key and fetch rendered pages with one call.