The Best Way to Convert HTML to Markdown for LLMs

If you're feeding web pages to an LLM, you've probably already learned the hard way that you can't just dump raw HTML into the context window. It works, technically. But it's expensive, it's noisy, and it quietly degrades the quality of everything downstream — retrieval, summarization, extraction. Markdown is the format most models were trained on and reason over well, so converting HTML to clean markdown is usually the right move. The question is how to do it without losing the parts that matter.

This post walks through why raw HTML is a bad input, what to keep and what to throw away, the popular open-source tools and where they fall short, and finally the one-call option if you'd rather not maintain any of it.

Why Raw HTML Is a Bad Input

A typical article page is mostly not the article. You're looking at a nav bar, a cookie banner, a newsletter modal, a sidebar of "related posts," a comments widget, a footer with a few hundred links, and a pile of inline <script> and <style> tags. The actual prose might be 5-10% of the byte count.

That has two consequences for an LLM pipeline:

• Token cost. Every <div class="...">, every tracking script, every SVG path is tokens you pay for and the model has to read. A page that's 2,000 useful words can easily be tens of thousands of tokens of HTML.
• Retrieval quality. This is the subtler one. If you're chunking and embedding pages for RAG, boilerplate pollutes your embeddings. A chunk that's half footer links doesn't embed to anything meaningful, and the repeated nav text across every page makes your vectors look artificially similar. You get worse recall and more junk in your top-k.

Markdown fixes both. It keeps the semantic signal — headings, lists, tables, links — and drops the presentational scaffolding. It's compact, and models handle its structure natively.

What to Strip and What to Keep

The whole job of HTML-to-markdown conversion is deciding what's content and what's chrome. A good rule of thumb:

Strip:

• Navigation, headers, footers, breadcrumbs
• Ads, cookie/consent banners, newsletter and login modals
• <script>, <style>, <noscript>, and inline event handlers
• Sidebars, "related articles," social share buttons, comment threads
• Tracking pixels and most <svg>/icon markup

Keep:

• Headings (<h1>–<h6>) — they carry the document's outline and make great chunk boundaries
• Paragraphs, ordered and unordered lists
• Tables — these matter more than people expect; flattening a table into a run-on sentence destroys the relationships in it
• Code blocks with their language hint
• Link text (the anchor text is usually meaningful even if you drop the raw URL)
• Image alt text as a short description, not the binary

The two things people most often get wrong are tables and headings. Tables get mangled into unreadable prose, and heading hierarchy gets flattened so you lose the structure you'd otherwise use to chunk.

The DIY Route: Readability + Turndown

The standard open-source stack is two tools doing two jobs.

Mozilla Readability (the library behind Firefox Reader View) handles content extraction — it scores the DOM and pulls out the main article, discarding nav and sidebars. Turndown handles conversion — it takes an HTML fragment and turns it into markdown.

A minimal Node pipeline looks like this:

import { JSDOM } from "jsdom";
import { Readability } from "@mozilla/readability";
import TurndownService from "turndown";
import { gfm } from "turndown-plugin-gfm";

const dom = new JSDOM(rawHtml, { url: pageUrl });
const article = new Readability(dom.window.document).parse();

const turndown = new TurndownService({ headingStyle: "atx" });
turndown.use(gfm); // adds table + strikethrough support

const markdown = turndown.turndown(article.content);

This is a completely reasonable starting point, and for clean, article-shaped pages it works well. But the limits show up fast:

• Readability is tuned for articles. On documentation, product pages, dashboards, forums, or SPA-rendered layouts, its heuristics misfire — it drops real content or keeps junk.
• JavaScript-rendered pages return almost nothing. fetch() gives you the pre-hydration HTML. If the content is rendered client-side, you need a headless browser (Puppeteer/Playwright), which is a whole separate operational burden.
• Turndown needs plugins for tables and still struggles with nested tables, definition lists, and complex layouts.
• You own the edge cases forever — encoding issues, relative URLs, <pre> whitespace, lazy-loaded images, sites that block datacenter IPs.

None of this is unsolvable. It's just an ongoing maintenance stream that has nothing to do with your actual product.

The One-Call Route: link.sc

link.sc collapses fetch, JS rendering, extraction, and markdown conversion into a single API call. You hand it a URL, it returns clean, LLM-ready markdown — nav/footer/ads/scripts already stripped, headings and tables preserved.

curl -X POST https://api.link.sc/v1/fetch \
  -H "x-api-key: lsc_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article", "format": "markdown"}'

Python, dropping straight into a chunking step:

import requests

resp = requests.post(
    "https://api.link.sc/v1/fetch",
    headers={"x-api-key": "lsc_your_key_here"},
    json={"url": "https://example.com/article", "format": "markdown"},
)
markdown = resp.json()["content"]

# markdown is clean — chunk on heading boundaries for embedding
chunks = re.split(r"\n(?=#{1,3} )", markdown)

Node:

const res = await fetch("https://api.link.sc/v1/fetch", {
  method: "POST",
  headers: {
    "x-api-key": "lsc_your_key_here",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ url, format: "markdown" }),
});
const { content } = await res.json();

If you'd rather run it yourself, link.sc is open core and self-hostable, so you can keep the same pipeline behind your own infrastructure.

Which Should You Pick?

If you're converting a handful of clean article pages and enjoy owning the stack, Readability + Turndown is genuinely fine — start there. The moment you hit JS-rendered pages, non-article layouts, or the need to do this reliably at volume, the maintenance cost of DIY starts to dominate, and a single call that already handles rendering and extraction is the pragmatic choice.

Either way, the principle holds: convert to markdown, strip the chrome, and protect your headings and tables. Your token bill and your retrieval quality both depend on it.

---

Want clean markdown from any URL in one call? link.sc gives you 500 free credits every month — grab an API key or read the docs.