How to Build a Deep-Research Agent That Reads Many Sources

Quick answer: A deep-research agent runs a loop: plan the question into sub-questions, search wide, fetch and read the full text of promising sources, take structured notes with citations, follow the leads those notes surface, and synthesize once it has enough coverage. The engineering challenges are the loop's stop conditions (a budget, so it doesn't run forever) and quality control (source diversity and contradiction checks, so it doesn't confidently repeat one wrong blog post). Everything else is plumbing on top of a search call and a fetch call.

I've built a few of these. Below is the architecture I'd use today, with a runnable code sketch and the cost math that decides whether it's viable.

The Loop, Not the Prompt

The mistake people make is treating deep research as one giant prompt: dump twenty search results into context and ask for a report. That produces confident, shallow, poorly-sourced writing, because the model never actually read anything closely and had no way to notice gaps.

Real research is iterative. You look something up, it raises three new questions, you chase the most important one, and you stop when new sources stop changing your answer. A deep-research agent is that behavior turned into a loop:

1. Plan the question into concrete sub-questions.
2. Search wide across those sub-questions.
3. Fetch and read the full content of promising results.
4. Take notes as atomic, cited claims.
5. Follow leads the notes surface (new sub-questions, cited sources worth reading).
6. Check the budget. If there's room and open questions remain, loop to step 2. Otherwise stop.
7. Synthesize a report with inline citations.

Two subsystems make this into an agent rather than a script: internet access and a controller loop. I covered the access side in give your AI agent internet access; this post is about the loop wrapped around it.

Search Wide, Then Read Deep

The single biggest quality lever is reading full pages, not search snippets. Snippets are fine for deciding which pages to read; they're useless as evidence, because the actual answer is in the paragraph the snippet truncated.

So the pattern is two-phase: search broadly to build a candidate set, score candidates cheaply, then fetch full content only for the ones worth the tokens.

With link.sc both phases are one API each. Search builds the candidate set, and fetch pulls clean markdown from any specific URL:

# Phase 1: search wide
curl -X POST "https://api.link.sc/v1/search" \
  -H "x-api-key: lsc_your_key" \
  -H "Content-Type: application/json" \
  -d '{"q": "grid-scale battery storage cost trends 2026", "engine": "google"}'

# Phase 2: read deep (a URL a note flagged as worth reading in full)
curl -X POST "https://api.link.sc/v1/fetch" \
  -H "x-api-key: lsc_your_key" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://iea.org/reports/battery-storage-2026", "format": "markdown"}'

This is the same search-then-read grounding pattern that reduces hallucinations; the difference is the agent does it many times over and accumulates notes.

The Code Sketch

Here's the loop condensed to its skeleton. It's deliberately simple; the structure is what matters.

import requests

LSC = {"x-api-key": "lsc_your_key"}

def search(q, n=8):
    r = requests.post("https://api.link.sc/v1/search", headers=LSC,
                      json={"q": q, "engine": "google"})
    return r.json()["serpData"]["results"][:n]

def read(url):
    r = requests.post("https://api.link.sc/v1/fetch", headers=LSC,
                      json={"url": url, "format": "markdown"})
    return r.json()["content"]

def deep_research(question, llm, max_fetches=25):
    subqs = llm(f"Break this into 3-5 researchable sub-questions, "
                f"one per line:\n{question}").splitlines()

    notes, seen, fetches = [], set(), 0
    frontier = list(subqs)

    while frontier and fetches < max_fetches:
        subq = frontier.pop(0)
        for hit in search(subq):
            url = hit["targetUrl"]
            if url in seen or fetches >= max_fetches:
                continue
            seen.add(url)

            content = read(url)
            fetches += 1

            # Extract atomic, cited claims + any new leads
            out = llm(f"From this source ({url}), extract factual claims "
                      f"relevant to '{question}' as 'claim [url]' lines. "
                      f"Then list up to 2 new sub-questions worth "
                      f"researching.\n\n{content[:8000]}")
            claims, leads = split_claims_and_leads(out)
            notes.extend(claims)
            frontier.extend(l for l in leads if l not in subqs)

    # Contradiction check before writing
    conflicts = llm(f"List any contradictory claims in these notes:\n"
                    + "\n".join(notes))

    return llm(f"Write a cited report answering '{question}'. Use inline "
               f"citations [url]. Note where sources disagree.\n\n"
               f"NOTES:\n{chr(10).join(notes)}\n\nCONFLICTS:\n{conflicts}")

Real implementations add retries, dedup by domain, and better lead scoring, but this is the whole idea. The controller is boring on purpose; the intelligence lives in the note-extraction and synthesis prompts.

Stop Conditions and Budget

An agent without a budget either loops forever or burns your account. Give it hard limits on more than one axis:

Budget dimension	Why it matters
Max fetches	Caps API cost and wall-clock time
Max loop iterations	Prevents runaway lead-chasing
Diminishing returns	Stop when N fetches add no new claims
Wall-clock timeout	A hard backstop for stuck runs

The most useful soft stop is diminishing returns: track how many new claims each fetch adds, and when the last several fetches added almost nothing, you've saturated the easily-findable sources. Continuing past that point spends money to reread what you already know.

Quality Tricks That Actually Matter

A research agent that reads many sources can be more wrong than one that reads few, because volume feels like rigor. Three cheap guards:

Source diversity. Cap how many claims come from a single domain. Ten pages that all cite the same press release is one source wearing ten hats. Track domains and force the frontier toward new ones.

Contradiction checks. Before synthesis, explicitly ask the model to surface conflicting claims (the step in the sketch above). Surfacing disagreement in the report is more honest and more useful than silently picking one side. AI search products routinely fail here by smoothing over conflicts into a confident wrong answer, which I wrote about in AI search engines are confidently wrong.

Citation discipline. Every claim carries its source URL from the moment it's extracted, not bolted on at the end. If a claim can't be traced to a fetched page, it doesn't go in the report.

This is the same grounding discipline behind building RAG pipelines with real-time web data; a research agent is essentially a RAG loop that builds its own corpus on the fly instead of querying a fixed index.

Estimating the Cost

Do this math before you ship, because it's easy to build something that costs several dollars per question.

A single run of the sketch above with a 25-fetch budget spends, roughly:

• 25 fetches + ~4 searches of web API calls.
• ~25 extraction LLM calls, each reading up to 8k tokens of source.
• 1-2 large synthesis calls over all accumulated notes.

The LLM tokens dominate. Twenty-five extraction passes at several thousand input tokens each is the bulk of the bill, so the biggest lever is using a cheap model for extraction and reserving a frontier model for synthesis only. That split alone can cut cost by more than half with little quality loss, because extraction is a narrow, easy task.

On the web-data side, 25 fetches plus a few searches per question is modest; link.sc's free tier of 500 credits a month covers a dozen-plus full research runs before you pay anything, which is enough to tune the loop before committing.

Where I'd Start

Build the smallest honest version first: plan into sub-questions, search, fetch full content, extract cited claims, synthesize. Get that producing traceable reports before adding lead-following or contradiction checks. Most of the value is in reading full pages instead of snippets and in carrying citations end to end; the fancy loop control is polish on top of that foundation. The quickstart gets the search and fetch calls working in a couple of minutes.

---

Building a research agent? Get a free link.sc key and wire up search plus full-content fetch on 500 free credits a month.