How to Add Web Search and Fetch to the OpenAI Agents SDK

The OpenAI Agents SDK gives you a clean loop: define an agent, hand it tools, and let the model decide when to call them. It even ships a hosted WebSearchTool so your agent can look things up without you writing any HTTP code. That is a great start, and for a lot of quick lookups it is all you need.

The gap shows up the moment your agent needs the actual page. Hosted web search returns short snippets and a synthesized answer. It does not hand you the full text of a JavaScript-rendered docs page, a changelog behind a bot wall, or a pricing table that only exists after the page hydrates. When the answer lives in the body of a specific URL, you want a tool that fetches and returns the whole thing as clean text.

This post shows how to add an external search and fetch tool to the Agents SDK with a @function_tool, and how to decide when to lean on the built-in search versus a full-content fetch.

What the built-in WebSearchTool gives you

The SDK's WebSearchTool is a hosted tool. It runs on OpenAI's side, so there is no key to manage and no request loop to write. You add it to the agent's tools list and the model calls it when a question needs current information.

from agents import Agent, Runner, WebSearchTool

agent = Agent(
    name="Researcher",
    instructions="Answer questions using web search when needed.",
    tools=[WebSearchTool()],
)

result = Runner.run_sync(agent, "What shipped in the latest Python release?")
print(result.final_output)

This is the right tool for "what happened recently" or "point me at the relevant sources." What it is not built for is deep extraction. You get search results and a summary, not the raw markdown of the page the model wants to read. For grounding an answer in the exact contents of a URL, you need a second tool.

Adding a fetch tool with @function_tool

The @function_tool decorator turns any Python function into a tool the model can call. It reads your type hints and docstring to build the schema, so the description the model sees is just the docstring you write. That makes a fetch tool about ten lines.

Here we point it at link.sc, which exposes a /v1/fetch endpoint that takes a URL and returns clean markdown. It handles the parts that make DIY fetching painful: rendering JavaScript, rotating proxies, getting past bot walls, and converting messy HTML into something a model can actually read.

import os
import requests
from agents import Agent, Runner, WebSearchTool, function_tool

LINKSC_KEY = os.environ["LINKSC_API_KEY"]  # lsc_...

@function_tool
def fetch_page(url: str) -> str:
    """Fetch a single web page and return its full content as clean markdown.

    Use this when you need the complete text of a specific URL, not just a
    search snippet. Works on JavaScript-heavy and bot-protected pages.
    """
    r = requests.post(
        "https://api.link.sc/v1/fetch",
        headers={"x-api-key": LINKSC_KEY, "Content-Type": "application/json"},
        json={"url": url, "format": "markdown"},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["content"]

Now give the agent both tools and let it choose:

agent = Agent(
    name="Researcher",
    instructions=(
        "Use web_search to find relevant pages. When you need the full "
        "contents of a specific page, call fetch_page on its URL. "
        "Answer only from what the tools return, and cite the source URL."
    ),
    tools=[WebSearchTool(), fetch_page],
)

result = Runner.run_sync(
    agent,
    "Read the latest Python 'what's new' page and list the three biggest changes.",
)
print(result.final_output)

The typical trajectory: the model calls web_search to locate the right URL, then calls fetch_page on that URL to pull the full markdown, and answers from the body text instead of from a snippet. You did not write a tool loop; the Runner handles the back and forth, running your function whenever the model asks and feeding the result back until the agent produces a final answer.

Adding a search tool of your own

You do not have to use the hosted search at all. If you want search results with full page content, or you want the same provider handling both search and fetch, wrap link.sc's /v1/search in a second @function_tool and drop WebSearchTool entirely.

@function_tool
def web_search(query: str) -> list[dict]:
    """Search the web and return results with page content for each hit."""
    r = requests.post(
        "https://api.link.sc/v1/search",
        headers={"x-api-key": LINKSC_KEY, "Content-Type": "application/json"},
        json={"q": query},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()

agent = Agent(
    name="Researcher",
    instructions="Search, then fetch full pages when you need detail. Cite sources.",
    tools=[web_search, fetch_page],
)

The advantage of owning both tools is consistency. You control the response shape, you decide how many results come back, and you get search and fetch billed and rate-limited through one account. That matters when you move from a demo to something running unattended.

When to swap search for fetch

The two are not competitors so much as different altitudes. A short rule that holds up in practice:

Situation	Reach for
"What is going on with X lately"	Search
"Which page should I read"	Search
"What does this specific URL actually say"	Fetch
Page is JavaScript-rendered or bot-walled	Fetch
You need tables, code blocks, or long-form text intact	Fetch
You need to cite exact wording	Fetch

The built-in WebSearchTool covers the top rows well and costs you nothing to set up. The moment your answer depends on the real contents of a page, hand the model a fetch tool. In most research agents you keep both: search to find, fetch to read.

Grounding beats raw access

Tools alone do not stop a model from filling gaps with confident guesses. The instruction that does the real work is the one telling the agent to answer only from tool output and to cite the URL behind each claim, which is exactly what the instructions above do. That shifts the job from recall to reading comprehension over text you handed it, and it makes wrong answers easy to catch because you can click the link and check. We go deeper on this pattern in How to Give an AI Agent Access to the Internet and on live retrieval in real-time web search for LLMs.

What it costs

Two meters run at once: the LLM tokens the Agents SDK spends, and the web-data calls your tools make. Keep tokens down by returning markdown rather than raw HTML, and by fetching only the top result or two instead of every hit. On link.sc a page or a search is roughly one credit, with 500 free credits a month, which is enough to build and test a real agent before you pay anything. If you would rather add the same fetch and search inside Claude Desktop or Cursor instead of code, the MCP server exposes the identical tools.

Wrapping up

The Agents SDK makes tools trivial to add: a @function_tool, a type hint, a docstring, and the Runner handles the loop. Start with the hosted WebSearchTool for quick lookups, add a fetch tool for the pages that actually hold your answer, and instruct the agent to ground every claim in what it read. That combination turns a model stuck at its training cutoff into an agent that reads the live web on demand.

---

Give your Agents SDK project live web access today. Grab a key at link.sc/register and start with 500 free credits a month.