AutoGen is Microsoft's framework for multi-agent conversations. You spin up an assistant agent and a user-proxy agent, let them talk, and the proxy executes any code or tools the assistant asks for. It is a clean model for building agents that plan, call functions, and check their own work. The one thing a fresh AutoGen setup cannot do is read the live web. The assistant answers from its training data, which stopped at some cutoff, and it rarely tells you when it is guessing.
This post shows the short path to fixing that: register two functions, web_search and fetch_url, backed by link.sc, so any AutoGen agent can pull current pages into the conversation. If you have followed a CrewAI or LangChain integration before, the idea is familiar, but the wiring in AutoGen goes through register_function and a caller/executor split that is worth getting right.
The two roles that make tool calling work
In AutoGen, a tool has two sides. One agent decides to call it (the caller), and another agent actually runs it (the executor). In the common two-agent setup, the AssistantAgent is the caller because it is the one talking to the LLM and choosing tools, while the UserProxyAgent is the executor because it runs code and functions on your machine.
register_function binds a Python function to both sides at once. It generates the JSON schema the LLM needs from your type hints and docstring, hands that schema to the caller, and registers the real callable on the executor. Get this pairing wrong and you see one of two failure modes: the assistant never sees the tool, or it calls a tool the proxy does not know how to run.
What link.sc gives the agent
link.sc collapses live web access into two HTTP endpoints. /v1/search takes a query and returns ranked results with full-page markdown already extracted. /v1/fetch takes a single URL and returns clean markdown, JSON, HTML, or a screenshot. Behind those endpoints it handles JavaScript rendering, proxy rotation, and retries by escalating cheapest-first, so you never have to decide which sites need a headless browser. For the deeper reasoning on why that beats a DIY scraper, see give your AI agent internet access and real-time web search for LLMs.
For AutoGen, the important part is that both endpoints are plain POST calls that return JSON. That maps directly onto two small Python functions.
The two functions
Write the functions first, as ordinary Python. Type hints and docstrings matter here because AutoGen turns them into the tool schema the LLM reads. Be specific in the docstring about when to use each one.
import os
import requests
from typing import Annotated
LINKSC_KEY = os.environ["LINKSC_API_KEY"]
HEADERS = {"x-api-key": LINKSC_KEY, "Content-Type": "application/json"}
def web_search(
query: Annotated[str, "The search query for current web information"],
) -> str:
"""Search the web for current information and return ranked
results with page content as markdown. Use this when you need
to find pages, not when you already have a URL."""
r = requests.post(
"https://api.link.sc/v1/search",
headers=HEADERS,
json={"q": query},
timeout=60,
)
r.raise_for_status()
return r.text
def fetch_url(
url: Annotated[str, "The full URL of the page to read"],
) -> str:
"""Fetch a single URL and return its content as clean markdown.
Use this to read a specific page, such as a result returned by
web_search or a link the user provided."""
r = requests.post(
"https://api.link.sc/v1/fetch",
headers=HEADERS,
json={"url": url, "format": "markdown"},
timeout=90,
)
r.raise_for_status()
return r.text
Returning r.text hands the raw markdown straight to the model. That keeps token counts low compared with raw HTML, which is the single biggest cost lever when you feed web data to an LLM.
Registering the functions with the agents
Now build the two agents and register each function against them. The AssistantAgent is the caller and the UserProxyAgent is the executor.
from autogen import AssistantAgent, UserProxyAgent, register_function
llm_config = {
"config_list": [
{"model": "gpt-4o", "api_key": os.environ["OPENAI_API_KEY"]}
]
}
assistant = AssistantAgent(
name="researcher",
llm_config=llm_config,
system_message=(
"You answer questions using live web data. Use web_search to "
"find pages and fetch_url to read them. Answer only from the "
"content you retrieve. If the tools do not return the answer, "
"say so. Cite the source URL for every claim. Reply TERMINATE "
"when the task is complete."
),
)
user_proxy = UserProxyAgent(
name="user_proxy",
human_input_mode="NEVER",
max_consecutive_auto_reply=6,
is_termination_msg=lambda m: "TERMINATE" in (m.get("content") or ""),
code_execution_config=False,
)
for fn, desc in [
(web_search, "Search the web for current information."),
(fetch_url, "Fetch and read a single URL as markdown."),
]:
register_function(
fn,
caller=assistant,
executor=user_proxy,
name=fn.__name__,
description=desc,
)
A few choices in there are load-bearing. human_input_mode="NEVER" lets the proxy run the loop without prompting you at every step. max_consecutive_auto_reply caps how many tool round-trips the agents take before stopping, which protects you from a runaway loop that keeps fetching pages. code_execution_config=False turns off arbitrary code execution, since here the proxy only needs to run your two registered functions, not run model-written scripts.
Running the conversation
Start the chat from the proxy. The assistant reads the question, calls web_search, gets markdown back through the proxy, decides which result to open, calls fetch_url, and then writes an answer grounded in what it actually read.
user_proxy.initiate_chat(
assistant,
message="What changed in the most recent Python release?",
)
Under the hood the flow is: assistant proposes a web_search call, proxy executes it and returns the JSON, assistant reads the results and proposes a fetch_url on the most promising link, proxy executes that, and the assistant composes the final answer with citations before replying TERMINATE. You did not write any of that orchestration. AutoGen's conversation loop drove it, and the two registered functions were the only new capability you added.
Grounding is a prompt, not a feature
Registering tools gives the model the ability to read the web. It does not by itself stop the model from ignoring what it read and answering from memory. The system message above does the real work: answer only from retrieved content, admit when the answer is not there, and cite the source URL for every claim. That last instruction is the cheap insurance policy, because a cited answer is one you can click and verify in seconds. For a fuller treatment of grounding across an agent stack, the RAG pipeline with live web data post covers the same principle for retrieval systems.
Keeping cost and noise down
Two meters run when an AutoGen agent browses: the LLM tokens and the web-data calls. Return markdown rather than HTML so each page costs a fraction of the tokens. Cap max_consecutive_auto_reply so a curious assistant does not fetch a dozen pages for a one-line answer. Fetch only the top result or two from a search rather than every link. If you want the agent to hit the same pages repeatedly, cache the fetch responses on your side so you are not paying to re-read a page that has not changed.
On link.sc a single search or fetch is roughly one credit, and there are 500 free credits a month, which is enough to build and test a working multi-agent research loop before you spend anything. If you also work inside Claude Desktop or Cursor, the same backend is available over MCP, described in connect Claude to the web.
Once these two functions are registered, every agent in your AutoGen graph can reach them. A planner agent can search, a reader agent can fetch, and a critic agent can verify claims against the sources, all sharing the same live view of the web.
Give your AutoGen agents live web access today. Grab a free key at link.sc/register.