Running a model locally with Ollama gets you privacy, zero API bills, and no rate limits. It also gets you a model that is sealed off from the world: no browsing, no fresh data, nothing after its training cutoff.
Cloud assistants like ChatGPT and Claude ship with built-in web search. Your local qwen3 or llama3.1 ships with nothing. The good news is that Ollama has supported tool calling for a while now, and wiring live search and page fetching into a local model takes about 60 lines of Python.
Why local models need this more than cloud models do
Two reasons, and they compound.
First, the models people actually run locally are small. An 8B model has a far lossier memory of the web than a frontier model. It half-remembers library APIs, mangles version numbers, and invents URLs with total confidence. Retrieval is not a nice-to-have at that size; it is the difference between usable and not.
Second, there is no platform doing retrieval for you. When ChatGPT searches the web, OpenAI runs the search, fetches the pages, and cleans them up behind the scenes. With Ollama, you are the platform. If you do not hand the model a search tool, it will never have one.
The upside: you control exactly what gets fetched, and the model itself never leaves your machine. Only the web requests do.
What you need
- Ollama installed, with a model that supports tool calling.
qwen3,llama3.1,llama3.2, andmistral-nemoall do. If a model lacks tool support, Ollama returns an error instead of silently ignoring your tools, which is helpful. - The Python client:
pip install ollama requests - A link.sc API key for the search and fetch calls. The free tier gives you 500 credits a month, which is plenty for a personal assistant.
You could point the tools at your own scraper instead, but then you own JavaScript rendering, bot walls, and HTML cleanup for every site the model wanders into. I covered that tradeoff in how to give an AI agent internet access; the short version is that a fetch API earns its keep the first time the model hits a Cloudflare-protected page.
Define the two tools
The Ollama Python client has a nice trick: pass plain Python functions as tools and it builds the JSON schemas from your type hints and docstrings. No hand-written schema blocks.
import requests
API = "https://api.link.sc/v1"
HEADERS = {"x-api-key": "lsc_YOUR_KEY"}
def web_search(query: str) -> str:
"""Search the web for current information.
Args:
query: The search query.
"""
r = requests.post(f"{API}/search", headers=HEADERS, json={"q": query})
serp = r.json().get("serpData", {})
lines = [
f"{res['realPosition']}. {res['title']}\n {res['targetUrl']}\n {res['description']}"
for res in serp.get("results", [])[:5]
]
return "\n".join(lines) or "No results."
def fetch_url(url: str) -> str:
"""Fetch a web page and return its content as clean markdown.
Args:
url: The full URL to fetch.
"""
r = requests.post(f"{API}/fetch", headers=HEADERS,
json={"url": url, "format": "markdown"})
return r.json().get("content", "")[:20000]
Note the slice at the end of fetch_url. Local models have small context windows and you do not want a 300KB page blowing through yours. Twenty thousand characters of markdown is roughly 5,000 tokens, enough for the model to read most articles and doc pages.
The agent loop
The loop is the same shape as any tool-calling agent: send the conversation with the tools attached, execute whatever the model asks for, feed results back, repeat until it answers.
import ollama
TOOLS = {"web_search": web_search, "fetch_url": fetch_url}
SYSTEM = (
"You have web_search and fetch_url tools. For anything involving "
"current events, prices, versions, or facts you are unsure of, search "
"first, then fetch the most promising result. Answer only from fetched "
"content and cite source URLs. If the tools return nothing useful, say so."
)
def ask(question: str) -> str:
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": question},
]
while True:
resp = ollama.chat(
model="qwen3",
messages=messages,
tools=[web_search, fetch_url],
options={"num_ctx": 16384},
)
messages.append(resp.message)
if not resp.message.tool_calls:
return resp.message.content
for call in resp.message.tool_calls:
fn = TOOLS[call.function.name]
result = fn(**call.function.arguments)
messages.append({
"role": "tool",
"tool_name": call.function.name,
"content": result,
})
print(ask("What is the latest stable version of Node.js and what changed?"))
Run it and you can watch the model search, pick a result, fetch the page, and answer with the actual current version number instead of whatever was true when its weights were frozen.
The setting everyone misses: num_ctx
That options={"num_ctx": 16384} line matters more than anything else in the loop.
Ollama's default context window is small, historically 2048 or 4096 tokens depending on version and model. Here is the nasty part: when your fetched page pushes the conversation past that limit, Ollama does not error. It silently truncates from the front, which can drop your system prompt and the beginning of the fetched content. The model then answers from fragments, and you will spend an evening wondering why grounding "does not work" locally when the identical prompt works against a cloud API.
Set num_ctx explicitly to 16k or 32k if your hardware allows it. This is the single most common failure mode I have seen when people wire retrieval into Ollama.
Feeding markdown instead of raw HTML is the other half of surviving a small context window. Raw HTML spends most of your tokens on tags and boilerplate; the same page as markdown is routinely 5 to 10 times smaller, which is why the fetch tool asks for "format": "markdown" instead of scraping and cleaning HTML yourself.
Which model to use
What I have seen from running this loop against local models, roughly in order of preference:
| Model | Tool calling | Notes |
|---|---|---|
| qwen3 (8B+) | Reliable | Best at deciding when to search vs. answer directly |
| llama3.1 8B | Good | Occasionally calls tools with malformed arguments, retry helps |
| mistral-nemo | Good | Solid, slightly chattier tool use |
| llama3.2 3B | Hit or miss | Fine for search, struggles to synthesize long fetched pages |
Below 3B, tool calling gets unreliable enough that I would not bother. The failure is rarely the call format; it is judgment, models either calling tools for questions they should answer directly or never calling them at all. A firm system prompt like the one above fixes most of it.
One more option worth knowing: if you use the local model through a client like Claude Desktop or Cursor rather than your own Python loop, an MCP server gets you the same two tools with zero code. I walked through that setup in MCP web tools for Claude and Cursor.
Where this lands
A local model with these two tools is a genuinely different machine. Recall becomes reading comprehension: the model searches, reads real pages, and answers from text it was just handed, with URLs you can check. That plays to what small models are actually good at, and the pattern extends naturally to full search-read-answer pipelines when you outgrow the single loop.
Your prompts and the model stay on your machine. The only thing that touches the network is the search and fetch traffic, and that costs nothing to try.
Get your local model reading the live web today: grab a free API key at link.sc/register and you have 500 credits a month to build with.