Analysts spend most of their time not analyzing. Surveys of data teams have put the share of hours spent gathering and cleaning data somewhere between 50 and 80 percent, and my own experience sits comfortably in that range. The actual analysis, the part someone is paid to think about, gets whatever time is left.
That is the real pitch for AI agents in data analysis. Not "the model finds insights for you," which is mostly marketing, but "the model automates the gathering, cleaning, and structuring so a human, or a final model call, can reason over something trustworthy."
There is a catch, though. Most data analysis agents only see two kinds of data: files you upload, and whatever was in the training set. Both are stale by definition. If your question is "what are competitors charging this week" or "what skills do data engineering job postings ask for right now," the agent needs live web data, and that changes the architecture. Let's build one that handles it.
What a data analysis agent actually automates
An analysis agent is not a chatbot with a CSV attached. Chat-with-your-CSV tools answer questions about data you already collected. An analysis agent runs the whole loop, including collection:
- Decompose: turn a vague question into concrete data requirements.
- Gather: find and fetch the sources that hold that data.
- Structure: turn messy pages into typed records.
- Compute: run the actual math in code, never in the model's head.
- Interpret: explain what the computed numbers mean.
Steps 1, 3, and 5 are judgment calls, so the LLM owns them. Steps 2 and 4 are mechanical and unforgiving, so plain code owns them. Every reliable agent I have built follows that split, and every unreliable one violated it somewhere.
| Stage | Owner | Why |
|---|---|---|
| Decompose question | LLM | Requires understanding intent |
| Fetch data | Code | Deterministic, auditable |
| Structure records | LLM | Pages are messy and varied |
| Compute statistics | Code | Models are bad at arithmetic |
| Interpret results | LLM | Numbers need context |
The compute row deserves emphasis. Language models will confidently produce a wrong median from a list of 30 numbers. Never ask the model to do arithmetic. Hand it computed results and ask for interpretation.
The worked example
Our question: "What is the typical advertised salary range for remote data engineer roles this week, and which skills come up most?"
You cannot answer that from training data. It changes weekly. So the agent has to search, fetch, extract, and then compute.
Step 1: decompose the question
import anthropic, json, requests, statistics
from collections import Counter
client = anthropic.Anthropic()
LSC = {"x-api-key": "lsc_..."}
plan = client.messages.create(
model="claude-sonnet-4-5", max_tokens=400,
messages=[{"role": "user", "content":
"Question: typical advertised salary and top skills for remote "
"data engineer roles this week. Return JSON only: "
'{"queries": [..3 web search queries for job listing pages..]}'}],
)
queries = json.loads(plan.content[0].text)["queries"]
The model proposes queries. It does not get to invent URLs or data.
Step 2: gather live pages
pages = []
for q in queries:
r = requests.get("https://api.link.sc/v1/search",
params={"q": q}, headers=LSC).json()
for item in r["results"][:5]:
f = requests.get("https://api.link.sc/v1/fetch",
params={"url": item["url"]}, headers=LSC)
if f.ok:
pages.append({"url": item["url"],
"content": f.json()["content"]})
I use link.sc here because it returns pages as clean markdown instead of raw HTML, and job boards are exactly the kind of JavaScript-heavy, bot-hostile sites that break a naive requests.get. If you want to understand what a DIY fetching stack involves, I covered it in how to give your AI agent internet access. The short version: it is a bigger project than the agent itself.
Step 3: structure pages into records
EXTRACT = """From the page content, extract every data engineering job
posting. Return JSON only:
{"jobs": [{"title": string, "salary_min_usd": number or null,
"salary_max_usd": number or null, "skills": [string]}]}
Use ONLY the page content. Null anything not stated. If the page has
no job postings, return {"jobs": []}."""
records = []
for p in pages:
msg = client.messages.create(
model="claude-sonnet-4-5", max_tokens=2000,
messages=[{"role": "user",
"content": EXTRACT + "\n\n" + p["content"][:30000]}])
records += json.loads(msg.content[0].text)["jobs"]
The "use ONLY the page content" rule plus an explicit null policy is what keeps the model from backfilling salaries it remembers from 2024. I go deeper on extraction and validation in how to build a web scraping AI agent; the same grounding tricks apply here.
Step 4: compute in code
mins = [j["salary_min_usd"] for j in records
if j["salary_min_usd"] and 30000 < j["salary_min_usd"] < 600000]
maxs = [j["salary_max_usd"] for j in records
if j["salary_max_usd"] and 30000 < j["salary_max_usd"] < 600000]
skills = Counter(s.lower() for j in records for s in j["skills"])
stats = {
"postings": len(records),
"median_min": statistics.median(mins) if mins else None,
"median_max": statistics.median(maxs) if maxs else None,
"top_skills": skills.most_common(10),
}
Note the range filter. Extraction will occasionally produce an hourly rate or a currency mixup, and a sanity bound catches most of it. Boring code, high value.
Step 5: interpret, with the numbers pinned
answer = client.messages.create(
model="claude-sonnet-4-5", max_tokens=800,
messages=[{"role": "user", "content":
"Write a short analysis for a hiring manager. Use ONLY these "
"computed statistics, and quote them exactly: "
+ json.dumps(stats)}])
print(answer.content[0].text)
Because the statistics arrive precomputed, the model cannot fudge a median. Its only job is framing: what the range means, which skills cluster together, what a manager should do with it.
Where this breaks, and what to do about it
Being honest about failure modes matters more in analysis than anywhere else, because a wrong number looks identical to a right one.
Sample bias is your biggest risk. Fifteen pages from three job boards is not the labor market. Either widen the gather step or, better, have the interpretation prompt state the sample explicitly: "based on N postings from these sources." Never let the agent generalize silently.
Extraction errors compound. One bad salary in a set of 200 barely moves a median. In a set of 12 it wrecks it. Small samples need tighter validation, not looser.
Context windows fill fast. Fetching 20 full pages can mean hundreds of thousands of tokens. Truncation helps, but if you are doing this at scale, read up on token optimization when feeding web data to LLMs before your bill does the teaching.
Recurring questions deserve recurring runs. The whole point of live data is that the answer changes. Wrap the script in a weekly cron and diff the stats between runs. Trend lines from your own collected snapshots beat any single point-in-time answer, and they are something no pretrained model can offer you.
The pattern generalizes well past salaries: competitor pricing, review sentiment, market sizing from public filings, content gap analysis. Anywhere the data lives on the open web and the math should live in code, this loop applies. The model decomposes and interprets. The code fetches and computes. Keep that boundary and your analysis agent produces numbers you can defend in a meeting.
Get search and fetch APIs built for analysis agents, with 500 free requests a month, at link.sc/register.