How to Scrape Weather Data (Use a Weather API Instead)

Quick answer: You almost never need to scrape weather data, because clean weather APIs already give it to you as JSON. In the US, the National Weather Service API at weather.gov is free and public with no key. Open-Meteo is free for non-commercial use and needs no key, and OpenWeather offers a free tier with a key. Use one of these; scraping a forecast page is slower, more fragile, and unnecessary.

Why this one is easy

Weather is the friendliest data domain there is. The agencies that produce forecasts publish them through documented APIs, on purpose, for exactly this use. There is no adversarial anti-bot layer to worry about and no gray area about whether the data is meant to be consumed programmatically. It is.

So if you find yourself writing CSS selectors to pull a temperature off a weather site, stop. There is a JSON endpoint that hands you the same number (usually with more detail) and will not break when the site redesigns.

The three APIs worth knowing

API	Coverage	Auth	Cost	Best for
NWS / weather.gov	United States	None (User-Agent required)	Free	US forecasts, alerts, official data
Open-Meteo	Global	None	Free for non-commercial	Quick global forecasts, historical
OpenWeather	Global	API key	Free tier plus paid	Global current, forecast, one-call

Check each provider's current terms and rate limits on its own site, since free-tier limits and commercial rules change.

National Weather Service (weather.gov)

The NWS API is free, public, and authoritative for the United States. There is no API key. The one requirement is a descriptive User-Agent header identifying your app, which is how they ask you to be a good citizen.

You look up a point, get its forecast URL, then fetch the forecast:

import requests

headers = {"User-Agent": "my-weather-app ([email protected])"}

# 1. Resolve a lat/lon to its NWS grid endpoint
point = requests.get(
    "https://api.weather.gov/points/38.8894,-77.0352",
    headers=headers,
).json()

# 2. Fetch the forecast for that grid
forecast_url = point["properties"]["forecast"]
periods = requests.get(forecast_url, headers=headers).json()["properties"]["periods"]

for p in periods[:3]:
    print(p["name"], p["temperature"], p["temperatureUnit"], "-", p["shortForecast"])

That is the whole thing. No scraping, no key, official data.

Open-Meteo

Open-Meteo is a clean, keyless global API that is free for non-commercial use. It is my go-to for a quick global forecast or historical weather.

import requests

resp = requests.get(
    "https://api.open-meteo.com/v1/forecast",
    params={
        "latitude": 51.5074,
        "longitude": -0.1278,
        "current": "temperature_2m,wind_speed_10m",
        "hourly": "temperature_2m",
    },
)
data = resp.json()
print(data["current"]["temperature_2m"], data["current_units"]["temperature_2m"])

OpenWeather

OpenWeather has broad global coverage and a free tier that needs a key passed as a query parameter. Note that the key goes in the URL for OpenWeather; that is their convention. It is unrelated to link.sc, which uses an x-api-key header.

import requests

resp = requests.get(
    "https://api.openweathermap.org/data/2.5/weather",
    params={"q": "Tokyo", "appid": "YOUR_OPENWEATHER_KEY", "units": "metric"},
)
data = resp.json()
print(data["main"]["temp"], data["weather"][0]["description"])

When would you ever fetch a page instead?

Rarely, but it happens. If you need something a weather API does not model (a specific local station's public bulletin, a marine or aviation notice on a public page, or a niche regional service with no API), then reading that one public page is reasonable.

For that, link.sc fetches the URL and returns clean markdown so you skip the HTML wrangling. The base is https://api.link.sc/v1 and the auth header is x-api-key.

curl -X POST https://api.link.sc/v1/fetch \
  -H "x-api-key: lsc_your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example-regional-weather.gov/bulletins/marine",
    "format": "markdown"
  }'

The response is { "content": "..." }. But be honest with yourself first: 95% of weather needs are already covered by the APIs above, and the API answer is cleaner every time.

If you are not sure which source has what you need, a search that returns full page content can help you find the primary source fast:

curl -X POST https://api.link.sc/v1/search \
  -H "x-api-key: lsc_your_key" \
  -H "Content-Type: application/json" \
  -d '{
    "q": "official marine weather bulletin API region",
    "engine": "google"
  }'

The search field is q, and the response includes serpData.

A quick note on rate limits and courtesy

Even keyless APIs have limits, and the NWS asks for a real User-Agent so they can contact you if something goes wrong. Cache responses, do not poll faster than the forecast actually updates (forecasts refresh on the order of hours, not seconds), and respect any documented limit. This is basic courtesy that keeps free services free.

Legal and ethics note

Government weather data (like NWS) is generally public and free to use, which is part of why this domain is so pleasant. Third-party APIs like OpenWeather have their own terms about commercial use and redistribution, so read them if you are building a product. None of this is legal advice. The general rule holds: prefer the official API, respect rate limits, and do not bypass access controls. For the broader picture, see ethical web scraping and compliance best practices.

Bottom line

Weather is the rare case where "how do I scrape this" has a clean answer: do not. Use weather.gov for the US, Open-Meteo or OpenWeather for global, and keep a fetching tool like link.sc around only for the genuinely odd public page that no API covers. You will write less code, and it will not break next month.

For the public pages an API does not model, link.sc turns any URL into clean, structured content. Start free.