Quick answer: HTTP 401 "Unauthorized" means the server does not know who you are. Your request either carried no credentials or carried ones it rejected: a missing or malformed Authorization header, an absent API key, an expired token, or a key sent in the wrong place. The fix is always on the authentication side. Confirm which credential the endpoint expects, send it in the exact header and format the docs specify, and check that it has not expired or been revoked.
Despite the word "Unauthorized," a 401 is almost never about permissions. It is about identity. The server is saying it cannot authenticate you at all, so it has nothing to make a permission decision about.
What's Actually Happening
Every protected endpoint runs an authentication check before it does anything else. If that check fails, the request stops there and you get:
HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer realm="api"
Content-Type: application/json
{"error": "invalid or missing credentials"}
The WWW-Authenticate response header is the part most people skip past, and it is the most useful thing in the whole response. It tells you exactly which scheme the server wants: Bearer for a token, Basic for a base64 username and password, or a custom scheme. If you are guessing at how to authenticate, that header ends the guessing.
The status code was standardized in the original HTTP spec and refined in RFC 7235. Its defining trait is that a compliant server should include WWW-Authenticate on every 401. Many APIs also return a JSON body explaining what went wrong, so read both.
The Difference That Ends Most Debugging Sessions
The three codes in the same family get confused constantly, and telling them apart is the entire diagnosis:
- 401 Unauthorized: the server does not know who you are. Fix your credentials.
- 402 Payment Required: the server knows you and wants payment. Fix your billing. (More in our 402 breakdown.)
- 403 Forbidden: the server knows who you are and the answer is still no. Fix your permissions, or accept the no.
If you send a valid API key to an endpoint your plan does not include, you get 403, not 401. If you send no key at all, you get 401. Knowing which one you are looking at tells you whether to check your credentials or your permissions, and that saves you from editing the wrong thing.
Why You Hit 401 When Scraping
Scraping and API work throw up 401s for a small, predictable set of reasons:
| Cause | What it looks like | Fix |
|---|---|---|
| Missing key | No auth header sent at all | Add the key the endpoint expects |
| Wrong header name | Key sent as Authorization when the API wants x-api-key |
Match the docs exactly |
| Malformed value | Bearer prefix missing, or an extra space |
Format the value precisely |
| Expired token | Worked yesterday, 401s today | Refresh or reissue the token |
| Revoked or rotated key | Someone regenerated it | Update your stored key |
| Env var not loaded | Key is empty at runtime | Confirm the variable actually resolved |
That last row is the quiet one. A surprising share of production 401s come from an environment variable that never loaded, so the code sends an empty string as the key and the server rejects it. Before you suspect the API, print the length of the credential your code is actually about to send. An empty or truncated value explains more 401s than expired keys do.
Getting the Header Exactly Right
Most authentication failures are formatting failures. The API expects one specific header, and anything else reads as no credential at all. Two common shapes:
import requests
# Bearer token in the Authorization header
resp = requests.get(
"https://api.example.com/data",
headers={"Authorization": "Bearer YOUR_TOKEN"},
)
# API key in a custom header (very common)
resp = requests.get(
"https://api.example.com/data",
headers={"x-api-key": "YOUR_API_KEY"},
)
The mistakes cluster in a few places. Sending the raw token without the Bearer prefix when the scheme requires it. Putting a key in Authorization when the API reads x-api-key, or the reverse. Header names are conventionally case-insensitive, but the value and the scheme are not, so a stray space or a missing prefix is enough to fail.
When an API documents x-api-key, that is exactly the header link.sc uses. You send your key in that header on every request, and a 401 back from it means one thing: the key is missing, mistyped, or no longer valid. Check the value first before you check anything else.
Debugging a 401, Step by Step
- Read
WWW-Authenticateand the body. The header names the scheme; the body usually names the specific problem. Together they solve most cases immediately. - Verify the credential is present at runtime. Log the length, not the value, of the key or token your code is about to send. Zero length means an env or config problem, not an API problem.
- Compare against the docs character by character. Right header name, right scheme prefix, right placement. This is where the bug usually lives.
- Check expiry and rotation. Tokens expire and keys get rotated. A credential that worked last week may simply be dead now.
- Reproduce with curl. A one-line
curl -H "x-api-key: ..."isolates the problem to the credential itself, away from your framework, retry logic, and env loading.
Handling 401 in Code
Unlike a 429, a 401 is not transient. Waiting and retrying does nothing, because the credential will still be wrong on the next attempt. Retrying a 401 in a loop just burns requests and buries the real cause. Treat it as terminal and surface it loudly:
resp = requests.get(url, headers=headers)
if resp.status_code == 401:
# Credentials rejected. Do not retry. This needs a human.
raise AuthError(
f"401 from {url}: check the API key/token and header. "
f"Server said: {resp.text}"
)
resp.raise_for_status()
There is one narrow exception. If you use short-lived tokens that you refresh, a single 401 can be a legitimate signal to fetch a fresh token and retry exactly once. That is a deliberate refresh-and-retry, not a blind loop, and it should give up immediately if the refreshed token also 401s.
When Scraping Turns a 401 Into a Different Problem
If you are hitting a website rather than an API and getting 401s, the site is telling you the resource is behind a login. No amount of header tuning gets you past that without valid session credentials, and using someone else's is not authentication, it is a different conversation entirely. The clean path is an official API with a real key, which is what a 401 is quietly pushing you toward in the first place.
If you would rather not manage keys, tokens, and auth headers across a dozen target APIs, a managed fetch layer like link.sc collapses it to one credential: you authenticate once with your own key, send a URL, and get clean content back. Your scraping code stops carrying a wallet full of other services' credentials.
The Bottom Line
A 401 is the web's "I don't know you." It is an identity problem, not a permission problem and not a rate problem. Read WWW-Authenticate, confirm the credential actually exists at runtime, match the header and scheme exactly to the docs, and check for expiry. Do not retry it blindly, because the credential will not fix itself between attempts. Get authentication right once and 401 disappears from your logs for good.
Skip the auth-header juggling: start free with link.sc, authenticate once with your x-api-key, and get clean web data from a single call.