The HTTP Status Codes Problem
The situation with HTTP status codes demonstrated a disastrous colliding of a well-meaning specification design with a ruthless reality as nothing before. This collision actually comes from three sides.
As we discussed in the Chapter 10, one goal of making errors semantic is to help clients understand, what caused an error. HTTP errors, outlined in the corresponding RFCs (most recently in the RFC 7231), are specifically designed bearing this purpose in mind. Furthermore, the REST architectural constraints, as defined by Fielding, imply that not only end user agents should understand error code, but also every network proxy between a client and a server (the ‘layered’ architecture principle). And, in accordance to Fielding’s writings, HTTP status code nomenclature does extensively describe virtually every situation which could happen with your HTTP request: wrong
Accept-* headers value,
Content-Length is absent, HTTP method is unsupported, URI too long, etc.
What the RFC fails to describe is what to do with the error. As we discussed, errors could be resolvable or not. If the error is unresolvable, all this status codes and headers stuff is simply irrelevant to clients, even more so to interim proxies. In fact, three error codes are enough:
400to denote persistent situation (error couldn't be resolved by just repeating the request);
404to denote ‘uncertainity’ cases (the request could be repeated — possibly with different outcomes);
500to denote server-side problems, with
Retry-Afterheader to indicate the desirable retry period.
Aside note: mark a design flaw here. All
4xx status codes are by default not cacheable, except for
414, which are cacheable. We presume that editors of the spec did this with the best intentions, but the number of people who knows this nuance is probably quite close to the number of the spec editors. As a result, there are lots of cases (the author of this book had to deal with one) when
404 was returned erroneously and cached on clients, thus prolonging the outage for an indefinite time.
As for resolvable errors, having status codes nomenclature partially helps. Some of them are concrete, like
411 Length Required, for example; others are not. There are several situations where just having a code is not enough:
400 Bad Requestcode when some parameters are invalid or missing. This error makes absolutely no sense to clients unless specific missing or invalid field is specified — but that's exactly the thing the standard does nothing with! There are no conventional standards to specify which parameter is wrong exactly. Yes, we can, of course, invent a standard of ourselves, but that would contradict the REST idea of protocol transparency.
NB: some purists insist, that a
400code indicates a problem with the request itself, i.e. malformed URI, or header, or body, etc. Sometimes
422 Unprocessable Entityor
412 Precondition Failedare claimed to be the ‘right’ code for invalid parameters error. It doesn't change anything, of course.
403 Forbiddenwhen some authorization or authentication error occurs. There are several quite different
Forbiddensituations, which require quite different actions from the client:
— an authorization token is missing — the user must be invited to log in;
— the token is expired — the token refreshing procedure must be conducted;
— the token belongs to other user — usually indicates that some caches are stall;
— the token is revoked — usually a result of user logging out on all devices;
— the user is bruteforcing the authorization endpoint — some antifraud action is required;
403reason indicates quite different scenarios, some of them (bruteforcing) have nothing in common with others.
- tons of them.
So we quite naturally are moving to the idea of denoting error details with headers and/or response bodies, not trying to invent a specific error code to each situation. It’s quite obvious that we can’t design a new error code for every possible parameter missing in case of
400 error, for example.
Aside note: the spec authors understood this too, adding the following sentence: ‘The response message will usually contain a representation that explains the status’. We couldn’t agree more, but this sentence not only renders the entire spec section redundant (why use status codes at all?), but also contradicts to the REST paradigm: other agents in the layered system couldn’t understand what the response message explains, thus making the error appear opaque to them.
The conclusion seems to be: use status codes just to indicate a general error class, expressed in the HTTP protocol terms, and fill the response body with details. But here the third collision occurs: the implementation practice. From the very beginning of the Web frameworks and server software started relying on status codes for logging and monitoring purposes. I think I wouldn’t exaggerate gravely if I said that there were literally no platform which natively supported building charts and graphs using custom semantic data in the responses, not status codes. One severe implication is that developers started inventing new codes to have their monitoring work correctly, cutting off insignificant errors and escalating vital ones.
Not only the number of status codes soared, but also their semantic meaning stirs. Many developers simply never read specs. The most evident example is the
401 Unauthorized code: the spec prescribes the servers to return
WWW-Authenticate header, which they never do — for obvious reasons, since the only usable value for this header is
Basic. Furthermore, the spec is extensible at this point, new authentication realms could be introduced and standardized — but nobody cares. Right now using
401 to indicate an absence of authorization headers is a common practice — omitting the
WWW-Authenticate header, of course.
In a modern world we have to deal with a literal mess: HTTP status codes are used not for the protocol’s purity sake, but to build the graphs; their semantic meaning forgotten; and clients often don’t event try to get some useful information from the status codes, reducing them to the first digit. It’s also a common practice to return resolvable errors as
So, what are you proposing, pal?
Actually, there are three different approaches to solve this situation.
- Abandon REST paradigm, stick to pure RPC. Use HTTP status codes to indicate the problems with the HTTP network layer itself. So you would actually need just 2 of them:
200 OKif the server got the request, regardless of the result — execution errors are to be returned as
500 Internal Server Errorif the request can't reach the server.
You may employ
400 Bad Requestalso to denote client errors; it slightly complicates the setup, but allows for using some interim software like API gateways.
- ‘Run with scissors’, using common practices, just cautiously, avoiding violating HTTP semantics. Use HTTP status codes to separate graphs (sometimes using exotic codes). Describe errors semantically and make sure clients don’t try to detect anything valuable from the status codes.
NB: some industrial-grade platforms manage to do both, i.e. combine a pure RPC-style approach with extensively employing various HTTP status codes to indicate a subset of problems (403s and 429s for instance, which are purely business logic-bound, having nothing to do with the HTTP itself). Though in a practical sense this approach seems to work, it’s very hard to to tell which problems they face in modern smart-proxy rich environments, not mentioning aesthetic impressions.
- Try organizing the mess. Including, but not limited to:
— using HTTP codes to indicate the problems expressible in HTTP terms (like using
406 Unacceptableto indicate invalid
Accept-Languagerequest header value);
— defining additional machine-readable error response details, preferably in a form of HTTP headers (since reading them doesn’t require parsing the entire response body, so interim proxies and API gateways might operate them less expensively); for example, use something like an
X-My-API-Error-Reasonheader containing with pre-defined semantic errors nomenclature;
— customize graphs and monitoring to make them use this specific data in addition to the status codes or instead of them;
— make sure that clients are treating status codes and the error details correctly, especially with regard to dealing with unknown errors.
The choice is yours, though we have to warn you that option #3 is quite expensive in implementing.
This is the draft for the future ‘HTTP API’ section of the book; the work continues at Github.