Introducing HTTP Headers and Status Codes

An introduction to HTTP headers and status codes

Overview

HTTP requests and responses include HTTP headers, which contain additional information that the client and server pass on to each other. Request and response bodies are located at the bottom of the message. Depending on the method, not all requests require bodies. Requests with methods that pertain to fetching resources like GET, HEAD, DELETE, and OPTIONS usually don’t need them.

Requests with methods that send data to servers to update them do need bodies. One example is the POST method.

Header directives

Header directives are located after the request and response lines in the form of one or more header fields. There are several types of HTTP headers, each of which attaches more information to the request or response.

Header types

Following is some of the information that HTTP headers define.

  • Caching (as in cache control)
  • Client identification (e.g., browser cookies, IP address, user-agent) or anonymity (e.g., VPN or proxy masking, user-agent spoofing)
  • Encoding of sent or received information
  • Session verification
  • Instructions for how the server should handle data
  • Age of document being downloaded

HTTP response status codes

HTTP response status codes indicate whether a specific request successfully completed. They fall into the following numbered categories.

  • Informational responses (100-199)
  • Successful responses (200-299)
  • Redirection messages (300-399)
  • Client error messages (400-499)
  • Server error messages (500-599)

Webscale usage of custom HTTP response status codes

The 467 and 499 HTTP response status codes are custom codes. Webscale uses them to respond to the following situations.

  • 467 code: Indicates a client error regarding a CAPTCHA challenge. The incoming request does not accept HTML, so Webscale cannot return a CATCHA challenge page. Instead, it sends this error status response.
  • 499: Indicates the client has closed the connection while waiting for the response

HTTP Cache-Control headers

HTTP Cache-Control headers pertain to requests and responses that relate to cache. The headers consist of instructions that guide how to handle caching. Requests and responses include different types of information.

For example, a request with the max-age directive can use the header to request a copy of the resource that is less than an hour old. A response with this directive indicates that the response remains fresh until [N] seconds (equivalent to 59 minutes) after the response is generated.

Webscale caching

If you enable Webscale’s optional Dynamic Site Cache, Webscale proxies can cache pages and resources for your application. Thus, your web servers don’t have to cache them. By leaving the caching to Webscale, the result is improved performance, reliability, and scalability of your web sites and applications.

Webscale only caches resources according to the site cache rules that you configure and ignores HTTP Cache-Control headers. It does not modify or strip the headers unless you choose to do so by using a Webscale Web Control.

Webscale CDN caching

Webscale CDN Caching is handled differently if you enable the optional Webscale CDN feature. By default, Webscale CDN does not cache resources. CDN Controls enable you to configure how CDN caches. You can access CDN Controls through the Webscale Control Panel. You can configure Webscale CDN to not ignore Cache-Control header directives

Request and response discrepancies

If the caching policy of your servers differs from the Webscale site cache rules or Webscale CDN Controls, you might experience unexpected Webscale caching behavior. For example, by setting Cache-Control: no-store on your application responses, you are asking caches between your server and the customer’s browser to refrain from caching the response. However, the response may still be cached by Webscale CDN and site cache depending on your rules.

Mismatches are also possible between site cache rules and CDN Controls.

HTTP X-Forwarded-For Headers

HTTP X-Forwarded-For headers are requests for the original IP address of a client that is connecting to a web server through one or more proxies. The header instructions ask for the client’s IP address and, possibly, the IP addresses of the proxies.

This header can be used for debugging, statistics, and generating content relevant to a specific location. In Webscale, this IP address functions as the user’s request address. It primarily uses this request address for proxy logs. Webscale can also use it in Web Controls.

Directives

There are three directives for the X-Forwarded-For header. It must contain <client>, which stands for the client IP address.

Optionally, it can contain the <proxy1>, <proxy2> directives, which represent the IP addresses for the proxies that have handled the message. These directives are located to the right of the <client> directive.

When each proxy receives the request, it checks if the request includes the X-Forwarded-For header. If it does, it adds its own IP to the right part of the value of the header. If it doesn’t have one, it adds the header with its IP as the only value.

Example

In the following example, three proxies have handled the message before it comes to Webscale.

Client IP address: 192.0.0.11 Proxy IP addresses: 1.2.3.4,  2.2.2.2,  3.3.2.1 Header format: X-Forwarded-For: 192.0.0.11, 1.2.3.4,  2.2.2.2, 3.3.2.1

How Webscale processes X-Forwarded-For headers

The Webscale Trusted Proxies feature enables you to designate proxy IP addresses for providers that you trust. To enable this feature, select address sets and choose to enable Trusted Proxies.

Trusted proxies will add the X-Forwarder-For header to messages. If Webscale receives the message from a trusted proxy, it will use the encoded client IP address as the request address. If it receives the message from a proxy that it does not trust, it will strip the header and use the most recent peer address as the request address.

For more information on the Trusted Proxies feature, see How to Configure Trusted Proxies.

When you enable Webscale CDN, Webscale automatically adds an address set with the CDN IP addresses.

Further regarding


Last modified November 9, 2021