How to Build Filters in the Traffic Viewer

How to build filters using the Traffic Viewer in the Webscale Control Panel

Traffic Viewer filter bar

The filter builder allows you to easily navigate records to find the specific logs interactions that interest you. As stated in How to Use the Traffic Viewer, the filter bar connects with each logs view. There are many ways to build filters; some methods are dependent on if you are in the records or group view.

General Filter Usage

Regardless of the current logs view type, you can write in filters directly to the filter bar.

  • Example Query: You can find an example query inside the filter bar to have an idea of how to model your query.
  • Filter Parameters: You can see the available search parameters by clicking on the info icon just above the filter bar.
  • Filter History: The filter history remembers the last twenty filter queries made during your current traffic viewer session.
  • Apply Filter: You can apply a filter by either clicking the “Apply” button or by pressing the enter key.
  • Filter Errors: If a query is malformed, an error appears below the bar to help guide corrections.
    Traffic Viewer filter error message example

Records View Filter Usage

Traffic Viewer filter options

All general filter usage principles apply to usage in the records view, along with a few additional features. You can access the options for any cell within the table by right-clicking the cell value or by clicking the dropdown arrow that appears on cell hover.

  • Add to Filter: You can add the cell value to the filter by selecting ‘Add to filter’ within the cell context menu. This action takes the cell value and header in the appropriate filter format and appends it to your current filter. This action stays within its view; it does not trigger a new logs view.
  • Replace Filter: You can replace your current filter entirely with a cell’s value by selecting “Replace filter” within the cell context menu. This action clears the current filter and replaces it with the cell value and header in the appropriate filter format. This action stays within its view; it does not trigger a new logs view.

Group View Filter Usage

Traffic Viewer group, show logs

Again, all general filter usage principles apply to usage in the group view, with one additional feature.

  • Show Logs: To see all logs that are within a specific group, you can select the arrow at the end of the line to show all logs within that group. This creates a new view with that grouping value and header appended to your current filter.

Filter Parameters

Below are the filterable parameters that can be used for the corresponding logs types.

General Filter Parameters

These are the filterable parameters that are used commonly across all logs.

Parameter Explanation
arrival The ISO8601-formatted date when the request to be processed was received.
country A two-letter ISO 3166-2 country code for the location where a request originated.
peer_address The peer at the other end of a request connection.
request_address The remote IP address of the user agent that made a request.
session_id The unique number that a server assigns a specific user for the duration of that session.
useragent The contents of the user agent request header within a record.

Traffic Viewer and Pageviews Filter Parameters

These are the filterable parameters that are common to the traffic viewer and pageviews logs.

Parameter Explanation
completed The ISO8601-formatted date when the processing of the request was completed.
labels A list of tags applied to a request via a web control.
load_id A unique ID to identify a pageload initiated by the user. The ID is shared between all resource requests to render the page.
request_host The contents of the host HTTP header for a request.
request_path The virtual path of a HTTP request. This is the latter portion of the request URL separated by backslashes after the main website name.
request_port The port number that received the request.
request_protocol The HTTP protocol version.
request_query The query parameters within an HTTP URL request located in the later sections of the request URL.
request_url The entire string containing the host, scheme, path, and query parameters for an HTTP request.
threat “Y” if the IP address is a potential attacker. “N” otherwise.
tls_cipher The encryption algorithms for the transport layer security.
tls_version The highest version of the transport layer security protocol the client supports.
useragent_device The type of device used to make a request, for example, smartphone, desktop, etc.
useragent_name The type of browser used to make a request, for example, Google Chrome, Safari, etc.
useragent_os The type of operating system used to make a request, for example, Android, Mac, etc.

Traffic Viewer and CSP Filter Parameters

These are the filterable parameters that are common to the traffic viewer and CSP logs.

Parameter Explanation
referrer The contents of the referrer request header within a request.
status_code The response code given by a server that helps identify the status of a request or error.

Traffic Viewer Filter Parameters

These are the filterable parameters that are unique to the traffic viewer logs.

Parameter Explanation
bytes_in Bytes received by the Webscale proxy, including the HTTP request line and headers.
bytes_out Bytes sent out by the Webscale proxy, including the headers.
cache_source Two-letter code returned by the source that handled the request. See the Cache Source section below for more details.
delivery_status Returns zero for normal traffic; when non-zero, the request was handled directly by the Webscale proxy. See the Delivery Status section below for more details.
elapsed Duration of the request being received by the proxy and the complete response being sent out.
internal_transport_latency The internal request processing delay on the proxy.
proxy_address The IP address of the proxy that handled a request.
queue_latency The time spent by the request being queued on the proxy due to a slow backend.
request_id A unique ID made for each and every request served by the proxy.
request_method The HTTP method for a request. Possible values are: GET, PATCH, POST, DELETE, PUT, etc.
response_content_type The outgoing content-type header specified in an HTTP response.
server_address The application server address used to fulfill a request.
server_latency Duration of the server receiving the request and the browser receiving the last byte of the response from the server.
ttfb Duration of the data plane receiving the request and sending the first byte of the response.
url_map The name of the url map that was used in a web control that resulted in the request being redirected.
webcontrols A list of the web control IDs applied to a logged request.

Cache Source

Source Code Explanation
NE It stands for “Not Eligible”. The request did not match any site cache conditions and pagespeed decided not to cache it as well.
PH It stands for “Pagespeed Hit”. The request was served from the cache when optimization was turned on and the resources required to render the page were present in the cache.
PM It stands for “Pagespeed Miss”. The request was not served from the cache when optimization was turned on and the resources required to render the page were not present in the cache.
PF It stands for “Pagespeed Fill”. The request initiated a pagespeed fill when optimization was turned on and the resources required to render the page were seen for the very first time.
SH It stand for “Site Cache Hit”. The request was served from the cache when site cache was turned on and the request matched a site cache condition which was present in the cache.
SM It stands for “Site Cache Miss”. The request was not served from the cache when site cache was turned on and the request matched a site cache condition which was not present in the cache.
SF It stands for “Site Cache Fill”. The request initiated a site cache fill when site cache was turned on and the request matched one of the site cache conditions for the very first time.

Delivery Status

All non-NULL values of the delivery status code indicate that the Webscale proxy server intercepted the request.

Status Code Explanation
0 Request was fulfilled by an application server.
1 Request was rejected because the source IP address was in the blacklist.
2 Request was rejected due to activated WAF rule uploaded by the user.
3 Request was rejected because it was sitting in the suspended queue for a time equal to or greater than maximum_queue_time.
4 Request was rejected because the proxy could not process the request as the suspended queue was full.
5 Shield mode was turned on and the request was presented with the CAPTCHA page.
6 Shield mode was turned on and the CAPTCHA page was validated successfully by the client.
7 Shield mode was turned on and an unsuccessful attempt was made by the user to verify the CAPTCHA successfully.
8 Shield mode was turned on and the client presented an invalid token. The request was presented with the CAPTCHA page.
9 Request was denied by the web control action “Deny Request”.
10 Request was redirected by either of the web control actions “Redirect Request” or “Redirect Using Map”.
11 Request had a mod_pagespeed_beacon.
12 Internal request.
13 Request was satisfied by Amazon Cloudfront.
14 Request is satisfied from the proxy. This includes, but is not necessarily limited to:
  • 421 misdirected errors where SNI host does not match HTTP header host
  • 502 bad gateway when the backend cannot be connected
  • 404 not found when the path contains escaped slashes (%2F)
  • 200 delivering robots.txt from the CDN hostname (vhost)
  • 405 method not allowed for TRACE method requests
  • 400 bad request for bad URLs
  • 403 backend certificate could not be validated
15 Request that is responsible for filling the cache, in case of a miss.
16 Request that can be satisfied from the cache either now or in the future.
  • If the page was cached then the request is satisfied from the cache.
  • If the page was not cached then a fill request will be initiated to fill the cache.
17 Request that had a bad port number.

Pageviews Filter Parameters

These are the filterable parameters that are unique to the pageviews logs.

Parameter Explanation
dom_content_loaded The time taken to completely load and parse the initial HTML document without waiting for stylesheets, images and subframes to finish loading.
dom_interactive The time taken before the browser’s parser finishes its work on the main document and the browser sets the readiness of the current document to interactive and the corresponding readystatechange event is emitted.
elapsed Duration of the load event. The load event is fired when the whole page has loaded, including all dependent resources such as stylesheets and images.
first_contentful_paint The time taken by the browser to render the first piece of DOM content after a user navigates to the page, thereby providing the first feedback to the user that the page is actually loading. Images, non-white <canvas> elements, SVGs on the page and text with pending webfonts are considered DOM content; anything inside an iframe is not included.
largest_contentful_paint The time taken for the largest content element in the viewport to be rendered to the screen.
ready_state_interactive The time taken for the loading state of the browser to become interactive. The loading state is interactive when the browser document has finished loading and it has been parsed but sub-resources such as scripts, images, stylesheets and frames are still loading.
time_to_first_byte Duration of the browser sending the request and receiving the first byte of the response.

CSP Filter Parameters

These are the filterable parameters that are unique to the CSP logs.

Parameter Explanation
blocked_uri The URI of the resource that was blocked from loading by the Content Security Policy. If the blocked URI is from a different origin than the document-uri, then the blocked URI is truncated to contain just the scheme, host, and port.
disposition Either “enforce” or “report” depending on whether the Content-Security-Policy-Report-Only header or the Content-Security-Policy header is used.
document_uri The URI of the document in which the violation occurred.
effective_directive The directive whose enforcement caused the violation.
original_policy The original policy as specified by the Content-Security-Policy HTTP header.
script_sample The first 40 characters of the inline script, event handler, or style that caused the violation.
violated_directive The name of the policy section that was violated.
host The URL of the application that was requested.

Examples

To see all traffic viewer results, use delivery_status>-1. From here, you can make changes to the query pattern and drill down the results to find specific types of traffic. For traffic viewer parameters that can accept regex, use the ~ (tilde) character before the regex you wish to match.

  • Find all instances of a 500 error: delivery_status>-1 and status_code>500.
  • To see all results from a specific IP address, use the request_address parameter: delivery_status>-1 and request_address=192.0.2.0
  • To check a specific URL on your application, use request_path: request_path="/checkout/cart/" and status_code>=500
  • Sort traffic based on useragent and use the bot useragent to see if bot traffic is an issue: delivery>-1 and useragent~*bot*
  • Find all php requests that are not going to standard index.php location: delivery_status>-1 and requested_path~*.php* and requested_path!~*index.php* and status_code>=400
  • Check traffic to the Magento admin page: delivery_status>-1 and request_path~*magento_admin_path*
  • Check customer post sql requests: delivery_status>-1 and request_method~*POST*
  • Isolate traffic from a specific country: delivery_status>-1 and country!~*US*
  • Global subnet-based requests: delivery_status>-1 and request_address~*192.0.2.0/16*

Further Reading

Have questions not answered here? Please Contact Support to get more help.


Last modified April 17, 2020