Formatting Log Files¶

This section covers the creation of logging formats. All but a few logging output related settings in Traffic Server are performed in logging.yaml and consulting the documentation for that file is recommended in addition to this section. Any configurations or settings performed outside that file will be clearly noted.

Defining Formats¶

Logging formats in Traffic Server are defined by editing logging.yaml and adding new format entries for each format you wish to define. The syntax is fairly simple: every format must contain a Format attribute, which is the string defining the contents of each line in the log, and may also contain an optional Interval attribute defining the log aggregation interval for any logs which use the format (see Summary Logs for more information). It may also contain an optional escape attribute defining the escape of log fields. Possible values for escape are json or none.

The return value from the format function is the log format object which may then be supplied to the appropriate log.* functions that define your logging destinations.

A very simple example, which contains only the timestamp of when the event began and the canonical URL of the request, would look like:

formats:
- name: myformat
  format: '%<cqtq> %<cauc>'

You may include as many custom field codes as you wish. The full list of codes available can be found in Log Fields. You may also include any literal characters in your format. For example, if we wished to separate the timestamp and canonical URL in our customer format above with a slash instead of a space, or even a slash surrounded by spaces, we could do so by just adding the desired characters to the format string:

formats:
- name: myformat
  format: '%<cqtq> / %<cauc>'

You may define as many custom formats as you wish. To apply changes to custom formats, you will need to run the command traffic_ctl config reload after saving your changes to logging.yaml.

Log Fields¶

The following sections detail all the available Traffic Server logging fields, broken down into the following broad categories for (hopefully) easier reference:

Authentication
Cache Details
Connections and Transactions
Content Types
Hierarchical Proxies
HTTP Headers
HTTP Methods
Identifiers
Lengths and Sizes
Network Addresses, Ports, and Interfaces
Plugin Details
Protocols and Versions
Request Details
SSL / Encryption
Status Codes
TCP Details
Timestamps and Durations
URLs, Schemes, and Paths

Individual log fields are used within a log format string by enclosing them in angle brackets and prefixing with a percent sign. For example, to use the log field cqhl (the length in bytes of the client request headers), you would do the following:

Format = '%<cqhl>'

Literal characters may be used, but they must be outside the log fields’ placeholders, as so:

Format = 'Client Header Length (bytes): %<cqhl>'

You may combine many fields into a single format string (logs wouldn’t be very useful if you couldn’t). Some fields do require a little extra treatment, which is noted clearly in their descriptions below. This affects, primarily, those fields which provide access to HTTP header values as you need to specify which header’s value you wish to appear in the log data. For these, the header name is noted inside the angle brackets, before the log field name, and are enclosed within a curly braces pair. For example, to include the value of the Age header from an origin server response you would do:

Format = '%<{Age}ssh>'

Authentication¶

These log fields provide access to various details of a client or proxy’s means of request authentication to their destination (whether it be the client request to a proxy server, or the proxy server’s request to an origin).

Field	Source	Description
caun	Client Request	Authentication User name as a result of the RFC931/ident lookup for the client-provided name.

Cache Details¶

These log fields reveal details of the Traffic Server proxy interaction with its own cache while attempting to service incoming client requests.

Field	Source	Description
cluc	Client Request	Cache Lookup URL, also known as the cache key, which is the canonicalized version of the client request URL.
crc	Proxy Cache	Cache Result Code. The result of Traffic Server attempting to obtain the object from cache; Logging Cache Results.
crsc	Proxy Cache	Cache Result Sub-Code. More specific code to complement the Cache Result Code.
chm	Proxy Cache	Cache Hit-Miss status. Specifies the level of cache from which this request was served by Traffic Server. Currently supports only RAM (`HIT_RAM`), disk (`HIT_DISK`) and Read While Write (`HIT_RWW`). HIT_RWW could imply either HIT_RAM or HIT_DISK.
cwr	Proxy Cache	Cache Write Result. Specifies the result of attempting to write to cache: not relevant (`-`), no cache write (`WL_MISS`), write interrupted (`INTR`), error while writing (`ERR`), or cache write successful (`FIN`).
cwt	Proxy Cache	Cache Write Transform Result.
crra	Proxy Cache	Cache read retry attempts to read the object from cache.
cwra	Proxy Cache	Cache write retry attempts to write a fresh or updated object to cache.
cccs	Proxy Cache	Cache collapsed connection success; -1: collapsing was attempted but failed, request went upstream 0: collapsing was unnecessary 1: attempted to collapse and got a cache hit on subsequent read attempts

Connections and Transactions¶

The following log fields are used to list various details of connections and transactions between Traffic Server proxies and origin servers.

Field	Source	Description
sca	Proxy	Number of attempts within the current transaction by Traffic Server in connecting to the origin server.
surc	Proxy	Parent unavailable retry count within the current transaction by Traffic Server.
ssrc	Proxy	Parent simple server retry count within the current transaction by Traffic Server.
sstc	Proxy	Number of transactions between the Traffic Server proxy and the origin server from a single session. Any value greater than zero indicates connection reuse.
ccid	Client Request	Client Connection ID, a non-negative number for a connection, which is different for all currently-active connections to clients.
ctid	Client Request	Client Transaction ID, a non-negative number for a transaction, which is different for all currently-active transactions on the same client connection. For client HTTP/2 transactions, this value is the stream ID for the transaction.
ctpw	Client Request	Client Transaction Priority Weight, the priority weight for the underlying HTTP/2 protocol.
ctpd	Client Request	Client Transaction Priority Dependence, the transaction ID that the current transaction depends on for HTTP/2 priority logic.

Content Types¶

Log fields used to reveal the HTTP content types in effect for transactions.

Field	Source	Description
psct	Origin Server Response	Content type of the document obtained by Traffic Server from the origin server response.

Error Code¶

The log fields of error code which is triggered session close or transaction close. The first byte of this field indicates that the error code is session level (S) or transaction level (T). When no error code is received or transmitted, these fields are -. For HTTP/2, error code are described in RFC 7540 section 7.

Field	Source	Description
crec	Client Request	Error code in hex which Traffic Server received
ctec	Client Response	Error code in hex which Traffic Server transmitted

Hierarchical Proxies¶

The log fields detail aspects of transactions involving hierarchical caches.

Field	Source	Description
phr	Proxy	Proxy Hierarchy Route. Specifies the route through configured hierarchical caches used to retrieve the object.

HTTP Headers¶

The following log tables provide access to the values of specified HTTP headers from each phase of the transaction lifecycle. Unlike many of the other log fields, these require a little extra notation in the log format string, so that Traffic Server knows the individual HTTP header from which you aim to extract a value for the log entry.

This is done by specifying the name of the HTTP header in curly braces, just prior to the log field’s name, as so:

Format = '%<{User-agent}cqh>'

The above would insert the User Agent string from the client request headers into your log entry (or a blank string if no such header was present, or it did not contain a value).

Field	Source	Description
cqh	Client Request	Logs the value of the named header from the client’s request to the Traffic Server proxy.
pqh	Proxy Request	Logs the value of the named header from the Traffic Server proxy’s request to the origin server.
psh	Proxy Response	Logs the value of the named header from the Traffic Server proxy’s response to the client.
ssh	Origin Response	Logs the value of the named header from the origin server’s response to the proxy.
cssh	Cached Origin Response	Logs the value of the named header from the cached origin server response.

Each of these also includes a URI-encoded variant, which replaces various characters in the string with entity encodings - rendering them safe for use in URL path components or query parameters. The variants’ names follow the pattern of the origin field named prefixed with e, as shown here:

Original Field	URL-Encoded Variant
cqh	ecqh
pqh	epqh
psh	epsh
ssh	essh
cssh	ecssh

It is also possible to log all of the headers in a transaction message with a single field. For each original field, there is a variant which ends in ah rather than h, as shown here:

Original Field	All Headers Variant
cqh	cqah
pqh	pqah
psh	psah
ssh	ssah
cssh	cssah

No particular header is specified when using these variants, for example:

Format = '%<cqah>'

The output generated by these fields has the pattern:

{{{tag1}:{value1}}{{tag2}:{value2}}...}

(The size of some messages may exceed internal buffer capacity. This may result in the value of the last header being truncated, in which case, the value will end with ...}. This may also result in the omission of entire tag/value pairs.)

HTTP Methods¶

These fields are used to log information about the HTTP methods/verbs used by requests.

Field	Source	Description
cqhm	Client Request	HTTP method used in the client request to the Traffic Server proxy (e.g. `GET`, `POST`, etc.).

Identifiers¶

Logging fields used to obtain various unique identifiers for transactions or objects.

Field	Source	Description
crid	Client Request	Sequence number of the current client request. Resets to `0` on every Traffic Server restart.
cruuid	Client Request	UUID of the current client request; generated by concatenating the puuid and crid field values.
puuid	Proxy Server	UUID for the currently running traffic_server process. Regenerated on every Traffic Server startup.

Lengths and Sizes¶

These log fields are used to obtain various lengths and sizes of transaction components (headers, content bodies, etc.) between clients, proxies, and origins. Unless otherwise noted, all lengths are in bytes.

Field	Source	Description
cqcl	Client Request	Client request content length, in bytes.
cqhl	Client Request	Client request header length, in bytes.
cqql	Client Request	Client request header and content length combined, in bytes.
csscl	Cached Origin Response	Content body length from cached origin response.
csshl	Cached Origin Response	Header length from cached origin response.
cssql	Cached Origin Response	Content and header length from cached origin response.
fsiz	Origin	Size of the file as seen by the origin server.
pqcl	Proxy Request	Content body length of the Traffic Server proxy request to the origin server.
pqhl	Proxy Request	Header length of the Traffic Server proxy request to the origin server.
pqql	Proxy Request	Content body and header length combined, of the Traffic Server request to the origin server.
pscl	Proxy Response	Content body length of the Traffic Server proxy response.
pshl	Proxy Response	Header length of the Traffic Server response to client.
psql	Proxy Response	Content body and header length combined of the Traffic Server response to client.
sscl	Origin Response	Content body length of the origin server response to Traffic Server.
sshl	Origin Response	Header length of the origin server response.
ssql	Origin Response	Content body and header length combined of the origin server response to Traffic Server.

Network Addresses, Ports, and Interfaces¶

The following log fields are used to log details of the network (IP) addresses, incoming/outgoing ports, and network interfaces used during transactions.

Field	Source	Description
chi	Client	IP address of the client’s host. If Proxy Protocol is used, this represents the IP address of the previous hop.
chih	Client	IP address of the client’s host, in hexadecimal.
hii	Proxy	IP address for the proxy’s incoming interface (to which the client connected).
hiih	Proxy	IP address for the proxy’s incoming interface (to which the client connected), in hexadecimal.
chp	Client	Port number of the client’s host.
php	Proxy Response	TCP port number from which Traffic Server serviced the request.
pqsi	Proxy Request	IP address from which Traffic Server issued the proxy request to the origin server. Cache hits will result in a value of `0`.
pqsp	Proxy Request	Port number from which Traffic Server issued the proxy request to the origin server. Cache hits will yield a value of `0`.
shi	Origin Server	IP address resolved via DNS by Traffic Server for the origin server. For hosts with multiple IP addresses, the address used by Traffic Server for the connection will be reported. See note below regarding misleading values from cached documents.
shn	Origin Server	Host name of the origin server.
nhi	Origin Server	Destination IP address of next hop
nhp	Origin Server	Destination port of next hop
ppv	Proxy Protocol Version	Proxy Protocol Version used (if any) between the Load Balancer and Traffic Server
pps	Proxy Protocol Source IP	Source IP received via Proxy Protocol context from the LB to the Traffic Server
ppd	Proxy Protocol Dest IP	Destination IP received via Proxy Protocol context from the LB to the Traffic Server

Note

This can be misleading for cached documents. For example: if the first request was a cache miss and came from IP1 for server S and the second request for server S resolved to IP2 but came from the cache, then the log entry for the second request will show IP2.

Plugin Details¶

Logging fields which may be used to obtain details of plugins involved in the transaction.

Field	Source	Description
piid	Proxy Plugin	Plugin ID for the current transaction. This is set for plugin driven transactions via `TSHttpConnectWithPluginId()`. or `TSHttpConnectPlugin()`.
pitag	Proxy Plugin	Plugin tag for the current transaction. This is set for plugin driven transactions via `TSHttpConnectWithPluginId()`. or `TSHttpConnectPlugin()`.
cqint	Client Request	If a request was generated internally (via a plugin), then this has a value of `1`, otherwise `0`. This can be useful when tracking internal only requests, such as those generated by the `authproxy` plugin.

Protocols and Versions¶

These logging fields may be used to determine which protocols and/or versions were in effect for a given event.

Field	Source	Description
cqpv	Client Request	Client request protocol and version.
sqpv	Proxy Request	Origin negotiated protocol and version
csshv	Cached Proxy Response	Origin server’s HTTP version from cached version of the document in Traffic Server proxy cache.
sshv	Origin Response	Origin server’s response HTTP version.

Request Details¶

The following logging fields show the details of the actual HTTP request and can be used in combination to reproduce a client request line (e.g. GET http://www.company.com HTTP/1.0).

Field	Source	Description
cqhm	Client Request	HTTP method used in the client request to the Traffic Server proxy (e.g. `GET`, `POST`, etc.).
pqu	Proxy Request	URL of the proxy request from Traffic Server to the origin. If a remap rule is used (`remap.config`), the original client URL is replaced with the URL in the target of the remap rule.
cqpv	Client Request	Client request protocol and version.

SSL / Encryption¶

Fields which expose the use, or lack thereof, of specific SSL and encryption features.

Field	Source	Description
cssn	Client TLS Hello	SNI server name in client Hello message in TLS handshake. If no server name present in Hello, or the transaction was not over TLS (over TCP), this field will contain `-`.
cscert	Client Request	1 if Traffic Server requested certificate from client during TLS handshake. 0 otherwise.
cqssl	Client Request	SSL client request status indicates if this client connection is over SSL.
cqssr	Client Request	SSL session ticket reused status; indicates if the current request hit the SSL session ticket and avoided a full SSL handshake.
cqssv	Client Request	SSL version used to communicate with the client.
cqssc	Client Request	SSL Cipher used by Traffic Server to communicate with the client.
cqssu	Client Request	SSL Elliptic Curve used by Traffic Server to communicate with the client when using an ECDHE cipher.
cqssa	Client Request	ALPN Protocol ID negotiated with the client.
pqssl	Proxy Request	Indicates whether the connection from Traffic Server to the origin was over SSL or not.
pqssr	Proxy Request	SSL session ticket reused status from Traffic Server to the origin; indicates if the current request hit the SSL session ticket and avoided a full SSL handshake.
pscert	Proxy Request	1 if origin requested certificate from Traffic Server during TLS handshake but no client certificate was defined. 2 if origin requested certificate from Traffic Server during TLS handshake and a client certificate was defined. 0 otherwise.

Status Codes¶

These log fields provide a variety of status codes, some numeric and some as strings, relating to client, proxy, and origin transactions.

Field	Source	Description
cfsc	Client Request	Finish status code specifying whether the client request to Traffic Server was successfully completed (`FIN`) or interrupted (`INTR`).
csssc	Cached Proxy Response	HTTP response status code of the origin server response, as cached by Traffic Server.
pfsc	Proxy Request	Finish status code specifying whether the proxy request from Traffic Server to the origin server was successfully completed (`FIN`), interrupted (`INTR`), or timed out (`TIMEOUT`).
prrp	Proxy Response	HTTP response reason phrase sent by Traffic Server proxy to the client.
pssc	Proxy Response	HTTP response status code sent by Traffic Server proxy to the client.
sssc	Origin Response	HTTP response status code sent by the origin server to the Traffic Server proxy.

TCP Details¶

The following logging fields reveal information about the TCP layer of client, proxy, and origin server connections.

Field	Source	Description
cqtr	Client Request	TCP reused status of the connection between the client and Traffic Server proxy, indicating whether the request was delivered through an already established connection.
cqmpt	Client Request	Indicates the MPTCP state of the connection. `-1` means MPTCP was not enabled on the listening port, whereas `0` and `1` indicates whether MPTCP was successfully negotiated or not.

Timestamps and Durations¶

The logging fields expose a variety of timing related information about client, proxy, and origin transactions. Variants of some of the fields provide timing resolution of the same underlying detail in milliseconds and seconds (both fractional and rounded-down integers). These variants are particularly useful in accommodating the emulation of other HTTP proxy softwares’ logging formats.

Other fields in this category provide variously formatted timestamps of particular events within the current transaction (e.g. the time at which a client request was received by Traffic Server).

Field	Source	Description
cqtd	Client Request	Client request timestamp. Specifies the date of the client request in the format `YYYY-MM-DD` (four digit year, two digit month, two digit day - with leading zeros as necessary for the latter two).
cqtn	Client Request	Client request timestamp in the Netscape timestamp format.
cqtq	Client Request	The time at which the client request was received expressed as fractional (floating point) seconds since midnight January 1, 1970 UTC (epoch), with millisecond resolution.
cqts	Client Request	Same as cqtq, but as an integer without sub-second resolution.
cqth	Client Request	Same as cqts, but represented in hexadecimal.
cqtt	Client Request	Client request timestamp in the 24-hour format `hh:mm:ss` (two digit hour, minutes, and seconds - with leading zeros as necessary).
crat	Origin Response	Retry-After time in seconds if specified in the origin server response.
ms	Proxy	Timestamp in milliseconds of a specific milestone for this request. See note below about specifying which milestone to use.
msdms	Proxy	Difference in milliseconds between the timestamps of two milestones. See note below about specifying which milestones to use.
stms	Proxy-Origin Connection	Time (in milliseconds) spent accessing the origin server. Measured from the time the connection between proxy and origin is established to the time it was closed.
stmsh	Proxy-Origin Connection	Same as stms, but represented in hexadecimal.
stmsf	Proxy-Origin Connection	Same as stms, but in fractional (floating point) seconds.
sts	Proxy-Origin Connection	Same as stms, but in integer seconds (no sub-second precision).
ttms	Client-Proxy Connection	Time in milliseconds spent by Traffic Server processing the entire client request. Measured from the time the connection between the client and Traffic Server proxy was established until the last byte of the proxy response was delivered to the client.
ttmsh	Client-Proxy Connection	Same as ttms, but represented in hexadecimal.
ttmsf	Client-Proxy Connection	Same as ttms, but in fraction (floating point) seconds.
tts	Client Request	Same as ttms, but in integer seconds (no sub-second precision).

Note

Logging fields for transaction milestones require specifying which of the milestones to use. Similar to how header logging fields are used, these log fields take the milestone name(s) in between curly braces, immediately before the logging field name, as so:

%<{Milestone field name}ms>
%<{Milestone field name1-Milestone field name2}msdms>

For more information on transaction milestones in Traffic Server, refer to the documentation on TSHttpTxnMilestoneGet().

URLs, Schemes, and Paths¶

These log fields allow capture of URLs, or components (such as schemes and paths), from transactions processed by Traffic Server.

Field	Source	Description
cqu	Proxy Request	URL of the proxy request from Traffic Server to the origin. If a remap rule is used (`remap.config`), the original client URL is replaced with the URL in the target of the remap rule. Deprecated since 10.0. Use `pqu` instead.
cquc	Proxy Request	Canonical URL from the proxy request to origin. This field differs from cqu by having its contents URL-escaped (spaces and various other characters are replaced by percent-escaped entity codes). Deprecated since 10.0. Use `pquc` instead.
cqup	Proxy Request	Path component from the proxy request. Deprecated since 10.0. Use `pqup` instead.
cqus	Proxy Request	URL scheme from the proxy request. Deprecated since 10.0. Use `pqus` instead.
pqu	Proxy Request	URL of the proxy request from Traffic Server to the origin. If a remap rule is used (`remap.config`), the original client URL is replaced with the URL in the target of the remap rule.
pquc	Proxy Request	Canonical URL from the proxy request to origin. This field differs from pqu by having its contents URL-escaped (spaces and various other characters are replaced by percent-escaped entity codes).
pqup	Proxy Request	Path component from the proxy request.
pqus	Proxy Request	URL scheme from the proxy request.
cquuc	Client Request	Canonical (prior to remapping) effective URL from client request. It is possible that plugins prior to the remap phase have changed the client request, so this field may not match the original incoming client request.
cquup	Client Request	Canonical (prior to remapping) path component from the client request. Compare with cqup.
cquuh	Client Request	Unmapped URL host from the client request.

Line Length¶

The maximum line size for a log entry can be configured via the following parameters, the details of which are documented in the linked records.yaml descriptions:

Log Field Slicing¶

It is sometimes desirable to slice a log field to limit the length of a given log field’s output.

Log Field slicing can be specified as below:

%<field[start:end]>
%<{field}container[start:end]>

Omitting the slice notation defaults to the entire log field.

Slice notation only applies to a log field that is of type string and can not be applied to IPs or timestamp which are converted to string from integer.

The below slice specifiers are allowed.

[start:end]: Log field value from start through end-1
[start:]: Log field value from start through the rest of the string
[:end]: Log field value from the beginning through end-1
[:]: Default - entire Log field

Some examples below

'%<pqup>'       // the whole characters of <pqup>.
'%<pqup>[:]'    // the whole characters of <pqup>.
'%<pqup[0:30]>' // the first 30 characters of <pqup>.
'%<pqup[-10:]>' // the last 10 characters of <pqup>.
'%<pqup[:-5]>'  // everything except the last 5 characters of <pqup>.

Note when escape in format is set to json, the start is the position before escaping JSON strings, and escaped values are sliced at the length (= end - start). If slicing cuts in the middle of escaped characters, the whole character is removed.