Background Fetch Plugin¶
This is a plugin for Traffic Server that allows you to proactively fetch content from Origin in a way that it will fill the object into cache. This is particularly useful when all (or most) of your client requests are of the byte-Range type. The underlying problem being that Traffic Server is not able to cache request / responses with byte ranges.
Using the plugin¶
This plugin functions as either a global or per remap plugin, and it takes an optional argument for specifying a config file with inclusion or exclusion criteria. The config file can be specified both via an absolute path or via a relative path to the install dir
To activate the plugin in global mode, in plugin.config
, simply add:
background_fetch.so --config <config-file>
To activate the plugin in per remap mode, in remap.config
, simply append the
below to the specific remap line:
@plugin=background_fetch.so @pparam=<config-file>
Functionality¶
Examining the responses from origin, we decide to trigger a background fetch of the original (Client) request under these conditions:
The request is a
GET
request (we only support these right now)The response is a
206
responseThe original client request, and the Origin server response, is clearly indicating that the response is cacheable. This uses the new API
TSHttpTxnIsCacheable()
, which also implies honoring current Traffic Server configurations.
Once deemed a good candidate to performance a background fetch, we’ll replay the original client
request through the Traffic Server proxy again, except this time eliminating the Range
header.
This is transparent to the original client request, which continues as normal.
Only one background fetch per URL is ever performed, making sure we do not accidentally put pressure on the origin servers.
The plugin now supports a config file that can specify exclusion or inclusion of background fetch based on any arbitrary header or client-ip:
background_fetch.so --config <config-file>
The contents of the config-file could be as below:
include User-Agent ABCDEF
exclude User-Agent *
exclude Content-Type text
exclude X-Foo-Bar text
exclude Content-Length <1000
exclude Client-IP 127.0.0.1
include Client-IP 10.0.0.0/16
Important
The include
configuration directive is only used when there is a corresponding exclude
to exempt.
For example, a single line directive, include Host example.com
would not make the plugin
only act on example.com. To achieve classic allow (only) lists, one would need to have a broad
exclude line, such as:
exclude Host *
include Host example.com
The plugin also now supports per remap activation. To activate the plugin for a given remap, add the below on the remap line:
@plugin=background_fetch.so @pparam=<config-file>
Dealing with origins that don’t support range requests¶
The background fetch plugin will not properly work if an origin does not support range requests and returns a 200 instead of a 206.
In this case header rewrites can be used to force the background fetch to trigger.
given a remap.config line like:
map <source> <dest> \
... \
@plugin=header_rewrite.so @pparam=hdr_rw_prev.config \
@plugin=background_fetch.config @pparam=bg_fetch.config \
@plugin=header_rewrite.so @pparam=hdr_rw_next.config \
...
hdr_rw_prev.config:
cond %{READ_RESPONSE_HDR_HOOK}
cond %{CLIENT-HEADER:Range} = "" [NOT]
cond %{STATUS} = 200
set-status 206
set-header @Was200 true
hdr_rw_next.config:
cond %{READ_RESPONSE_HDR_HOOK}
cond %{HEADER:@Was200} = "" [NOT]
set-status 200
Will cause a 200 parent/origin response from a client range request to trigger a background fetch.
Future additions¶
Limiting the background fetches to content of certain sizes.