What’s New in ATS v9.x?
ATS v9.x is the current major release train, with many new features and improvements. This document describes new features and changes in these releases. Currently three minor versions have been released:
ATS v9.0.x (last version was
v9.0.2
)ATS v9.1.x (last version was
v9.1.4
)ATS v9.2.x (current stable version
v9.2.0
)
Keep in mind that only the latest (current) minor version is actively maintained, which is v9.2.x. This is expected to the the last minor relase in this train, with an upcoming v10.0.0 release.
Version 9.0.x Updates
New Features
This version of ATS has a number of new features (details below), but we’re particularly excited about the following features:
Experimental QUIC support (draft 27 and 29)
TLS v1.3 0RTT support
Significant HTTP/2 performance improvement
PROXY protocol support
Significant improvments to parent selection, including a new configuration file
strategies.yaml
Several new plugins
A new infrastructure and tool chain for end-to-end testing and replaying traffic is introduced, the Proxy Verifier.
PROXY protocol
ATS now supports the PROXY protocol, on the inbound side. The
incoming PROXY data gets transformed into the Forwarded
header.
Developer features
Add support for dtrace style markers (SDT) and include a few markers at locations of interest to users of SystemTap, dtrace, and gdb. See Debug Builds.
Command line tools
The traffic_server program now has two new maintenance commands:
verify_global_plugin
andverify_remap_plugin
. These commands load a plugin’s shared object file and verify it meets minimal global or remap plugin API requirements.
New configurations
Some new configurations were added, on existing features.
HTTP/2
The following new configurations are available to rate limit some potentially abusive clients.
Overall, the performance for HTTP/2 is significantly better in ATS v9.x.
Parent Selection
A new directive, ignore_self_detect, is added to the parent.config
format. This allows you to parent proxy to
sibling proxies, without creating loops. The setting for proxy.config.http.parent_proxy_routing_enable
is no longer needed, it’s implicit by the usage of the parent.config
configuration file.
A new option was added to parent.config
for which status codes triggers a simple retry,
simple_server_retry_responses.
A configuration file for the new parent selection strategy is added, strategies.yaml
.
SNI
A new option to control how host header and SNI name mismatches are handled has been added. With this new
setting, proxy.config.http.host_sni_policy
, when set to 0, no checking is performed. If this
setting is 1 or 2 (the default), the host header and SNI values are compared if the host header value
would have triggered a SNI policy.
A new blind tunneling option was added, partial_blind_route, which is similar to the existing forward_route feature.
The captured group from a FQDN matching in sni.yaml
can now be used in the tunnel_route, as e.g.
$1 and $2.
Plugin reload
A new setting was added to turn off the dynamic plugin reload, proxy.config.plugin.dynamic_reload_mode
.
By default, the feature is still enabled.
Updated host matching settings
The options for host matching configuration, ts:cv:proxy.config.http.server_session_sharing.match has been augmented significantly, the full list of host matches now are:
Value
Description
none
Do not match and do not re-use server sessions.
ip
Re-use server sessions, checking only that the IP address and port of the origin server matches.
host
Re-use server sessions, checking that the fully qualified domain name matches. In addition, if the session uses TLS, it also checks that the current transaction’s host header value matchs the session’s SNI.
both
Equivalent to
host,ip
.
hostonly
Check that the fully qualified domain name matches.
sni
Check that the SNI of the session matches the SNI that would be used to create a new session. Only applicable for TLS sessions.
cert
Check that the certificate file name used for the server session matches the certificate file name that would be used for the new server session. Only applicable for TLS sessions.
RAM cache per volumes
In volume.config
, you can now specify if a volume should have a RAM cache or not. This
can be particularly useful when a volume is assigned to a RAM disk already. The new option flag
is ramcache, with a value of true or false.
Incompatible records.config settings
These are the changes that are most likely to cause problems during an upgrade. Take special care making sure you have updated your configurations accordingly.
Connection management
The old settings for origin connection management included the following settings:
proxy.config.http.origin_max_connections
proxy.config.http.origin_max_connections_queue
proxy.config.http.origin_min_keep_alive_connections
These are all gone, and replaced with the following set of configurations:
proxy.config.http.per_server.connection.max
(overridable)proxy.config.http.per_server.connection.match
(overridable)
Logging and Metrics
In addition to logging indivdiual headers, we now have comparable log tags that dumbs the entire header. This table shows the additions, and what they correspond with.
Original Field
All Headers Variant
cqh
cqah
pqh
pqah
psh
psah
ssh
ssah
cssh
cssah
A new log field for the elliptic curve was introduced, cqssu. Also related to TLS, the new log field cssn allows logging the SNI server name from the client handshake.
New origin metrics
A large number of metrics were added for the various cases where ATS closes an origin connection. This includes:
Metric scaling
In previous versions of ATS, you had to rebuild the software to increase the maximum number of metrics that the system could handle. This is replaced with a new command line option for traffic_manager, –maxRecords. The old build configure option for this, –with-max-api-stats, is also eliminated.
TLS version metrics
A set of new metrics for SSL and TLS versions have been added:
proxy.process.ssl.ssl_total_sslv3
proxy.process.ssl.ssl_total_tlsv1
proxy.process.ssl.ssl_total_tlsv11
proxy.process.ssl.ssl_total_tlsv12
proxy.process.ssl.ssl_total_tlsv13
Plugins
statichit
This is a new, generic static file serving plugin. This can replace other plugins as well, for example the healthchecks.
maxmind
A new geo-ACL plugin was merged, maxmind. This should support the newer APIs from MaxMind, and should likely replace the old plugin in future release of ATS.
memory_profile
This is a helper plugin for debugging and analyzing memory usage with e.g. JEMalloc.
slice
The slice plugin has had a major overhaul, and has a significant number of new features and configurations. If you use the slice plugin, we recommend you take a look at the documentation again.
xdebug
A new directive, fwd=<n> to control the number of hops the header is forwarded for.
cert_reporting_tool
This is a new plugin to log examine and log information on loaded certificates.
Plugin APIs
The API for server push is promoted to stable, and modified to return an error code, to be consistent with other similar APIs. The new prototype is:
TSReturnCode TSHttpTxnServerPush(TSHttpTxn txnp, const char *url, int url_len);
A new set of APIs for scheduling continuations on a specific set of threads has been added:
TSAction TSContScheduleOnPool(TSCont contp, TSHRTime timeout, TSThreadPool tp)
TSAction TSContScheduleOnThread(TSCont contp, TSHRTime timeout, TSEventThread ethread)
There is a new API for redoing a cache lookup, typically after a URL change, or cache key update:
TSReturnCode TSHttpTxnRedoCacheLookup(TSHttpTxn txnp, const char *url, int length)
New APIs for TLS client context retrievals were added:
TSReturnCode TSSslClientContextsNamesGet(int n, const char **result, int *actual)
TSSslContext TSSslClientContextFindByName(const char *ca_paths, const char *ck_paths)
In addition to these, a new set of APIs were added for the effective TLS handshake with the client. These APIs also have equivalent Lua APIs:
int TSVConnIsSslReused(TSVConn sslp)
const char *TSVConnSslCipherGet(TSVConn sslp)
const char *TSVConnSslProtocolGet(TSVConn sslp)
const char *TSVConnSslCurveGet(TSVConn sslp)
We have also added two new alert mechanisms for plugins:
void TSEmergency(const char *fmt, ...) TS_PRINTFLIKE(1, 2)
void TSFatal(const char *fmt, ...) TS_PRINTFLIKE(1, 2)
A new API was added to allow control over whether the plugin will participate in the dynamic plugin reload mechanism:
TSReturnCode TSPluginDSOReloadEnable(int enabled)
User Arg Slots
The concept of user argument slots for plugins was completely redesigned. The old behavior still exists, but we now have a much cleaner, and smaller, set of APIs. In addition, it also adds a new slot type, global, which allows a slot for plugins to retain global data across reloads and cross-plugins.
The new set of APIs has the following signature:
typedef enum {
TS_USER_ARGS_TXN, ///< Transaction based.
TS_USER_ARGS_SSN, ///< Session based
TS_USER_ARGS_VCONN, ///< VConnection based
TS_USER_ARGS_GLB, ///< Global based
TS_USER_ARGS_COUNT ///< Fake enum, # of valid entries.
} TSUserArgType;
TSReturnCode TSUserArgIndexReserve(TSUserArgType type, const char *name, const char *description, int *arg_idx);
TSReturnCode TSUserArgIndexNameLookup(TSUserArgType type, const char *name, int *arg_idx, const char **description);
TSReturnCode TSUserArgIndexLookup(TSUserArgType type, int arg_idx, const char **name, const char **description);
void TSUserArgSet(void *data, int arg_idx, void *arg);
void *TSUserArgGet(void *data, int arg_idx);
One fundamental change here is that the opaque data parameter to TSUserArgSet and TSUserArgGet are context aware. If you pass in a Txn pointer, it behaves as a transaction user arg slot. If you pass in a nullptr, it becomes the new global slot behavior (since there is no context). The valid contexts are:
data type
Semantics
TSHttpTxn
The implicit context is for a transaction (
TS_USER_ARGS_TXN
)
TSHttpSsn
The implicit context is for a transaction (
TS_USER_ARGS_SSN
)
TSVConn
The implicit context is for a transaction (
TS_USER_ARGS_VCONN
)
nullptr
The implicit context is global (
TS_USER_ARGS_GLB
)
Version 9.2.x Updates
New Features
In addition to the new features from v9.0.x, this release includes:
A number of performance improvements have been made, and this should show noticeable gains.
Log throttling can now be controlled via two new settings.
A new feature to create and manage pre-warmed TLS tunnels was added.
The setting for
proxy.config.ssl.client.sni_policy
can now be controlled viasni.yaml
.New features, and overall improvements for Parent Selection were added.
There is now an internal inspector to produce stats for remap rule frequency.
BoringSSL is now properly supported, and should be a drop in replacement for OpenSSL.
A new allocator, mimalloc, is now available via the
--with-mimalloc
configure option.A large number of autest have been added, and improvements made to the existing tests.
A new infrastructure and tool chain for end-to-end testing and replaying traffic is introduced, the Proxy Verifier.
Log throttling
The throttling of logging is controlled by the new configurations:
This feature is useful to control logging pressure under traffic spikes, bad configurations or DDoS attacks.
Parent Selection Improvements
A setting to control the ring mode was added,
ring_mode
, with three possible settings: exhaust_ring, alternate_ring, or peering_ring.Two new settings are now available to control how parents are marked down. The configurations are
proxy.config.http.parent_proxy.enable_parent_timeout_markdowns
andproxy.config.http.parent_proxy.disable_parent_markdowns
.
Remap inspector
This feature can help identifying which remap rule is most frequently used. This helps when ordering remap rules, and also when removing unused rules. This is not well documented unfortunately, for some details see https://github.com/apache/trafficserver/pull/7936 .
HTTP/2
New metrics were added for streams status,
proxy.process.http2.max_concurrent_streams_exceeded_in
andproxy.process.http2.max_concurrent_streams_exceeded_out
.The new setting
proxy.config.http2.stream_error_sampling_threshold
allows for control over the sampling rate of stream errors.
New configuration options
The
proxy.config.cache.log.alternate.eviction
configuration allows control over logging of alternate eviction events. This is useful to get a better idea of how many alternates are being evicted, and to tune the setting.Better control over loop detection was added, via the
proxy.config.http.max_proxy_cycles
configuration.An option to control tunnel activity check frequencey was added,
proxy.config.tunnel.activity_check_period
. In addition, a new metric was added as well,proxy.process.tunnel.current_active_connections
.For debugging, a new option allowing to log TLS session keys was added, the new
proxy.config.ssl.keylog_file
setting specifies the log file to write to.The setting
proxy.config.net.sock_notsent_lowat
was added, allowing for better control over the TCP send buffer. This can both reduce memory usage and reduce buffer bloat.
Misc changes
The default for
proxy.config.ssl.origin_session_cache
is now 1 (enabled).A new log tag is available for custom log formats,
pqssr
. This is an indicator if TLS session reuse to origin was succesful.Two new metrics around loop detection was added,
proxy.process.http.http_proxy_loop_detected
andproxy.process.http.http_proxy_mh_loop_detected
.A metric
proxy.process.hostdb.total_serve_stale
now tracks HostDB lookups that served stale data.A new setting
proxy.config.track_config_files
allows for control over tracking configuration files for changes. This should be used with care.
Plugins
The following plugins have new features or configurations.
header_rewrite
A new directive
rm-destination
was added to remove either the query parameters or the PATH from the incoming request.A new
%{CACHE}
condition was added, exposing the cache lookup status on the request.The new directive
set-http-cntl
allows control over various HTTP transaction features, such as turning off logging, cacheability etc.The
set-body
directive lets you override the body factory, and force a body on requests that are considered errors early on (before going to cache or origin).As a special matcher (‘{}’), the inbound IP addresses can be matched against a list of IP ranges, e.g.
cond %{INBOUND:REMOTE-ADDR} {192.168.201.0/24,10.0.0.0/8}
.
generator
POST request are now handled as well as GET requests.
cache_promote
We can now count bytes served as the threshold within the LRU.
A new setting,
--internal-enabled
, was added to allow promotion on internal request.
xdebug
A new header X-Effective-URL
, to expose the effective (remapped) URL.
regex_revalidate
New metrics for misses and stale counts were added.
url_sig
Add a new directive, url_type = pristine
, to use the pristine URL for signing.
prefetch
A new option was added, --fetch-query
. This allows giving hints for subsequent
URLs via a query parameter.
authproxy
A new option to enable the cache for internal requests was added, --cache-internal
.
slice
The plugin now supports prefetching future ranges, similar to to the prefetch
plugin.
cache_range_requests
The setting
--verify-cacheability
now gives better control over what is cacheable. This is useful together with other plugins such as cache_promote.A new setting
--cache-complete-responses
now allows for “200 OK” responses to be cached as well.
rate_limit
Two new options were added for metrics,
--prefix
and--tag
.An IP reputation system was added to the plugin.
static_hit
The plugin can now serve content out of a directory.
otel_tracer
This is a new plugin for OpenTelemetry tracing.
http_stats
This new plugin provides stats over HTTP, but is implemented as a remap plugin. This should
make the older stats_over_http
obsolete.
Plugin APIs
The transction control APIs where refactored and promoted to that ts.h
public APIs. This adds
TSHttpTxnCntlGet()
and TSHttpTxnCntlSet()
, and the c:enum::TSHttpCntlType enum.
For introspection into the SNI, the new TSVConnSslSniGet()
was added.