What’s New in ATS v9.x

This version of ATS includes over <x> commits, from <y> pull requests. A total of <z> contributors have participated in this development cycle.

New Features

This version of ATS has a number of new features (details below), but we’re particularly excited about the following features:

• Experimental QUIC support (draft 27 and 29)

• TLS v1.3 0RTT support

• Significant HTTP/2 performance improvement

• PROXY protocol support

• Significant improvments to parent selection, including a new configuration file strategies.yaml

• Several new plugins

A new infrastructure and tool chain for end-to-end testing and replaying traffic is introduced, the Proxy Verifier.

PROXY protocol

ATS now supports the PROXY protocol, on the inbound side. The incoming PROXY data gets transformed into the Forwarded header.

Developer features

• Add support for dtrace style markers (SDT) and include a few markers at locations of interest to users of SystemTap, dtrace, and gdb. See Debug Builds.

Command line tools

• The traffic_server program now has two new maintenance commands: verify_global_plugin and verify_remap_plugin. These commands load a plugin’s shared object file and verify it meets minimal global or remap plugin API requirements.

New configurations

Some new configurations were added, on existing features.

HTTP/2

The following new configurations are available to rate limit some potentially abusive clients.

• proxy.config.http2.max_settings_per_frame

• proxy.config.http2.max_settings_per_minute

Overall, the performance for HTTP/2 is significantly better in ATS v9.x.

Parent Selection

A new directive, ignore_self_detect, is added to the parent.config format. This allows you to parent proxy to sibling proxies, without creating loops. The setting for proxy.config.http.parent_proxy_routing_enable is no longer needed, it’s implicit by the usage of the parent.config configuration file.

A new option was added to parent.config for which status codes triggers a simple retry, simple_server_retry_responses.

A configuration file for the new parent selection strategy is added, strategies.yaml.

SNI

A new option to control how host header and SNI name mismatches are handled has been added. With this new setting, proxy.config.http.host_sni_policy, when set to 0, no checking is performed. If this setting is 1 or 2 (the default), the host header and SNI values are compared if the host header value would have triggered a SNI policy.

A new blind tunneling option was added, partial_blind_route, which is similar to the existing forward_route feature.

The captured group from a FQDN matching in sni.yaml can now be used in the tunnel_route, as e.g. $1 and $2.

Plugin reload

A new setting was added to turn off the dynamic plugin reload, proxy.config.plugin.dynamic_reload_mode. By default, the feature is still enabled.

Updated host matching settings

The options for host matching configuration, ts:cv:proxy.config.http.server_session_sharing.match has been augmented significantly, the full list of host matches now are:

Value	Description
none	Do not match and do not re-use server sessions.
ip	Re-use server sessions, checking only that the IP address and port of the origin server matches.
host	Re-use server sessions, checking that the fully qualified domain name matches. In addition, if the session uses TLS, it also checks that the current transaction’s host header value matchs the session’s SNI.
both	Equivalent to host,ip.
hostonly	Check that the fully qualified domain name matches.
sni	Check that the SNI of the session matches the SNI that would be used to create a new session. Only applicable for TLS sessions.
cert	Check that the certificate file name used for the server session matches the certificate file name that would be used for the new server session. Only applicable for TLS sessions.

RAM cache per volumes

In volume.config, you can now specify if a volume should have a RAM cache or not. This can be particularly useful when a volume is assigned to a RAM disk already. The new option flag is ramcache, with a value of true or false.

Incompatible records.config settings

These are the changes that are most likely to cause problems during an upgrade. Take special care making sure you have updated your configurations accordingly.

Connection management

The old settings for origin connection management included the following settings:

• proxy.config.http.origin_max_connections

• proxy.config.http.origin_max_connections_queue

• proxy.config.http.origin_min_keep_alive_connections

These are all gone, and replaced with the following set of configurations:

• proxy.config.http.per_server.connection.max (overridable)

• proxy.config.http.per_server.connection.match (overridable)

• proxy.config.http.per_server.connection.alert_delay

• proxy.config.http.per_server.connection.queue_size

• proxy.config.http.per_server.connection.queue_delay

• proxy.config.http.per_server.connection.min

Logging and Metrics

In addition to logging indivdiual headers, we now have comparable log tags that dumbs the entire header. This table shows the additions, and what they correspond with.

Original Field	All Headers Variant
cqh	cqah
pqh	pqah
psh	psah
ssh	ssah
cssh	cssah

A new log field for the elliptic curve was introduced, cqssu. Also related to TLS, the new log field cssn allows logging the SNI server name from the client handshake.

New origin metrics

A large number of metrics were added for the various cases where ATS closes an origin connection. This includes:

• proxy.process.http.origin_shutdown.pool_lock_contention

• proxy.process.http.origin_shutdown.migration_failure

• proxy.process.http.origin_shutdown.tunnel_server

• proxy.process.http.origin_shutdown.tunnel_server_no_keep_alive

• proxy.process.http.origin_shutdown.tunnel_server_eos

• proxy.process.http.origin_shutdown.tunnel_server_plugin_tunnel

• proxy.process.http.origin_shutdown.tunnel_transform_read

• proxy.process.http.origin_shutdown.release_no_sharing

• proxy.process.http.origin_shutdown.release_no_keep_alive

• proxy.process.http.origin_shutdown.release_invalid_response

• proxy.process.http.origin_shutdown.release_invalid_request

• proxy.process.http.origin_shutdown.release_modified

• proxy.process.http.origin_shutdown.release_misc

• proxy.process.http.origin_shutdown.cleanup_entry

• proxy.process.http.origin_shutdown.tunnel_abort

Metric scaling

In previous versions of ATS, you had to rebuild the software to increase the maximum number of metrics that the system could handle. This is replaced with a new command line option for traffic_manager, –maxRecords. The old build configure option for this, –with-max-api-stats, is also eliminated.

TLS version metrics

A set of new metrics for SSL and TLS versions have been added:

• proxy.process.ssl.ssl_total_sslv3

• proxy.process.ssl.ssl_total_tlsv1

• proxy.process.ssl.ssl_total_tlsv11

• proxy.process.ssl.ssl_total_tlsv12

• proxy.process.ssl.ssl_total_tlsv13

Plugins

statichit

This is a new, generic static file serving plugin. This can replace other plugins as well, for example the healthchecks.

maxmind

A new geo-ACL plugin was merged, maxmind. This should support the newer APIs from MaxMind, and should likely replace the old plugin in future release of ATS.

memory_profile

This is a helper plugin for debugging and analyzing memory usage with e.g. JEMalloc.

slice

The slice plugin has had a major overhaul, and has a significant number of new features and configurations. If you use the slice plugin, we recommend you take a look at the documentation again.

xdebug

• A new directive, fwd=<n> to control the number of hops the header is forwarded for.

cert_reporting_tool

This is a new plugin to log examine and log information on loaded certificates.

Plugin APIs

The API for server push is promoted to stable, and modified to return an error code, to be consistent with other similar APIs. The new prototype is:

TSReturnCode TSHttpTxnServerPush(TSHttpTxn txnp, const char *url, int url_len);

A new set of APIs for scheduling continuations on a specific set of threads has been added:

TSAction TSContScheduleOnPool(TSCont contp, TSHRTime timeout, TSThreadPool tp)
TSAction TSContScheduleOnThread(TSCont contp, TSHRTime timeout, TSEventThread ethread)

There is a new API for redoing a cache lookup, typically after a URL change, or cache key update:

TSReturnCode TSHttpTxnRedoCacheLookup(TSHttpTxn txnp, const char *url, int length)

New APIs for TLS client context retrievals were added:

TSReturnCode TSSslClientContextsNamesGet(int n, const char **result, int *actual)
TSSslContext TSSslClientContextFindByName(const char *ca_paths, const char *ck_paths)

In addition to these, a new set of APIs were added for the effective TLS handshake with the client. These APIs also have equivalent Lua APIs:

int TSVConnIsSslReused(TSVConn sslp)
const char *TSVConnSslCipherGet(TSVConn sslp)
const char *TSVConnSslProtocolGet(TSVConn sslp)
const char *TSVConnSslCurveGet(TSVConn sslp)

We have also added two new alert mechanisms for plugins:

void TSEmergency(const char *fmt, ...) TS_PRINTFLIKE(1, 2)
void TSFatal(const char *fmt, ...) TS_PRINTFLIKE(1, 2)

A new API was added to allow control over whether the plugin will participate in the dynamic plugin reload mechanism:

TSReturnCode TSPluginDSOReloadEnable(int enabled)

User Arg Slots

The concept of user argument slots for plugins was completely redesigned. The old behavior still exists, but we now have a much cleaner, and smaller, set of APIs. In addition, it also adds a new slot type, global, which allows a slot for plugins to retain global data across reloads and cross-plugins.

The new set of APIs has the following signature:

typedef enum {
   TS_USER_ARGS_TXN,   ///< Transaction based.
   TS_USER_ARGS_SSN,   ///< Session based
   TS_USER_ARGS_VCONN, ///< VConnection based
   TS_USER_ARGS_GLB,   ///< Global based
   TS_USER_ARGS_COUNT  ///< Fake enum, # of valid entries.
 } TSUserArgType;


TSReturnCode TSUserArgIndexReserve(TSUserArgType type, const char *name, const char *description, int *arg_idx);
TSReturnCode TSUserArgIndexNameLookup(TSUserArgType type, const char *name, int *arg_idx, const char **description);
TSReturnCode TSUserArgIndexLookup(TSUserArgType type, int arg_idx, const char **name, const char **description);
void TSUserArgSet(void *data, int arg_idx, void *arg);
void *TSUserArgGet(void *data, int arg_idx);

One fundamental change here is that the opaque data parameter to TSUserArgSet and TSUserArgGet are context aware. If you pass in a Txn pointer, it behaves as a transaction user arg slot. If you pass in a nullptr, it becomes the new global slot behavior (since there is no context). The valid contexts are:

data type	Semantics
TSHttpTxn	The implicit context is for a transaction (TS_USER_ARGS_TXN)
TSHttpSsn	The implicit context is for a transaction (TS_USER_ARGS_SSN)
TSVConn	The implicit context is for a transaction (TS_USER_ARGS_VCONN)
nullptr	The implicit context is global (TS_USER_ARGS_GLB)