.. Licensed to the Apache Software Foundation (ASF) under one
   or more contributor license agreements.  See the NOTICE file
   distributed with this work for additional information
   regarding copyright ownership.  The ASF licenses this file
   to you under the Apache License, Version 2.0 (the
   "License"); you may not use this file except in compliance
   with the License.  You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing,
   software distributed under the License is distributed on an
   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
   KIND, either express or implied.  See the License for the
   specific language governing permissions and limitations
   under the License.

.. include:: ../common.defs

.. _whats_new:

What's New in ATS v9.x
=======================

This version of ATS includes over <x> commits, from <y> pull requests. A total of <z> contributors have participated in this
development cycle.

.. toctree::
   :maxdepth: 1

New Features
------------

This version of ATS has a number of new features (details below), but we're particularly excited about the following features:

* Experimental QUIC support (draft 27 and 29)
* TLS v1.3 0RTT support
* Significant HTTP/2 performance improvement
* PROXY protocol support
* Significant improvments to parent selection, including a new configuration file :file:`strategies.yaml`
* Several new plugins

A new infrastructure and tool chain for end-to-end testing and replaying traffic is introduced, the Proxy Verifier.

PROXY protocol
~~~~~~~~~~~~~~

ATS now supports the `PROXY <https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt>`_ protocol, on the inbound side. The
incoming PROXY data gets transformed into the ``Forwarded`` header.

Developer features
~~~~~~~~~~~~~~~~~~

* Add support for dtrace style markers (SDT) and include a few markers at locations of interest to users of SystemTap, dtrace, and
  gdb. See :ref:`developer-debug-builds`.

Command line tools
~~~~~~~~~~~~~~~~~~

* The :program:`traffic_server` program now has two new maintenance commands: ``verify_global_plugin`` and ``verify_remap_plugin``.
  These commands load a plugin's shared object file and verify it meets minimal global or remap plugin API requirements.

New configurations
------------------

Some new configurations were added, on existing features.

HTTP/2
~~~~~~

The following new configurations are available to rate limit some potentially abusive clients.

* :ts:cv:`proxy.config.http2.max_settings_per_frame`
* :ts:cv:`proxy.config.http2.max_settings_per_minute`

Overall, the performance for HTTP/2 is significantly better in ATS v9.x.

Parent Selection
~~~~~~~~~~~~~~~~

A new directive, `ignore_self_detect`, is added to the :file:`parent.config` format. This allows you to parent proxy to
sibling proxies, without creating loops. The setting for `proxy.config.http.parent_proxy_routing_enable`
is no longer needed, it's implicit by the usage of the :file:`parent.config` configuration file.

A new option was added to :file:`parent.config` for which status codes triggers a simple retry,
`simple_server_retry_responses`.

A configuration file for the new parent selection strategy is added, :file:`strategies.yaml`.

SNI
~~~

A new option to control how host header and SNI name mismatches are handled has been added. With this new
setting, :ts:cv:`proxy.config.http.host_sni_policy`, when set to `0`, no checking is performed. If this
setting is `1` or `2` (the default), the host header and SNI values are compared if the host header value
would have triggered a SNI policy.

A new blind tunneling option was added, `partial_blind_route`, which is similar to the existing
`forward_route` feature.

The captured group from a FQDN matching in :file:`sni.yaml` can now be used in the `tunnel_route`, as e.g.
`$1` and `$2`.

Plugin reload
~~~~~~~~~~~~~

A new setting was added to turn off the dynamic plugin reload, :ts:cv:`proxy.config.plugin.dynamic_reload_mode`.
By default, the feature is still enabled.

Updated host matching settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The options for host matching configuration, ts:cv:`proxy.config.http.server_session_sharing.match` has been
augmented significantly, the full list of host matches now are:

   ============= ===================================================================
   Value         Description
   ============= ===================================================================
   ``none``      Do not match and do not re-use server sessions.
   ``ip``        Re-use server sessions, checking only that the IP address and port
                 of the origin server matches.
   ``host``      Re-use server sessions, checking that the fully qualified
                 domain name matches. In addition, if the session uses TLS, it also
                 checks that the current transaction's host header value matchs the session's SNI.
   ``both``      Equivalent to ``host,ip``.
   ``hostonly``  Check that the fully qualified domain name matches.
   ``sni``       Check that the SNI of the session matches the SNI that would be used to
                 create a new session.  Only applicable for TLS sessions.
   ``cert``      Check that the certificate file name used for the server session matches the
                 certificate file name that would be used for the new server session.  Only
                 applicable for TLS sessions.
   ============= ===================================================================

RAM cache per volumes
~~~~~~~~~~~~~~~~~~~~~

In :file:`volume.config`, you can now specify if a volume should have a RAM cache or not. This
can be particularly useful when a volume is assigned to a RAM disk already. The new option flag
is `ramcache`, with a value of `true` or `false`.

Incompatible records.config settings
------------------------------------

These are the changes that are most likely to cause problems during an upgrade.
Take special care making sure you have updated your configurations accordingly.

Connection management
~~~~~~~~~~~~~~~~~~~~~

The old settings for origin connection management included the following settings:

* `proxy.config.http.origin_max_connections`
* `proxy.config.http.origin_max_connections_queue`
* `proxy.config.http.origin_min_keep_alive_connections`

These are all gone, and replaced with the following set of configurations:

* :ts:cv:`proxy.config.http.per_server.connection.max` (overridable)
* :ts:cv:`proxy.config.http.per_server.connection.match` (overridable)
* :ts:cv:`proxy.config.http.per_server.connection.alert_delay`
* :ts:cv:`proxy.config.http.per_server.connection.queue_size`
* :ts:cv:`proxy.config.http.per_server.connection.queue_delay`
* :ts:cv:`proxy.config.http.per_server.connection.min`


Logging and Metrics
-------------------

In addition to logging indivdiual headers, we now have comparable log tags that dumbs
the entire header. This table shows the additions, and what they correspond with.

    ============== ===================
    Original Field All Headers Variant
    ============== ===================
    cqh            cqah
    pqh            pqah
    psh            psah
    ssh            ssah
    cssh           cssah
    ============== ===================


A new log field for the elliptic curve was introduced, `cqssu`. Also related to TLS, the
new log field `cssn` allows logging the SNI server name from the client handshake.

New origin metrics
~~~~~~~~~~~~~~~~~~

A large number of metrics were added for the various cases where ATS closes an origin
connection. This includes:

* :ts:stat:`proxy.process.http.origin_shutdown.pool_lock_contention`
* :ts:stat:`proxy.process.http.origin_shutdown.migration_failure`
* :ts:stat:`proxy.process.http.origin_shutdown.tunnel_server`
* :ts:stat:`proxy.process.http.origin_shutdown.tunnel_server_no_keep_alive`
* :ts:stat:`proxy.process.http.origin_shutdown.tunnel_server_eos`
* :ts:stat:`proxy.process.http.origin_shutdown.tunnel_server_plugin_tunnel`
* :ts:stat:`proxy.process.http.origin_shutdown.tunnel_transform_read`
* :ts:stat:`proxy.process.http.origin_shutdown.release_no_sharing`
* :ts:stat:`proxy.process.http.origin_shutdown.release_no_keep_alive`
* :ts:stat:`proxy.process.http.origin_shutdown.release_invalid_response`
* :ts:stat:`proxy.process.http.origin_shutdown.release_invalid_request`
* :ts:stat:`proxy.process.http.origin_shutdown.release_modified`
* :ts:stat:`proxy.process.http.origin_shutdown.release_misc`
* :ts:stat:`proxy.process.http.origin_shutdown.cleanup_entry`
* :ts:stat:`proxy.process.http.origin_shutdown.tunnel_abort`

Metric scaling
~~~~~~~~~~~~~~

In previous versions of ATS, you had to rebuild the software to increase the maximum
number of metrics that the system could handle. This is replaced with a new command
line option for :program:`traffic_manager`, `--maxRecords`. The old build configure
option for this, `--with-max-api-stats`, is also eliminated.

TLS version metrics
~~~~~~~~~~~~~~~~~~~

A set of new metrics for SSL and TLS versions have been added:

* proxy.process.ssl.ssl_total_sslv3
* proxy.process.ssl.ssl_total_tlsv1
* proxy.process.ssl.ssl_total_tlsv11
* proxy.process.ssl.ssl_total_tlsv12
* proxy.process.ssl.ssl_total_tlsv13

Plugins
-------

statichit
~~~~~~~~~

This is a new, generic static file serving plugin. This can replace other plugins as well,
for example the :program:`healthchecks`.

maxmind
~~~~~~~

A new geo-ACL plugin was merged, :program:`maxmind`. This should support the newer APIs
from MaxMind, and should likely replace the old plugin in future release of ATS.

memory_profile
~~~~~~~~~~~~~~

This is a helper plugin for debugging and analyzing memory usage with e.g. JEMalloc.

slice
~~~~~

The :program:`slice` plugin has had a major overhaul, and has a significant number of new features
and configurations. If you use the slice plugin, we recommend you take a look at the
documentation again.

xdebug
~~~~~~

* A new directive, `fwd=<n>` to control the number of hops the header is forwarded for.

cert_reporting_tool
~~~~~~~~~~~~~~~~~~~

This is a new plugin to log examine and log information on loaded certificates.

Plugin APIs
-----------

The API for server push is promoted to stable, and modified to return an error code, to be consistent with other similar APIs. The
new prototype is:

.. code-block:: c

    TSReturnCode TSHttpTxnServerPush(TSHttpTxn txnp, const char *url, int url_len);

A new set of APIs for scheduling continuations on a specific set of threads has been added:

.. code-block:: c

    TSAction TSContScheduleOnPool(TSCont contp, TSHRTime timeout, TSThreadPool tp)
    TSAction TSContScheduleOnThread(TSCont contp, TSHRTime timeout, TSEventThread ethread)


There is a new API for redoing a cache lookup, typically after a URL change, or cache key update:

.. code-block:: c

    TSReturnCode TSHttpTxnRedoCacheLookup(TSHttpTxn txnp, const char *url, int length)

New APIs for TLS client context retrievals were added:

.. code-block:: c

    TSReturnCode TSSslClientContextsNamesGet(int n, const char **result, int *actual)
    TSSslContext TSSslClientContextFindByName(const char *ca_paths, const char *ck_paths)

In addition to these, a new set of APIs were added for the effective TLS handshake with
the client. These APIs also have equivalent Lua APIs:

.. code-block:: c

    int TSVConnIsSslReused(TSVConn sslp)
    const char *TSVConnSslCipherGet(TSVConn sslp)
    const char *TSVConnSslProtocolGet(TSVConn sslp)
    const char *TSVConnSslCurveGet(TSVConn sslp)

We have also added two new alert mechanisms for plugins:

.. code-block:: c

    void TSEmergency(const char *fmt, ...) TS_PRINTFLIKE(1, 2)
    void TSFatal(const char *fmt, ...) TS_PRINTFLIKE(1, 2)

A new API was added to allow control over whether the plugin will participate in the
dynamic plugin reload mechanism:

.. code-block:: c

    TSReturnCode TSPluginDSOReloadEnable(int enabled)

User Arg Slots
~~~~~~~~~~~~~~

The concept of user argument slots for plugins was completely redesigned. The old behavior
still exists, but we now have a much cleaner, and smaller, set of APIs. In addition, it also
adds a new slot type, `global`, which allows a slot for plugins to retain global data across
reloads and cross-plugins.

The new set of APIs has the following signature:

.. code-block:: c

    typedef enum {
       TS_USER_ARGS_TXN,   ///< Transaction based.
       TS_USER_ARGS_SSN,   ///< Session based
       TS_USER_ARGS_VCONN, ///< VConnection based
       TS_USER_ARGS_GLB,   ///< Global based
       TS_USER_ARGS_COUNT  ///< Fake enum, # of valid entries.
     } TSUserArgType;


    TSReturnCode TSUserArgIndexReserve(TSUserArgType type, const char *name, const char *description, int *arg_idx);
    TSReturnCode TSUserArgIndexNameLookup(TSUserArgType type, const char *name, int *arg_idx, const char **description);
    TSReturnCode TSUserArgIndexLookup(TSUserArgType type, int arg_idx, const char **name, const char **description);
    void TSUserArgSet(void *data, int arg_idx, void *arg);
    void *TSUserArgGet(void *data, int arg_idx);

One fundamental change here is that the opaque `data` parameter to `TSUserArgSet` and `TSUserArgGet` are context aware.
If you pass in a `Txn` pointer, it behaves as a transaction user arg slot. If you pass in a nullptr, it becomes the new
`global` slot behavior (since there is no context). The valid contexts are:

   ============== =======================================================================
   data type      Semantics
   ============== =======================================================================
   ``TSHttpTxn``  The implicit context is for a transaction (``TS_USER_ARGS_TXN``)
   ``TSHttpSsn``  The implicit context is for a transaction (``TS_USER_ARGS_SSN``)
   ``TSVConn``    The implicit context is for a transaction (``TS_USER_ARGS_VCONN``)
   ``nullptr``    The implicit context is global (``TS_USER_ARGS_GLB``)
   ============== =======================================================================