Threads and Event Processing

Internally Traffic Server is a cooperative multi-threaded environment. There are a fixed number of threads for core operations, determined at process start time. All core operations take place on one of these existing threads. Plugins may spawn additional threads but these are outside the scope of this document.

Threads

Traffic Server has a taxonomy of thread types which are layered to create the threading infrastructure. At the most basic are threads as the operating system provides. Classes provide additional data and operations on these threads to make them operate properly for Traffic Server.

Thread

The abstract Thread is the base class for thread operations. It contains a mutex and a thread identifier. The logic for starting the thread at the system level is embedded in this class. All threads started by Traffic Server have an instance of this class (or subclass). Plugins can directly start their own threads via system calls and those are not tracked. Thread sets up thread local storage via pthread_setspecific. Threads can be started via an explicit function provided to Thread::start() or by subclassing Thread and overriding Thread::execute().

Thread also performs the basic time keeping for Traffic Server. The class contains a global static value which is treated as the current time for Traffic Server. Usually this class is accessed as a static but it can also be accessed in a way to update the current time. Because of the large number of threads the static use is generally sufficiently accurate because it contains the last time any thread updated.

EThread

EThread is a subclass of Thread which provides support for Traffic Server core operations. It is this class that provides support for using Continuation instances. EThread overrides the Thread::execute() method to gain control after the underlying thread is started. This method executes a single continuation at thread start. If the thread is :enumerator:

ThreadType::DEDICATED it returns after invoking the start continuation. No join is executed, the presumption is the start continuation will run until process termination. This mechanism is used because it is, from the Traffic Server point of view, the easiest to use because of the common support of continuations.

A ThreadType::REGULAR thread will first execute its start continuation and then process its event queue until explicitly stopped after executing the start continuation.

Despite the name EventProcessor is primarily a thread management class. It enables the creation and management of thread groups which are then used by the Traffic Server core for different types of computation. The set of groups is determined at run time via subsystems making calls to the EventProcessor::register_event_type(). Threads managed by EventProcessor have the EThread start continuation controlled by EventProcessor. Each thread group (event type) has a list of continuations to run when a thread of that type starts. Continuations are added to the list with EventProcessor::schedule_spawn(). There are two variants of this method, one for continuations and one for just a function. The latter creates a continuation to call the function and then schedules that using the former. The EventProcessor internal start continuation for the EThread executes the continuations on this list for the appropriate thread group and then returns, after which EThread::execute() loops on processing its event queue.

EventProcessor is intended to be a singleton and the global instance is eventProcessor.

In general if a subsystem in the Traffic Server core is setting up a thread group, it should use code of the form

int ET_GROUP; // global variable, where "GROUP" is replaced by the actual group / type name.
int n_group_threads = 3; // Want 3 of these threads by default, possibly changed by configuration options.
constexpr size_t GROUP_STACK_SIZE = DEFAULT_STACK_SIZE; // stack size for each thread.
void Group_Thread_Init(EThread*); // function to perform per thread local initialization.

ET_GROUP = eventProcessor::registerEventType("Group");
eventProcessor.schedule_spawn(&Group_Per_Thread_Init, ET_GROUP);
eventProcessor.spawn_event_threads(ET_GROUP, n_group_threads, GROUP_STACK_SIZE);

The function Group_Thread_Init can be replaced with a continuation if that’s more convenient. One advantage of a continuation is additional data (via cookie) can be provide during thread initialization.

If there is no thread initialization needed, this can be compressed in to a single call

ET_GROUP = eventProcessor.spawn_event_threads("Group", n_group_threads, GROUP_STACK_SIZE);

This registers the group name and type, starts the threads, and returns the event type.

Types

type EventType: A thread classification value that represents the type of events the thread is expected to process.

EventType ET_CALL: A predefined EventType which always exists. This is deprecated, use ET_NET instead.

EventType ET_NET: A synonym for ET_CALL.

EventProcessor eventProcessor: The global single instance of EventProcessor.

type ThreadFunction: The type of function invoked by Thread::start(). It is a function returning void* and taking no arguments.

class Thread

Wrapper for system level thread.

start(const char *name, void *stack, size_t stacksize, ThreadFunction const &f): Start the underlying thread. It is given the name name. If stack is nullptr then a stack is allocated for it of size stacksize. Once the thread is started, f is invoked in the context of the thread if non nullptr, otherwise the method Thread::execute() is called. The thread execution returns immediately after either of these, leaving a zombie thread. It is presumed both will execute until process termination.

void execute(): A pure virtual method that must be overridden in a subclass.

class EThread

Event processing thread.

EventType registerEventType(const char *name): Register an event type by name. This reserves an event type index which is returned as EventType.

void execute(): Call the start continuation, if any. If a regular (not dedicated) thread, continuously process the event queue.

enum ThreadType

enumerator DEDICATED: A thread which executes only the start continuation and then exits.

enumerator REGULAR: A thread which executes the start continuation and then processes its event queue.

class Continuation: A future computation. A continuation has a handler which is a class method with a specific signature. A continuation is invoked by calling its handler. A future computation can be referenced by an Action instance. This is used primarily to allow the future work to be canceled.

class Action: Reference to a future computation for a Continuation.

class Event : public Action : Reference to code to dispatch. Note that an Event is a type of Action. This class combines the future computational reference of Action

type ThreadSpawnFunction: A function that takes a single argument of pointer to EThread and returns void. The argument will be the EThread in which the function is executing.

class EventProcessor

EventType register_event_type(char const *name): Register an event type with the name name. The unique type index is returned.

Event *schedule_spawn(Continuation *c, EventType ev_type, int event = EVENT_IMMEDIATE, void *cookie = nullptr): When the EventProcessor starts a thread of type ev_type, c will be called before any events are dispatched by the thread. The handler for c will be called with an event code of event and data pointer of cookie.

Event *schedule_spawn(void (*f)(EThread*), EventType ev_type): When the EventProcessor starts a thread of type ev_type the function f will be called with a pointer to the EThread instance which is starting.