#include "ts/Extendible.h"

Extendible allows Plugins to append additional storage to Core data structures.

Use Case:


Defines class Host as Extendible

TSPlugin HealthStatus

Appends field "down reason code" to Host Now the plugin does not need to create completely new lookup table and all implementation that comes with it.


A data class that inherits from Extendible, uses a CRTP (Curiously Recurring Template Pattern) so that its static Schema instance is unique among other Extendible types.

class ExtendibleExample : Extendible<ExtendibleExample> {
   int real_member = 0;

During system init, code and plugins can add fields to the Extendible’s schema. This will update the Memory Layout of the schema, and the memory offsets of all fields. The schema does not know the field’s type, but it stores the byte size and creates lambdas of the type’s constructor and destructor. And to avoid corruption, the code asserts that no instances are in use when adding fields.

ExtendibleExample::FieldId<ATOMIC,int> fld_my_int;

void PluginInit() {
  fld_my_int = ExtendibleExample::schema.addField("my_plugin_int");

When an Extendible derived class is instantiated, new() will allocate a block of memory for the derived class and all added fields. There is zero memory overhead per instance, unless using ACIDPTR field access type.

ExtendibleExample* alloc_example() {
  return new ExtendibleExample();

Memory Layout

One block of memory is allocated per Extendible, which include all member variables and appended fields. Within the block, memory is arranged in the following order:

  1. Derived members (+padding align next field)
  2. Fields (largest to smallest)
  3. Packed Bits

Strongly Typed Fields

template<AccessEnum FieldAccess_e, typename T> Extendible::FieldId

FieldId is a templated T reference. One benefit is that all type casting is internal to the Extendible, which simplifies the code using it. Also this provides compile errors for common misuses and type mismatches.

// Core code
class Food : Extendible<Food> {}
class Car : Extendible<Car> {}

// Example Plugin
Food::FieldId<STATIC,float> fld_food_weight;
Food::FieldId<STATIC,time_t> fld_expr_date;
Car::FieldId<STATIC,float> fld_max_speed;
Car::FieldId<STATIC,float> fld_car_weight;

PluginInit() {
   Food.schema.addField(fld_food_weight, "weight");
   Food.schema.addField(fld_expr_date,"expire date");
   Car.schema.addField(fld_car_weight,"weight"); // 'weight' is unique within 'Car'

PluginFunc() {
   Food banana;
   Car camry;

   // Common user errors

   float expire_date = banana.get(fld_expr_date);
   //^^^                                              Compile error: cannot convert time_t to float
   float speed = banana.get(fld_max_speed);
   //                       ^^^^^^^^^^^^^             Compile error: fld_max_speed is not of type Extendible<Food>::FieldId
   float weight = camry.get(fld_food_weight);
   //                       ^^^^^^^^^^^^^^^           Compile error: fld_food_weight is not of type Extendible<Car>::FieldId

Field Access Types

Enums Allocates API Pros Cons
ATOMIC std::atomic<T> get() Leverages std::atomic API. No Locking. Only works on small data types.
BIT 1 bit from packed bits get(), readBit(), writeBit() Memory efficient. Cannot return reference.
ACIDPTR std::make_shared<T> get(), writeAcidPtr() Avoid skew in non-atomic structures. Non-contiguous memory allocations. Uses locking.
STATIC T get(), init() Const reference. Fast. Type safe. No concurrency protection.
DIRECT T get() Direct reference. Fast. Type safe. No concurrency protection.
C_API a number of bytes get() Can use in C. No concurrency protection. Not type safe.

operator[](FieldId) has been overridden to call get(FieldId) for all access types.

Unfortunately our data is not “one type fits all”. I expect that most config values will be stored as AccessEnum::STATIC, most states values will be AccessEnum::ATOMIC or AccessEnum::BIT, while vectored results will be AccessEnum::ACIDPTR.


enum AccessEnum
enumerator ATOMIC

Represents atomic field reference (read, write or other atomic operation).

enumerator BIT

Represents compressed boolean fields.

enumerator STATIC

Represents immutable field, value is not expected to change, no internal thread safety.

enumerator ACIDPTR

Represents a pointer promising Atomicity, Consistency, Isolation, Durability

enumerator DIRECT

Represents a mutable field with no internal thread safety.

enumerator C_API

Represents C-Style pointer access.

template<typename Derived_t>
class Extendible

Allocates block of memory, uses FieldId or schema to access slices of memory.

Template Parameters:
 Derived_t – The class that you want to extend at runtime.
template<AccessEnum FieldAccess_e, typename T>
class FieldId

The handle used to access a field. These are templated to prevent human error, and branching logic.

Template Parameters:
  • FieldAccess_e – The type of access to the field.
  • T – The type of the field.
class FieldSchema

Stores attributes, constructor and destructor of a field.

class Schema

Manages fields and memory layout of an Extendible type.

template<AccessEnum FieldAccess_e, typename T>
bool addField(FieldId<FieldAccess_e, T> &field_id, std::string const &field_name)

Add a new field to this record type.

static Schema schema

one schema instance per Extendible to define contained fields

template<AccessEnum FieldAccess_e, typename T>
std::atomic<T> &get(FieldId<AccessEnum::ATOMIC, T> const &field)
using BitFieldId = FieldId<AccessEnum::BIT, bool>
bool const get(BitFieldId field) const
bool const readBit(BitFieldId field) const
void writeBit(BitFieldId field, bool const val)
template<AccessEnum FieldAccess_e, typename T>
T const &get(FieldId<AccessEnum::STATIC, T> field) const
template<AccessEnum FieldAccess_e, typename T>
T &init(FieldId<AccessEnum::STATIC, T> field)
template<AccessEnum FieldAccess_e, typename T>
std::shared_ptr<const T> get(FieldId<AccessEnum::ACIDPTR, T> field) const
template<AccessEnum FieldAccess_e, typename T>
AcidCommitPtr<T> writeAcidPtr(FieldId<AccessEnum::ACIDPTR, T> field)
void *get(FieldId_C &field)
typedef const void *FieldId_C

The handle used to access a field through C API. Human error not allowed by convention.