Stupid Compiler Tricks

A coworker approached me today with an interesting problem and I figured I’d talk about the crazy acrobatics that solved it.

He wanted to fill out a list of operations to perform that could be registered easily “at compile time”, so that the list could be traversed later to perform these operations. Basically, he wanted his code to look something like this:

typedef std::map< const char *, void (*)( void ) > WorkItemList;
static WorkItemList sWork;

void some_function( void ) {
  // Do some work
}
REGISTER( "operation 1", some_function );
REGISTER( "operation 2", some_function );

void some_other_function( void ) {
  // Do some work
}
REGISTER( "operation 3", some_other_function );
REGISTER( "operation 4", some_other_function );
REGISTER( "operation 5", some_other_function );

// Later
void DoWork( const char *workItem ) {
  WorkItemList::iterator iter = sWork.find( workItem );
  if (iter != sWork.end())
    iter->second();
}

So how do you make this work?

The first thing to remember is that top-level statements basically need to be declarations (or preprocessor commands). So in order for the REGISTER functionality to work, we need to turn it into a declaration of some kind. Except the goal is to fill out the sWork map — how do you do that with a declaration? By using class constructors. Static global variables are initialized sometime before the main() function is called (while not strictly true, the observable effect remains the same), and so we can use this to our advantage. Create a class whose constructor performs the work of initializing the work list.

class Registrar {
public:
  Registrar( const char *str, void (*fp)( void ) ) {
    sWork[ str ] = fp;
  }
};

Now, when a Registrar object is created, the constructor will fire and the item will be added to the work list. This allows us to create static variable declarations to register the work items. So our REGISTER macro can look something like this:

#define REGISTER( n, f )  static Registrar reg( n, f )

However, there are some obvious and not-so-obvious problems to this approach. One of the obvious problems is that calling REGISTER more than once will lead to duplicate declarations of “reg.” We need a way to create unique identifiers, but without forcing it on the caller. This is where the macro concatenation operator shines!

The macro concatenation operator is ##, and it allows you to concatenate any two tokens at macro expansion time. So if you have a macro like this:

#define REGISTER( n, f )  static Registrar reg_##f( n, f )

REGISTER( "Test", Foo );

Then when the REGISTER macro is called you wind up with reg_Foo. However, this still doesn’t fully solve our problem — the REGISTER macro is called with the same method multiple times, so we still have the same problem. For this we use the compiler-specific macro __COUNTER__. It exists in both MSVC and gcc, and provides exactly what we need. Each time __COUNTER__ is expanded into an integer, its value incremented by one. So we can use that to create a unique identifier like reg_Foo1, reg_Foo2, etc.

The naive implementation, unfortunately, doesn’t work.

#define REGISTER( n, f )  static Registrar reg_##f##__COUNTER__( n, f )

REGISTER( "Test", Foo );

This forms the identifier reg_Foo__COUNTER__ each time you call the REGISTER macro. That is because the concatenation happens before macro expansion. So the __COUNTER__ macro is not expanded since it is part of the concatenation. That means it must be passed in to the macro in already-expanded form; we want the integer value, not the name. The rules for macro expansions are quite complex! If you follow them closely, you will end up with a macro like this:

#define REG_OP_2( x, y )	reg_##x##y
#define REG_OP_1( name, func, val )	static Registrar REG_OP_2( func, val )( name, func )
#define REGISTER( name, func )	REG_OP_1( name, func, __COUNTER__ )

Following the macro expansion rules, our example would expand to these stages:

REGISTER( "Test", Foo );
REG_OP_1( "Test", Foo, __COUNTER__ );
static Registrar REG_OP_2( Foo, __COUNTER__ )( "Test", Foo );
static Registrar reg_Foo1( "Test", Foo );

This is exactly what we were after! Each of the calls to REGISTER will eventually expand out into a static class instance declaration with the text and function being passed as constructor parameters. In turn, this populates the static map.

However, there is a not-so-obvious gotcha here. Because we’re using a static std::map and static class instances, we have to worry about initialization order. This is another complex area of C++, but the rules that apply here aren’t too bad. Dynamic, non-local, static variable initialization within the same translation unit (cpp file, basically) are ordered based on the order of declaration. But not within the whole program! So if you have calls to REGISTER in separate files, you could find yourself in a situation where the std::map has not been initialized before the Registrar constructor attempts to mutate it. That would obviously be a bad thing!

One way to ensure this is not a problem would be to use a pointer to a std::map. Zero initialization of statics always happens before other initializations, so we can rely on being able to test the map against null. The first constructor call to find the null map will create the map. Since initializations are never threaded, there are no race conditions to worry about either. So our final implementation looks like this:

typedef std::map< const char *, void (*)( void ) > WorkItemList;
static WorkItemList *sWork;

class Registrar {
public:
  Registrar( const char *op, void (*fp)( void ) ) {
    if (!sWork)
      sWork = new WorkItemList;

    sWork->insert( std::make_pair( op, fp ) );
  }
};

#define REG_OP_2( x, y )             reg_##x##y
#define REG_OP_1( name, func, val )  static Registrar REG_OP_2( func, val )( name, func )
#define REGISTER( name, func )       REG_OP_1( name, func, __COUNTER__ )

This was a rather educational experiment for me; I had to learn a lot more about how macro expansion works, and take a hard refresher course on static/dynamic initialization. However, it was also a lot of fun! Hopefully someone else benefits from this aside from just my coworker and I.

This entry was posted in C/C++ and tagged , , . Bookmark the permalink.

6 Responses to Stupid Compiler Tricks

  1. Dan says:

    Aaron,

    Have been too busy to leave comments recently, but it’s Friday & I’m waiting to get on a plane, so no guilt!

    Definitely the whole initialization order stuff comes into play, good to see it featured prominently here.

    One of the systems I work on is considered “safety critical”, and one of the edicts from on high is no dynamic memory allocation (not even custom allocators or memory pools). I have a question: in the code near the end of the post, with the null check and the “new WorkItemList” – could we also just declare a static member of Registrar of type WorkItemList, and use that, instead of using new? Would we know for sure that the static member was already valid/initialized if we defined it right before sWork?

    (I guess if we did that, we might have to hoist the class definition of Registrar to the top, so when we went to define the static member outside the class, the compiler knew what we were talking about…)

  2. Aaron Ballman says:

    Hey Dan, glad to see you’re around! :-)

    I think you could get away with declaring it as a static member of Registrar because it would be defined within the same translation unit at that point. It’s still a non-local static variable, so it would follow the ordering rules. The only gotcha there is that Registrar cannot use a class template, generally speaking (specialized templates are ordered, unspecialized ones are unordered).

  3. Shiva says:

    Hi,
    DEFAULT __COUNTER__ value starts from ZERO.
    Instead of starting of __COUNTER__ value staring from 0 but I want to start from some X value.

    Thanks in advance.

  4. nobody says:

    very very THANKS!!

  5. Flamefire says:

    Careful: `const char *` as the map key, compares pointers. So there is no guarantee that getting entry “operation 1” later will get what you want as the address might be different. Also: Put the map in a singleton(-like) `get` function to avoid SIOF and use a better (=more unique) name for your registrar-instances. Then you can get rid of including `func` in the name as this breaks for non-c-identifier operation names.

  6. Alex says:

    Indexing map with char* is sketchy. Strings will typically be unique in your binary but if you strdup or load DLLs all bets are off.

Leave a Reply

Your email address will not be published. Required fields are marked *