Once again, it has been several months since I’ve posted anything. I’ve decided it would be beneficial to keep a development log of what I’m working on. So, here it goes.

The two main things I have been working on are re-writing the reflection system and how I handle engine modules. My goal is that when writing modules to extend the engine, I’d like as little boilerplate code as possible when hooking up the DLLs. Ideally, I’d just define the reflection and everything would automatically be picked up by the system. I think I have achieved something pretty close to that, and am very pleased with what I’ve come up with.

Reflection⌗

First, in your header file, you declare your reflection like this:

// Class can be in any namespace.
NS_SHIBBOLETH

// Inheriting from Gaff::IReflectionObject is optional.
// Only necessary if you care about casting from a base
// class to a derived class.
class ReflectedClass : public Gaff::IReflectionObject
{
public:
	int getA(void) const { return a; }
	void setA(int _a) { a = _a; }

	void someFunc(void) {}

private:
	int a = 1;

	// This declare is optional. Defines the interface
	// of Gaff::IReflectionObject. You can still cast
	// from a base to a derived without inheriting from
	// Gaff::IReflectionObject if you use this macro.
	SHIB_REFLECTION_CLASS_DECLARE(ReflectedClass);
};

NS_END

// This must come last and must be in the global namespace.
SHIB_REFLECTION_DECLARE(ReflectedClass);

And your implementation file would look like this:

// This must be in the global namespace.
SHIB_REFLECTION_DEFINE(ReflectedClass);

NS_SHIBBOLETH

SHIB_REFLECTION_CLASS_DEFINE_BEGIN(ReflectedClass)
	// Upper-case .BASE() call is for allowing to query
	// for non-reflected interfaces.
	.BASE(Gaff::IReflectionObject)


	// Similarly to the above, we can register another
	// reflected type as a base class using the lowercase
	// version of the base<>() function.
	//.base<MyBaseClass>()

	// Register a constructor that takes in no parameters.
	// Reflection also acts as a factory, and can be used to
	// create instances of reflected object types by name.
	.ctor<>()

	// Reflect property 'a'. Optional third parameter is a boolean
	// that states if the variable is read-only.
	.var("a", &ReflectedClass::a)

	// Same as above, but using getter/setter functions.
	// Setter is optional. Leaving out setter will make
	// variable read-only.
	.var("a_func", &ReflectedClass::getA, &ReflectedClass::setA)

	// Reflect a function that we can call via reflection.
	.func("someFunc", &ReflectedClass::someFunc)
SHIB_REFLECTION_CLASS_DEFINE_END(ReflectedClass)

// Condensed version would look like this.
SHIB_REFLECTION_CLASS_DEFINE_BEGIN(ReflectedClass)
	.BASE(Gaff::IReflectionObject)

	.ctor<>()

	.var("a", &ReflectedClass::a)
	.var("a_func", &ReflectedClass::getA, &ReflectedClass::setA)

	.func("someFunc", &ReflectedClass::someFunc)
SHIB_REFLECTION_CLASS_DEFINE_END(ReflectedClass)

NS_END

So, what do these macros do? The long story short is that they create template specializations of the Shibboleth::Reflection<> class in the Shibboleth namespace that internally houses a reference to a ReflectionDefinition. These template specializations all have a bunch of static functions that you can use to query reflection data and manipulate instances of reflected objects.

All ReflectionDefinitions lives inside a repository that is stored by the global App instance. Since the reflection definitions are stored inside of the App class, which is not statically initialized at application launch, all reflection is manually instantiated with a call to Shibboleth::Reflection<>::Init(). More on this later. This also means that any reflected types that are referenced by multiple DLLs will have their Init() called in each DLL. What happens is that before initialization, it checks with the repository to see if it already has that definition. If it does, it then checks to see if their version numbers match. If they do, then it will use the already existing reflection definition.

The SHIB_REFLECTION_CLASS_DEFINE_BEGIN/END() macros are defining a template function that can be used to build a ReflectionDefinition. Or any object that implements the reflection builder interface. What does this mean? You can write reflection once and use it to build all kinds of data about your reflected object.

An example is the versioning mentioned above. There is a ReflectionVersion data structure that generates a hash of the object version using the reflection definition we provided above. So when each module initializes their version of an object’s reflection, it can compare to what is already registered with the App instance and detect version mismatches. I can then use this version number when serializing out data for checking compatability.

Another capability of the reflection system is that you can extend the Init() function. Using the above class, here is how we would extend the Init() function.

// Replace SHIB_REFLECTION_DEFINE(ReflectedClass) with the BEGIN()/END() macros below.

SHIB_REFLECTION_DEFINE_BEGIN(ReflectedClass)
	AngelScriptClassRegister<ReflectedClass> asr(e);
	BuildReflection(asr); // SHIB_REFLECTION_CLASS_DEFINE_BEGIN()/END() define this function.
SHIB_REFLECTION_DEFINE_END(ReflectedClass)

In this case I have called BuildReflection() using a class that registers the reflected type with a scripting language (in this case, AngelScript). I think this is a pretty cool property of the reflection system. This means I can write reflection once, and have that data work automatically for me in other systems!

The reflection system can also reflect classes that don’t use the SHIB_REFLECTION_CLASS_DECLARE() macro. This is useful for if you want to reflect things such as math library types (e.g. Vec3, Quaternion, etc.) or any data type that you don’t want to inject vtables into. The syntax is slightly different when defining the reflection. You would use SHIB_REFLECTION_EXTERNAL_DEFINE_BEGIN_() instead of SHIB_REFLECTION_DEFINE_BEGIN() and use SHIB_REFLECTION_BUILDER_BEGIN()/END() instead of SHIB_REFLECTION_CLASS_DEFINE_BEGIN()/END().

Modules⌗

So we have all this reflection data split across several module DLLs in the engine. How do we initialize our DLL? Well, it’s simple.

#include <Shibboleth_Utilities.h>
#include "Gen_ReflectionInit.h"

DYNAMICEXPORT_C bool InitModule(Shibboleth::IApp* app)
{
	Shibboleth::SetApp(*app);
	Gen::InitReflection();
	return true;
}

That’s it. Obviously there’s more going on in Gen::InitReflection(). But every single module file has this as a bare minimum. What I’ve done is written a Python script that runs as a pre-build step that generates the Gen_ReflectionInit.h file for a given module. The Gen::InitReflection() function will register every class defined within the module and initialize any reflection types referenced in the module. No need for any special markup in your code, just the reflection syntax mentioned above. Compared to how modules used to work, this is a million times better.

Cool Stuff⌗

One really cool thing that I’m doing in the engine with the new reflection system is doing automatic registration of types to systems that care. Take, for example, this resource class used by the resource management system.

// Shibboleth_AngelScriptResource.h
NS_SHIBBOLETH

class AngelScriptResource final : public IResource
{
public:
	// Public stuff here.

private:
	// Private stuff here.

	SHIB_REFLECTION_CLASS_DECLARE(AngelScriptResource);
}

NS_END

SHIB_REFLECTION_DECLARE(AngelScriptResource)


// Shibboleth_AngelScriptResource.cpp
SHIB_REFLECTION_DEFINE(AngelScriptResource)

NS_SHIBBOLETH

SHIB_REFLECTION_CLASS_DEFINE_BEGIN(AngelScriptResource)
	// This attribute says that this resource loads files ending
	// with the .as extension.
	.classAttrs(
		ResExtAttribute(".as")
		// You can chain multiple attributes in one call
		// to classAttrs().
		// Resources can have any number of ResExtAttributes.
	)

	// This tells the ResourceManager that this type is a resource type.
	// The resource system will then grab the ResExtAttribute(s) above to determine
	// which files this resoure class is capable of loading.
	// When a file with the .as extension is loaded, it will use the factory registered
	// in the .ctor<>() call below to create an instance of the resource and load it
	// through the IResource interface. And it never needs to know about the concrete
	// implementation!
	.BASE(IResource)

	.ctor<>()
SHIB_REFLECTION_CLASS_DEFINE_END(AngelScriptResource)

NS_END

This is pretty freakin’ cool to me. Using the reflection data I was going to write anyways, I automatically get my type reigstered with whatever systems they need to with no extra work. No more calling GetApp().getManagerT<SomeManager>().RegisterType<MyType>() inside of module initialization code! This is, of course, extensible to any system that needs automatic registration. For an example of how to do this, check out src/Resource/Shibboleth_ResourceManager.cpp in the Shibboleth repo in the shibboleth_refactor branch.

Summary⌗

To summarize, I’ve completely revamped the reflection system. I’ve also removed a lot of the pains I had in previous versions of Shibboleth with registering DLL modules. The new reflection system is slightly more verbose, but I think that is a worthwhile tradeoff for the power it gives. That said, my intention is to use reflection mainly for initialization and setup and very, VERY little in actual runtime code. Reflection is nice and fancy, but it does come at a cost.