# C++ formatted serialization [closed]

I've decided it's time to implement serialization in my simple engine but this has caused me many headaches for the past couple of days/weeks. My engine uses an entity/component based approach similar to what is described in the T=Machine blog.

To keep it simple, my first goal was to implement serialization of entities. Since I'm going to use Lua for scripting, I thought it would be nice to serialize entities into Lua tables. Unfortunately, I find the information that is available on serialization in C++ is rather scarce. After many days of thinking I found one approach which is working for my implementation and is described here.

In essence, I have an EntityManager class from which I request new entities. This class holds entity as well as pointers of type BaseComponent from which all components derive. Because I store components as base pointers, I couldn't use a friend class for serialization, but had to make the BaseComponent inherit from a Serializable class, which has pure virtual serialization methods.

So, this is not that bad, I just have to inherit from the Serializable class and implement the serialization method. To support deserialization though, I have to do a few tricks according to the last link. That is, I have to include a static object inside each component whose sole purpose is to populate a map from component names to constructors. Here is a generic example (and please ignore any possible errors),


/* Base.h */
struct Base: public Serializeable{
static std::map<std::string, std::function<Base*(void)> > deriveds;
virtual ~Base(void){}
};

/* Derived.h */
template<class T>
Base::deriveds[name] = [](void) -> Base* {
return new T;
};
}
}

class Derived: public Base{
public:
int x;
int y;
Derived(void){
std::cout << "Derived created" << std::endl;
}
void serialize(std::ostream&);
};



Then in the implementation file I also have to initialize the static map of the base class as well as the static object of each derived class.

I find this quite a mess and although it could probably be improved with the use of macros, I can't help to wonder if there are better ways. How do you handle this kind of formatted serialization?

PS: Please don't recommend changing to a more flexible programming language :) !

• This serialization is for saving/loading existing entities? Or it's for creating new entities? I'm using Java for my game, but I found it was easier to create my own serialization methods. Simply creating a toByteSteam function that writes out to a passed file handle and a constructor that takes a file handle as an argument. This way, you control exactly what gets written and in what order. The file handle can be passed to multiple entities because they just write their data and leave the stream open. Same for reading entities. So one file can hold all the entities. – MichaelHouse Mar 15 '13 at 13:45
• Eventually I want to be able to do both. I've seen your blog posts on how you assemble entities by reading a descriptions from custom formatted files. I want to do something similar. Unfortunately, what I'm asking is more implementation specific rather than about the general idea. I think that implementing such serialization in Java is much easier. – Grieverheart Mar 15 '13 at 17:32
• The actual assembly of the entities doesn't use any kind of serialization, which is why I asked. I do use the custom serialization I mentioned in my comment for saving/loading existing entities. For my implementation I read in the entities and their unique IDs in one loop, then I read in their attributes in a second loop. That's simply to avoid some dependency issues. A very basic version of what I'm talking about can be found in an my answer here. You seem to be on the right track and know what you're doing, so I'll leave you alone :) – MichaelHouse Mar 15 '13 at 17:50
• I hope I didn't sound offensive. I would definitely like to see how you handle the assembly in your code and if it would be possible to do something similar in C++. I would think you would need at least some kind of reflection to achieve this, no? – Grieverheart Mar 15 '13 at 18:04
• Nope, impossible to offend me, I'm a robot. No reflection needed. I've written about it a few times, the blog posts are the most complete. Perhaps this answer might be useful too. You're welcome to ping me in chat (with a @byte56 and I'll see a notification) if you want to discuss more about how it's done. – MichaelHouse Mar 15 '13 at 18:15

I'm not saying either your approach is good or not! Just simply note that it's very complex and might later give you a headache if you even need to change simplest things. But there is an issue with your current implementation. C++ compilers doesn't care about the order they initialize static variables. this won't cause that much problem with primitive types (integers/float/pointers/etc.) but it definitely might cause a problem in your case. simply because when Derived::adder_ is being initialized, base::deriveds might not be initialized yet. The solution is to use late initializers, something like providing a function to add a new entry in your Object list, and in that forcing derived function to be initialized. here is your code with my suggestion applied:

/* Base.h */
struct Base: public Serializeable{
static std::map<std::string, std::function<Base*(void)> >* deriveds;
{
if (deriveds == nullptr)
deriveds = new std::map<std::string, std::function<Base*(void)> >();
(*deriveds)[name] = [](void) -> Base* {
return new T;
};
}
virtual ~Base(void){}
};

/* Derived.h */
template<class T>
}
}

class Derived: public Base{
public:
int x;
int y;
Derived(void){
std::cout << "Derived created" << std::endl;
}
void serialize(std::ostream&);
};


if I remember correctly all global variables are guarantied to be initialized to zero before the application starts (correct me if I'm wrong) so you won't have such mentioned problem with this code.

• All statically declared variables (not just globals) will have been initialised before main() is called, yes. – Trevor Powell Mar 15 '13 at 14:26
• @TrevorPowell I'm talking about the order of initialization. I mean it's possible to see an adder_ variable is initialized before deriveds variable, and it'll cause a access violation even before main() is called. – Ali1S232 Mar 15 '13 at 15:14
• Within a single file order will follow the storage declarations, between files there is no guarantee on which file's variables get initialized first. IIRC – Patrick Hughes Mar 15 '13 at 16:14
• @Chunk-e-Yamani Thank you very much for your feedback. You are indeed correct that the way I implemented it in the example code, problems can be caused during static initialization because of the order of initialization. I will definitely check your suggestion if I decide to go with the method I presented. You also mention it's very complex and might later give you a headache. Do you have any alternatives in mind which might be less complex? – Grieverheart Mar 15 '13 at 17:22
• @Grieverheart not really :| in fact I've once implemented similar structure, for another experimental project. but I didn't really had to change any thing after the base was complete. though you might want to look into boost library and their serialization mechanism, it has it's own features and drawbacks. – Ali1S232 Mar 15 '13 at 17:44