What is the most efficient way to convert to binary and back in C#?

Question

I'm trying to write a general purpose socket server for a game I'm working on. I know I could very well use already built servers like SmartFox and Photon, but I wan't to go through the pain of creating one myself for learning purposes.

I've come up with a BSON inspired protocol to convert the the basic data types, their arrays, and a special GSObject to binary and arrange them in a way so that it can be put back together into object form on the client end. At the core, the conversion methods utilize the .Net BitConverter class to convert the basic data types to binary. Anyways, the problem is performance, if I loop 50,000 times and convert my GSObject to binary each time it takes about 5500ms (the resulting byte[] is just 192 bytes per conversion). I think think this would be way too slow for an MMO that sends 5-10 position updates per second with a 1000 concurrent users. Yes, I know it's unlikely that a game will have a 1000 users on at the same time, but like I said earlier this is supposed to be a learning process for me, I want to go out of my way and build something that scales well and can handle at least a few thousand users.

So yea, if anyone's aware of other conversion techniques or sees where I'm loosing performance I would appreciate the help.

GSBitConverter.cs

This is the main conversion class, it adds extension methods to main datatypes to convert to the binary format. It uses the BitConverter class to convert the base types. I've shown only the code to convert integer and integer arrays, but the rest of the method are pretty much replicas of those two, they just overload the type.

public static class GSBitConverter
{
    public static byte[] ToGSBinary(this short value)
    {
        return BitConverter.GetBytes(value);
    }

    public static byte[] ToGSBinary(this IEnumerable<short> value)
    {
        List<byte> bytes = new List<byte>();
        short length = (short)value.Count();

        bytes.AddRange(length.ToGSBinary());
        for (int i = 0; i < length; i++)
            bytes.AddRange(value.ElementAt(i).ToGSBinary());

        return bytes.ToArray();
    }

    public static byte[] ToGSBinary(this bool value);
    public static byte[] ToGSBinary(this IEnumerable<bool> value);

    public static byte[] ToGSBinary(this IEnumerable<byte> value);

    public static byte[] ToGSBinary(this int value);
    public static byte[] ToGSBinary(this IEnumerable<int> value);

    public static byte[] ToGSBinary(this long value);
    public static byte[] ToGSBinary(this IEnumerable<long> value);

    public static byte[] ToGSBinary(this float value);
    public static byte[] ToGSBinary(this IEnumerable<float> value);

    public static byte[] ToGSBinary(this double value);
    public static byte[] ToGSBinary(this IEnumerable<double> value);

    public static byte[] ToGSBinary(this string value);
    public static byte[] ToGSBinary(this IEnumerable<string> value);

    public static string GetHexDump(this IEnumerable<byte> value);
}

Program.cs Here's the the object that I'm converting to binary in a loop.

class Program
{
    static void Main(string[] args)
    {
        GSObject obj = new GSObject();
        obj.AttachShort("smallInt", 15);
        obj.AttachInt("medInt", 120700);
        obj.AttachLong("bigInt", 10900800700);
        obj.AttachDouble("doubleVal", Math.PI);
        obj.AttachStringArray("muppetNames", new string[] { "Kermit", "Fozzy", "Piggy", "Animal", "Gonzo" });

        GSObject apple = new GSObject();
        apple.AttachString("name", "Apple");
        apple.AttachString("color", "red");
        apple.AttachBool("inStock", true);
        apple.AttachFloat("price", (float)1.5);

        GSObject lemon = new GSObject();
        apple.AttachString("name", "Lemon");
        apple.AttachString("color", "yellow");
        apple.AttachBool("inStock", false);
        apple.AttachFloat("price", (float)0.8);

        GSObject apricoat = new GSObject();
        apple.AttachString("name", "Apricoat");
        apple.AttachString("color", "orange");
        apple.AttachBool("inStock", true);
        apple.AttachFloat("price", (float)1.9);

        GSObject kiwi = new GSObject();
        apple.AttachString("name", "Kiwi");
        apple.AttachString("color", "green");
        apple.AttachBool("inStock", true);
        apple.AttachFloat("price", (float)2.3);

        GSArray fruits = new GSArray();
        fruits.AddGSObject(apple);
        fruits.AddGSObject(lemon);
        fruits.AddGSObject(apricoat);
        fruits.AddGSObject(kiwi);

        obj.AttachGSArray("fruits", fruits);

        Stopwatch w1 = Stopwatch.StartNew();
        for (int i = 0; i < 50000; i++)
        {
            byte[] b = obj.ToGSBinary();
        }
        w1.Stop();

        Console.WriteLine(BitConverter.IsLittleEndian ? "Little Endian" : "Big Endian");
        Console.WriteLine(w1.ElapsedMilliseconds + "ms");

    }

Here's the code for some of my other classes that are used in the code above. Most of it is repetitive.

GSObject

GSArray

GSWrappedObject

Answer 1

Without having read through all your code (or, indeed, being very familiar with .NET) and thus not knowing how appropriate this suggestion is:

Make a wrapper around a class like MemoryStream, and add methods to objects that need to be serialized/deserialized. Don't just make a class that's an arbitrary bag of data. If you have a Widget class, it might read and write something like this:

public void WriteToBinary(StreamWrapper stream)
{
    stream.WriteString(name);
    stream.WriteString(color);
    stream.WriteBool(inStock);
    stream.WriteFloat(price);
}

public void ReadFromBinary(StreamWrapper stream)
{
    name = stream.ReadString();
    color = stream.ReadString();
    inStock = stream.ReadBool();
    price = stream.ReadFloat();
}

This means never creating a temporary byte[] array that will always be thrown away. It also can lead to smaller messages, because you don't need to encode the name of every single property: the order of reading/writing can disambiguate.

Answer 2

I would use something like ProtoBuf with RPC support.