2
\$\begingroup\$

I'm trying to reverse an animation file from a commercial multiplayer game, whose format was recently updated. The problem lies within joint rotation information, which changed from a 128bit format to a 48 bit format.

The old format actually was using a quaternion representation with 32 bit floats for each component (32 * 4 = 128 bit)

The new format was probably added for quantization purposes (serving better multiplayer performance). I played around trying to apply an euler angle representation (3 x 16 bit integers or 3 x 16 bit half floats) but i failed miserably so I think it should be still a quaternion representation of some sort.

I have located a model which has not changed at all structurally but obviously the model files have been updated to the newest format.

So I do have the byte data, I know the correct rotation angles (again from game data) that correspond to those values as well as the quaternion values from the old version of the files.

New Format Bytes       Rotation(deg) X,Y,Z       Quaternion
41 ED 3F ED 41 6D      0, 90.0, 89.99             0.5, 0.5, 0.5, 0.5
85 F7 2C A0 85 77      59.64, 90, 0               0.3516, 0.61347, -0.351634, 0.61347
AE CE FF BF FF 3F      -180, 0, 18.54             0.98 0.16 0 0
40 ED BE 92 BE 12      0,-90,90                   0.5, -0.5, -0.5, 0.5
DE 40 DA 42 CA 3C      -1.017, -3.633, 4.091      -0.009633918, -0.03154902, 0.03542599, 0.9988277

I'm currently searching the web for quaternion quantization techniques and I'm experimenting with different stuff, so if I find anything I'll make sure to update. Thanks in advance

\$\endgroup\$
  • 2
    \$\begingroup\$ The thing is that unless you give us some idea of what you are working with and who made the change to the numeric format, there is absolutely no way that any of us is going to be able to sit here and figure out the way the 48-bit format is encoded. It sounds like 12 bits per quaternion component, which is definitely non-standard. You need to find out what that encoding is. \$\endgroup\$ – Engineer Aug 13 '18 at 16:07
  • \$\begingroup\$ Well, I wish I had more information about that as well :/ Its a commercial game, I can tell you which one is it if its ok with stackoverflow, but other than that I also have no idea behind the change. I'm actually writing a model viewer which can preview models and their animations, and worked perfectly before using the uncompressed quaternions. If there is some more info that I can provide to help you out please let me know. PS: I just noticed that I didn't mention the origin of the file. I'll update the question. \$\endgroup\$ – Greg K. Aug 13 '18 at 16:48
  • 1
    \$\begingroup\$ As Arcane Engineer says, this is a very challenging problem to reverse engineer. We don't know whether any kind of compression /entropy coding has been applied, or whether part of the information might be looked up from a separate data source rather than fully-described in these 48 bits, or whether the orientations are influenced by parent bones, etc... So you might not get a satisfying answer at all. To improve your chances, more context about where the data came from and how it looks in the file structure, and more before/after pairs establishing the correct mapping would be helpful. \$\endgroup\$ – DMGregory Aug 13 '18 at 17:04
  • \$\begingroup\$ The quaternion values that I have provided are actually the "before" image. I think extra compression er entropy is really really far fetched for this application. I just re-compared the files and literally the only difference between the files are that the bytes regarding the old full quaternions are replaced from those 48bit entries. Other than that on the 2 sample entries that I have provided, the first one is the root node, so I think that in this particular case there are no related parent transformations. However I'll try to gather more old vs new values just for additional reference \$\endgroup\$ – Greg K. Aug 13 '18 at 17:32
  • 4
    \$\begingroup\$ there is a SE for reverse engineering, though you may want to collect more samples \$\endgroup\$ – ratchet freak Aug 14 '18 at 8:13
1
\$\begingroup\$

Took me 4 days of headache and research but I got there :D

So it turns out to be a modified CryEngine 3 48bit scheme.

Just to recap, when I started working on this, having 3 components I thought that they were supposed to be euler angles. The extra bit and values being all over the place out of the expected (-180,180) range ruled that out.

This is when I started researching about packing quaternions. After I tried different formats for smaller three schemes which failed because of the values being padded. Once I realized that, I kept unpacking the values as unsigned 16 bit values and I started skipping bits to find out the range of the values. I knew since the start that 0x3FFF somehow corresponded to 0 so this is the offset I used to move the values over the domain. Eventually I found that I had to skip just one bit and that 1/0x3FFF is also the normalization constant and soon enough I realized that the values lie within the (-1/sqrt(2), 1/sqrt(2)) range. This got me to a point where the values I calculated were 100% the same with the old quaternion values. The only thing that was changing things was the order, which is really easy to work out using the flag bits.

The main difference lies in the fact that despite all 3 components are 15bit values, they are padded with 1bit to fit in 2 bytes, and after unpacking those 3 bits are used as flags to indicate the component order. Other than that values are masked and after normalization they are brought into (-1/sqrt(2), 1/sqrt(2)) range.

//Load Rotations
UInt16 c_x, c_y, c_z;
UInt16 i_x, i_y, i_z;

c_x = readUint16();
c_y = readUint16();
c_z = readUint16();

i_x = (UInt16)(c_x >> 15);
i_y = (UInt16)(c_y >> 15);
i_z = (UInt16)(c_z >> 15);

ushort axisflag = (ushort) (i_x << 0 | i_y << 1 | i_z << 2);

//Mask Values
c_x = (UInt16) (c_x & 0x7FFF);
c_y = (UInt16) (c_y & 0x7FFF);
c_z = (UInt16) (c_z & 0x7FFF);

float norm = 1.0f / 0x3FFF;
float scale = 1.0f / (float) Math.Sqrt(2.0f);

float[] values = new float[4];
values[0] = ((float)(c_x - 0x3FFF)) * norm * scale;
values[1] = ((float)(c_y - 0x3FFF)) * norm * scale;
values[2] = ((float)(c_z - 0x3FFF)) * norm * scale;
//I assume that the calculated value is positive by default
values[3] = (float)Math.Sqrt(Math.Max(1.0f - values[0] * values[0] - values[1] * values[1] - values[2] * values[2], 0.0));

Quaternion q = new Quaternion();
switch (axisflag)
    {
        case 3:
           q = new Quaternion(values[3], values[0], values[1], values[2]);
           break;
        case 2:
           q = new Quaternion(values[0], values[1], values[3], values[2]);
           break;
        case 1:
           q = new Quaternion(values[0], values[3], values[1], values[2]);
           break;
        case 0:
           q = new Quaternion(values[0], values[1], values[2], values[3]);
           break;
        default:
           break;
    }
\$\endgroup\$
  • \$\begingroup\$ Could you elaborate a bit on the 'headache and research'? Doing so would make this this answer more useful to others trying to reverse engineer values. \$\endgroup\$ – Pikalek Aug 18 '18 at 3:23
  • 1
    \$\begingroup\$ Sure, after all this is why I posted the code as well :D \$\endgroup\$ – Greg K. Aug 18 '18 at 8:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.