Convert integer to float while also dividing down the scale, without data loss

Question

I'm working on a game, and my intent is to avoid use of floating point for unit positions. To that end, I'm using 32-bit integers for all positions, with a millimeter scale.

However, for rendering, I know the GPU expects everything to be float, so I need to convert from millimeter-scale integer to meter-scale float. Sounds simple, but I'd like to know the most efficient way to perform these two conversions at the same time without losing something along the way.

I know I can convert my 32-bit integer to a double and know that it will all fit, before dividing by 1000 to get meters and converting the result to a float:

int i = 1234567890;
double d = i / 1000.0;
f = (float)d;

But I wonder if that's taking more steps than is necessary. The only other option I can think of is to do something like:

int i = 1234567890;
float f = (i % 1000) / 1000.0f;
f += i / 1000;

But that requires multiple divides and more steps than using a double.

Is there perhaps another existing method of performing this sort of conversion? Or are these my only real options?

Answer 1

I think you're making this too complicated. The trick to preserving precision in your floating point numbers is just to keep their magnitude small. No chaining through doubles or two-stage fractional/whole conversion required.

If you want millimetre accuracy, single-precision floats will keep that as long as your numbers are less than about 16 km.

(If your play space is even smaller than that, then you actually lose precision with your millimetre integer scheme, so you might as well stick with floats throughout)

"But my world is bigger than 16 km" — that's not a problem, because the immediate neighbourhood of your camera, where millimetre-sized errors might actually be visible, is not.

When you compute the matrices to send down the pipe to your GPU, all you need to do is form the translation component as a relative offset from the camera.

Vector3Float translation = (object.integerPosition - camera.integerPosition) / 1000f;

So as far as the GPU is concerned, your camera is always at (0, 0, 0) this way, giving you maximum precision in any direction you look. Objects a very long way from the camera can still suffer precision loss, but a 1 mm offset over a 16 km distance barely changes the direction of the vector at all — your rounding to the pixel grid of the screen is actually a much larger source of error than anything you'd get from floats in this scenario.

The other members of the object transformation matrices will already tend to be near the -1...1 range where floats have very high precision, so you don't have to do anything fancy with them.

This goes in general for other values too. If it's an absolute measure, like a position in the world or a time stamp, use integers so that you have the same precision no matter where/when you are. If it's a relative measure, like an offset/displacement, duration (like deltaTime), velocity, etc. then use a float. This gets you consistent relative precision, where the errors stay small in proportion to the quantity you're measuring.

Answer 2

In a comment I mentioned in passing that you could switch to representing 1024ths of meters instead of millimeters. That also helps with representability generally. Your code would look like this:

int i = 1234567890;
float f = ((float)i) / 1024.f;

You don't need to worry about representability at all here. The temporary float value will already be the best representation of i possible in a single precision float, and the divisor 1024.f is not only a perfectly representable floating point number (feel free to write it with the wonky hexadecimal floating literal notation if you prefer) but will actually reduce to a trivial subtraction of the exponent of your floating point value when executed.

Of course you (and your compiler) will probably notice that a division isn't necessary at all at this point. The above code could we written with a similar multiplication (or even maybe some fancy hardware instructions):

int i = 1234567890;
float f = ((float)i) * 0x1p-10f;

(0x1p-10f is just a direct representation of the value 1.f / 1024.f)

Answer 3

Since you mentioned GPU, I will assume you mean a modern one. In this case, a conversion within a shader per vertex is going to have a performance impact of a single drop falling into the ocean. You will be able to benchmark it, but not see it.

Out of curiosity, why do you think you need integers ? I ask, despite having written a fully-integer SW rasterizing pipeline, but that was on sub-100 MHz CPUs. It does make very little sense on current CPU+GPU combos, though.