Python float 32bit to half float 16bit

Question

I'm trying to write a 3D model exporter addon for Blender in Python and need some help.

The spec of the 3D format uses some compression on the vertices, there is a vertex buffer that contains vertices as 32bit floats. When this is compressed it is stored as 16bit float or half precision float.

I've seen lots of examples online of code in C to convert a 32bit float to 16bit float but not much luck with Python.

I was wondering if anyone here can help?

So far I've attempted to take some of the code here conversion of float and put it into Python. I managed to get this working and testing with a triangulated cube rendered only half the cube.

F16_EXPONENT_BITS = 0x1F
F16_EXPONENT_SHIFT = 10
F16_EXPONENT_BIAS = 15
F16_MANTISSA_BITS = 0x3ff
F16_MANTISSA_SHIFT =  (23 - F16_EXPONENT_SHIFT)
F16_MAX_EXPONENT =  (F16_EXPONENT_BITS << F16_EXPONENT_SHIFT)

for byte in self._byteBuffer:
   f32 = int(byte)
   f16 = 0
   sign = (f32 >> 16) & 0x8000
   exponent = ((f32 >> 23) & 0xff) - 127
   mantissa = f32 & 0x007fffff

   if exponent == 128:
       f16 = sign | F16_MAX_EXPONENT
       if mantissa:
           f16 |= (mantissa & F16_MANTISSA_BITS)
   elif exponent > 15:
       f16 = sign | F16_MAX_EXPONENT
   elif exponent > -15:
       exponent += F16_EXPONENT_BIAS
       mantissa >>= F16_MANTISSA_SHIFT
       f16 = sign | exponent << F16_EXPONENT_SHIFT | mantissa
   else:
       f16 = sign
   file.write(struct.pack(">H",f16))

Answer 1

After some trial and error and trying various different things I finally got something that works for me. I hope this helps someone else - I've posted the code I used on my website. Quoted here:

---

I’ve been working with python for a while to create some addons for Blender which allowing me to import and export various formats that aren’t supported by default. One of these file formats I’ve been working with compresses the vertices/uv/tangent/normal data to reduce the space used. This data is usually stored as 32bit floating point values but in this case it is reduced to 16bit floating point or Half Precision floating point. Halfing the final size of this data.

I had a hard time recreating this with python, I’d never used python before starting using Blender so my understanding was still fairly new and most examples online I failed to understand or struggled to implement.

Finally I cobbled together something that works from different examples online and got the results I wanted. Like most things on my site I felt the need to share this with anyone else who may be trying a similar thing.

Here is my class in python:

class Float16Compressor:
    def __init__(self):
        self.temp = 0

    def compress(self,float32):
        F16_EXPONENT_BITS = 0x1F
        F16_EXPONENT_SHIFT = 10
        F16_EXPONENT_BIAS = 15
        F16_MANTISSA_BITS = 0x3ff
        F16_MANTISSA_SHIFT =  (23 - F16_EXPONENT_SHIFT)
        F16_MAX_EXPONENT =  (F16_EXPONENT_BITS << F16_EXPONENT_SHIFT)

        a = struct.pack('>f',float32)
        b = binascii.hexlify(a)

        f32 = int(b,16)
        f16 = 0
        sign = (f32 >> 16) & 0x8000
        exponent = ((f32 >> 23) & 0xff) - 127
        mantissa = f32 & 0x007fffff

        if exponent == 128:
            f16 = sign | F16_MAX_EXPONENT
            if mantissa:
                f16 |= (mantissa & F16_MANTISSA_BITS)
        elif exponent > 15:
            f16 = sign | F16_MAX_EXPONENT
        elif exponent > -15:
            exponent += F16_EXPONENT_BIAS
            mantissa >>= F16_MANTISSA_SHIFT
            f16 = sign | exponent << F16_EXPONENT_SHIFT | mantissa
        else:
            f16 = sign
        return f16

    def decompress(self,float16):
        s = int((float16 >> 15) & 0x00000001)    # sign
        e = int((float16 >> 10) & 0x0000001f)    # exponent
        f = int(float16 & 0x000003ff)            # fraction

        if e == 0:
            if f == 0:
                return int(s << 31)
            else:
                while not (f & 0x00000400):
                    f = f << 1
                    e -= 1
                e += 1
                f &= ~0x00000400
                #print(s,e,f)
        elif e == 31:
            if f == 0:
                return int((s << 31) | 0x7f800000)
            else:
                return int((s << 31) | 0x7f800000 | (f << 13))

        e = e + (127 -15)
        f = f << 13
        return int((s << 31) | (e << 23) | f)

and here is how to use it

#read half float from file and print float
h = struct.unpack(">H",file.read(struct.calcsize(">H")))[0]
fcomp = Float16Compressor()
temp = fcomp.decompress(h)
str = struct.pack('I',temp)
f = struct.unpack('f',str)[0]
print(f)

#write half float to file from float
fcomp = Float16Compressor()
f16 = fcomp.compress(float32)
file.write(struct.pack(">H",f16))

Answer 2

h = ((f32>>16)&0x8000)|((((f32&0x7f800000)-0x38000000)>>13)&0x7c00)|((f32>>13)&0x03ff)

The above should work assuming that none of the below "fatal flaws" are fatal in your situation: "The above expression suffers from a few fatal flaws, however. It doesn't handle zero, Infinity, NaN, or small float numbers which are only representable as subnormal half-floats" Otherwise I would make the table as demonstrated in near the bottom of the paper.

Source

Answer 3

For some reason none of the other answers worked for me. So I tried to convert the C solution given here. I used the ctypes module to emulate C unions and types. I have no idea how efficient this is, or even if it is crossplatform, but it is working for me. If I haven't made any mistake it should be fully IEEE compliant and branchless.

import ctypes

class Bits(ctypes.Union):
    _fields_ = [('f', ctypes.c_float), ('si', ctypes.c_int), ('ui', ctypes.c_uint)]

def compress(f):
    SIGN_N = 0x80000000
    SHIFT_SIGN = 16
    MUL_N = 0x52000000
    MIN_N = 0x38800000
    INF_N = 0x7F800000
    MAX_N = 0x477FE000
    SHIFT = 13
    INF_C = INF_N >> SHIFT
    NAN_N = (INF_C + 1) << SHIFT
    MIN_C = MIN_N >> SHIFT
    MAX_C = MAX_N >> SHIFT
    SUB_C = 0x003FF
    MAX_D = INF_C - MAX_C - 1
    MIN_D = MIN_C - SUB_C - 1

    v = Bits(f=f)
    s = Bits(si=MUL_N)
    sign = v.si & SIGN_N
    v.si ^= sign
    sign >>= SHIFT_SIGN
    s.f *= v.f
    v.si ^= (s.si ^ v.si) & -(MIN_N > v.si)
    v.si ^= (INF_N ^ v.si) & -((INF_N > v.si) & (v.si > MAX_N))
    v.si ^= (NAN_N ^ v.si) & -((NAN_N > v.si) & (v.si > INF_N))
    v.ui >>= SHIFT
    v.si ^= ((v.si - MAX_D) ^ v.si) & -(v.si > MAX_C)
    v.si ^= ((v.si - MIN_D) ^ v.si) & -(v.si > SUB_C)

    return v.ui | sign