I'm trying to interpolate coordinates in my fragment shader. Unfortunately if close to the upper edge the interpolated value of fVertexInteger seems to be rounded up instead of beeing floored. This happens above approximately fVertexInteger >= x.97.


  • floor(64.7) returns 64.0 --> correct

  • floor(64.98) returns 65.0 --> incorrect

The same happens on ceiling close above x.0, where ceil(65.02) returns 65.0 instead of 66.0.

Q: Any ideas how to solve this?


  • GL ES 2.0 with GLSL 1.0
  • highp floats are not supported in fragment shaders on my hardware
  • flat varying hasn't been a solution, because I'm drawing TRIANGLE_STRIP and can't redeclare the provoking vertex (only OpenGL 3.2+)

Fragment Shader:

varying float fVertexInteger;
varying float fVertexFraction;
void main() {

    // Fix vertex integer
    fixedVertexInteger = floor(fVertexInteger);

    // Fragment color
    gl_FragColor = vec4(
        fixedVertexInteger / 65025.0,
        fract(fixedVertexInteger / 255.0),
        1.0 );

  • 1
    \$\begingroup\$ Out of curiousity, how did you extract those values? Are you stepping through the shader somehow? \$\endgroup\$ Nov 13, 2013 at 16:48
  • \$\begingroup\$ I'm using glReadPixels() in my Android app to retrieve the color values at a specific pixel. \$\endgroup\$
    – Peter K.
    Nov 13, 2013 at 20:20

1 Answer 1


You said highp floats are not supported on your hardware, so I assume you're using mediump. The OpenGL ES shading language spec says that mediump has a minimum relative precision of 2^(-10), or 1/1024. So if a and b are two adjacent mediump values, the ratio (b-a)/a can be as large as 1/1024.

Unfortunately, if you look at 65.0 and 64.98, the ratio (65.0 - 64.98) / 64.98 is considerably less than 1/1024. Actually it's about three times smaller than that. So it seems likely that your hardware's mediump floats are simply not precise enough to represent the difference between 64.98, 65.0, and 65.02.

  • \$\begingroup\$ Makes sense. So I'll have to alter this method or probably get along with this decreased accuracy. Thanks a lot for this explanation! It's somehow disappointing that there are so many constraints in OpenGL ES... \$\endgroup\$
    – Peter K.
    Nov 13, 2013 at 20:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .