4
\$\begingroup\$
float2 uv = float2(fX % 1, fY % 1) * 1/16;

In the above example 1/16 is treated as a float, multiplying the float2 by 0.0625, but in:

float2 uv = float2(fX % 1, fY % 1) * 1/16 + float2(1/16,0);

The second 1/16 is taken as an integer and the second float2 becomes (0,0).

What's the order in which a specific type is used. And is there any suffix to force a float? I know 1.0/16.0 would work.

\$\endgroup\$
5
  • \$\begingroup\$ In C# the suffix F forces a float instead of either double (the default floating-point type) or any integer type. \$\endgroup\$ Commented Jul 14, 2013 at 18:05
  • 1
    \$\begingroup\$ @PieterGeerkens I know, but this is isn't C#? \$\endgroup\$
    – Levi H
    Commented Jul 14, 2013 at 18:09
  • \$\begingroup\$ @PieterGeerkens: this is for HLSL, not C#. The rules are a bit different. There is no f suffix in HLSL. Anything with a decimal is a 32-bit single-precision float and then you can use the L suffix to make it a 64-bit double. \$\endgroup\$ Commented Jul 14, 2013 at 18:10
  • \$\begingroup\$ @SeanMiddleditch: Yes, I read the tag - but since it is another Microsoft product there was a chance it was similar. \$\endgroup\$ Commented Jul 14, 2013 at 18:21
  • \$\begingroup\$ @PieterGeerkens: fair enough, but I figured it was worth clarifying. :) \$\endgroup\$ Commented Jul 14, 2013 at 18:29

1 Answer 1

5
\$\begingroup\$

Use a decimal to mark a type as a float. 1.0 is always floating point. Hence 1.0/16 or 1/16.0 will always be floating point.

Numeric literals are treated as floats if used in a floating-point context. Remember that operators like * and / are left-associative. So in your first example, the code is equivalent to (float(...) * 1) / 16. If you put the 1/16 in parentheses then it would be equal to 0 and that would be multiplied with the vector, e.g float2(...) * (1/16) == float2(0,0)

In your second example, the 1/16 is seen by itself, computed (as integers, because there's no float involved), and then the result of the expression is converted to float for the float2 construction. You need to cause either the 1 or the 16 to be a float in order for the / operator to be in a float context and force the other value to be treated as a float.

\$\endgroup\$
1
  • \$\begingroup\$ I voted to move the question to StackExchange but saw no answers to this question there. Hopefully this answer can be moved with the question if the vote goes through. \$\endgroup\$ Commented Jul 14, 2013 at 17:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .