Exploring The Precision Of Floating Point Numbers
Sanglard explains how floating point numbers are generally displayed in modern
computers (using the
IEEE754 Norm). His alternative perspective using
buckets and offsets is brilliant and explains a fact that may be
overlooked when using the standard explanation: the accuracy of floating point
numbers declines with a factor of 2 by each bucket! That is because the same
‘amount’ of numbers is dispersed in any bucket, but the size of the bucket
doubles every time.
I wanted to empirically show this and wrote this short C program:
One reason Unions are useful
This program uses a C union. Unions allow you to use the same memory for
multiple members, and every time you write one member, it also changes the
others. This may seem useless, but it is not. In this case, it allows you to
write to memory using the memory representation of
float, while reading the
same content of memory as individual bits. The union has 2 members, one
allowing to write and read a float, the other resembling the bit pattern of a
float as defined in the
IEEE754 Norm. After writing to the float, you can
read and manipulate the exponent and mantissa from the embedded struct.
Printing any memory content as binary [print_any(content pointer, size)]
Casting the pointer to the memory content to a byte pointer (
allows us to index individual bytes. To access individual bits, we need to
generate a bit mask for each of them. We simply start at
0b1000 0000 and
keep shifting left while using the index to access a single bit and print it.
Since the bit at position
n will still be interpreted as being of value
we need to check if it is set and print a
1 if this is the case, hence the
We need this function to inspect a floating point number.
Floating point numbers are discrete, too… [increase_float(content)]
We imagine floats as being very fine-grained. That is exactly what they are - they do not allow for arbitrary accuracy, they are not what mathematicians would call dense.
That means, we can take a float and increase it by exactly one tiny amount,
yielding the next-larger float in that bucket! That is what the function
We construct a
float_parts union using the float as content. Then, we look
at the mantissa: if it is filled to the brim, we reset it to 0 and increase the
exponent by one. Otherwise, we simply increase the mantissa.
Putting it together: How accuracy declines with growing exponent [print_accuracy(low, high)]
Here we keep increasing a float, and when the ‘step size’ changes, we log it. We also print the factor between the last and the current step size.
This function runs way faster than I expected for some reason - it visits every
float there is between
There you have it! Empirical demonstration of how inaccurate floats (and doubles, just later) really are.