Introducing the Half type!

Software Engineer

TheIEEE 754 specification defines many floating point types, including:binary16,binary32,binary64 andbinary128. Most developers are familiar withbinary32 (equivalent tofloat in C#) andbinary64 (equivalent todouble in C#). They provide a standard format to represent a wide range of values with a precision acceptable for many applications. .NET has always hadfloat anddouble and with .NET 5 Preview 7, we’ve added a newHalf type (equivalent tobinary16)!

AHalf is a binary floating-point number that occupies 16 bits. With half the number of bits as float, aHalf number can represent values in the range ±65504. More formally, theHalf type is defined as a base-2 16-bit interchange format meant to support the exchange of floating-point data between implementations. One of the primary use cases of theHalf type is to save on storage space where the computed result does not need to be stored with full precision. Many computation workloads already take advantage of theHalf type: machine learning, graphics cards, the latest processors, native SIMD libraries etc. With the newHalf type, we expect to unlock many applications in these workloads.

Let’s explore the`Half` type:

The 16 bits in theHalf type are split into:

Sign bit: 1 bit
Exponent bits: 5 bits
Significand bits: 10 bits (with 1 implicit bit that is not stored)

Despite that fact that the significand is made up of 10 bits, the total precision is really 11 bits. The format is assumed to have an implicit leading bit of value 1 (unless the exponent field is all zeros, in which case the leading bit has a value 0). To represent the number 1 in theHalf format, we’d use the bits:

0 01111 0000000000 = 1

The leading bit (our sign bit) is0, indicating a positive number. The exponent bits are01111, or15 in decimal. However, the exponent bits don’t represent the exponent directly. Instead, an exponent bias is defined that lets the format represent both positive and negative exponents. For theHalf type, that exponent bias is15. The true exponent is derived by subtracting15 from the stored exponent. Therefore,01111 represents the exponente = 01111 (in binary) - 15 (the exponent bias) = 0. The significand is0000000000, which can be interpreted as the number.significand(in base 2) in base 2,0 in our case. If, for example, the significand was0000011010 (26 in decimal), we can divide its decimal value26 by the number of values representable in10 bits (1 << 10): so the significand0000011010 (in binary) is26 / (1 << 10) = 26 / 1024 = 0.025390625 in decimal. Finally, because our stored exponent bits(01111) are not all0, we have an implicit leading bit of1. Therefore,

0 01111 0000000000 = 2^0 * (1 + 0/1024) = 1

In general, the 16 bits of a Half value are interpreted as -1^(sign bit) * 2^(storedExponent - 15) * (implicitBit + (significand/1024)). A special case exists for the stored exponent 00000. In this case, the bits are interpreted as -1^(sign bit) * 2^(-14) * (0 + (significand/1024)). Let’s look at the bit representations of some other numbers in theHalf format:

Smallest positive non-zero value

0 00000 0000000001 = -1^(0) * 2^(-14) * (0 + 1/1024) ≈ 0.000000059604645

(Note the implicit bit is 0 here because the stored exponents bits are all 0)

Largest normal number

0 11110 1111111111 = -1^(0) * 2^(15) * (1 + 1023/1024) ≈ 65504

Negative Infinity

1 11111 0000000000 = -Infinity

A peculiarity of the format is that it defines both positive and negative 0:

1 00000 0000000000 = -0

0 00000 0000000000 = +0

Conversions to/from float/double

A Half can be converted to/from a float/double by simply casting it:

float f = (float)half;Half h = (Half)floatValue;

AnyHalf value, becauseHalf uses only 16 bits, can be represented as afloat/double without loss of precision. However, the inverse is not true. Some precision may be lost when going fromfloat/double toHalf. In .NET 5.0, theHalf type is primarily an interchange type with no arithmetic operators defined on it. It only supports parsing, formatting and comparison operators. All arithmetic operations will need an explicit conversion to afloat/double. Future versions will consider adding arithmetic operators directly onHalf.

As library authors, one of the points to consider is that a language can add support for a type in the future. It is conceivable that C# adds ahalf type in the future. Language support would enable an identifier such asf16(similar to thef that exists today) and implicit/explicit conversions. Thus, the library defined typeHalf needs to be defined in a manner that does not result in any breaking changes ifhalf becomes a reality. Specifically, we needed to be careful about adding operators to theHalf type. Implicit conversions tofloat/double could lead to potential breaking changes if language support is added. On the other hand, having aFloat/Double property on theHalf type felt less than ideal. In the end, we decided to add explicit operators to convert to/fromfloat/double. If C# does add support forhalf, no user code would break, since all casts would be explicit.