Values of floating-point types

Contents

1. Revision history

1.1. Changes since R1

• Simplified §2. Introduction to no longer duplicate §3. Q&A questions
• Added §2.1. Representation levels and defining value, introducing a model of value consistent with Level 2 of ISO/IEC 60559, as well as an associated representation property consistent with Level 4
• Ensured in § [basic] that zero is always representable in a floating-point type
• Discussed in §3.8. Can an extended floating-point type have no finite values? that zero should always be representable in a floating-point type

1.2. Changes since R0

• Rationalized the status quo in §3.5. Is negative zero negative? What about infinity and NaN?
• Rebased §6. Wording on N5032
• In §6. Wording, fixed improper handling of negative zero floating-point-literals
• In §6. Wording, referenced [CWG3129]
• In §6. Wording, improved handling of signaling NaNs by relaxing the definition of adheres to ISO/IEC 60559

2. Introduction

The core language wording in the C++ standard does not specify what values a floating-point type may represent. There are a few questions that have no obvious answer, such as whether negative zero is distinct from positive zero, and many other questions listed in the Q&A.

Bits of information may be found in various parts of the standard, such as in the concept of "adhering to ISO/IEC 60559", numeric_limits requirements, the inheritance of C features such as std::fpclassify, etc. However, some of these questions are so deeply unclear that a core issue alone wouldn't be sufficient to solve the problem.

The goal of this paper is to answer these questions, not by making any evolutionary changes to the language, but by investigating what the status quo is and turning that into wording.

2.1. Representation levels and defining value

When discussing R1 of this paper in Brno, SG6 discussed the issue of what constitutes distinct floating-point values, and agreed that there is only one NaN value:

POLL: SG6 agrees that by “value” of a floating point type we generally mean the sense of value referred to as “Level 2” in Table 3.1 of ISO/IEC 60559 (2020): {-∞…-0} ∪ {+0…+∞} ∪ {NaN}; “Floating point data — an algebraically closed system” (As opposed to “Extended real numbers” or “Representations of floating point data”)

SF	F	N	A	SA
5	10	0	0	0

Outcome: strong consensus!

This poll refers to the following table in ISO/IEC 60559:

Table 3.1 — Relationships between different specification levels for a particular format

Level 1	{−∞ … 0 … +∞}	Extended real numbers.
many-to-one ↓	rounding	↑ projection (except for NaN)
Level 2	{−∞ … −0} ∪ {+0 … +∞} ∪ {NaN}	Floating point data — an algebraically closed system.
one-to-many ↓	representation specification	↑ many-to-one
Level 3	(sign, exponent, significand) ∪ {−∞, +∞} ∪ qNaN ∪ sNaN	Representations of floating-point data.
one-to-many ↓	encoding for representations of floating-point data	↑ many-to-one
Level 4	0111000...	Bit strings.

2.1.1. Template argument equivalence problem

However, this Level 2 model runs into a problem with the existing C++ mechanics: §3.14. How does template argument equivalence work for floating-point types? explains that every possible bit pattern of a floating-point type is treated as a distinct value, suggesting that our definition of value is Level 4. After all, a template argument is simply a value subject to lvalue-to-rvalue conversion, so this can only work if prvalues carry every minute detail of the object representation.

2.1.2. Issues with Level 4 values

However, defining value to be at Level 4 is not a good idea because it lacks any kind of abstraction. Simply saying that std::acos(1) returns the value zero would be a wording defect because the wording is ambiguous. We didn't specify the the sign bit of the result, and we didn't specify the quantum exponent of the result in the case of decimal floating-point types. We would need to say something like

std::acos(1) returns an implementation-defined value representing the number zero.

Another problem is that the wording in [conv.fpprom] would be defective because it states

A prvalue of type float can be converted to a prvalue of type double. The value is unchanged.

If every bit pattern is a distinct value, then changing the amount of bits in the operand is obviously changing the value, so this requirement becomes unimplementable.

In general, a Level 4 model of value either makes our wording defective or forces us to litter every part of the specification with implementation-defined or unspecified, and also results in a significant deviation from the C23 wording, which we generally try to stay symmetrical with.

2.1.3. Modeling excess precision

Another problem to solve is that C++ allows floating-point values to be represented with excess precision.

Excess precision is not just a hypothetical issue; GCC uses excess precision for std::float16_t to use the hardware's binary32 support, rather than strictly rounding every float16_t operation to binary16.

Excess precision with a Level 4 value is simply nonsensical. Clearly, value is somewhat abstract and detached from bit patterns, and there are some values that do not correspond to any bit pattern.

2.1.4. Solution: associated representations

The standard is currently contradictory on what constitutes a value, which is ultimately caused by using the same term to talk about different levels of abstraction. Simply saying that our value is Level 2 breaks some things, and saying it is Level 4 breaks other things.

To solve this, I propose a new term associated representation that is a property of values and that provides the additional Level 4 information so that value can be Level 2:

• The associated representation of a floating-point value inside an object is the value representation of that object.
• When lvalue-to-rvalue conversion loads the prvalue out of an object, the associated representation is preserved.
• In various other cases, the associated representation is unspecified, such as when computing the sum of two floating-point numbers.

This new term adds immense flexibility:

• By strictly specifying associated representations or requiring them to be implementation-defined (and thus deterministic), we can express deterministic floating-point implementations in our wording. This is useful if we ever want to add a deterministic mode, such as implemented by WASM's deterministic profile.
• By leaving associated representations unspecified, we can express optimization opportunities and additional implementation freedom.
• Excess precision is allowed in those situations where the associated representation is unspecified. Since value is Level 2, it is still an abstract term and capable of referring to values that do not have a corresponding bit pattern.
• We can word the behavior of CPU architectures where adjustments like canonicalization or flushing of subnormals to zero take place immediately upon lvalue-to-rvalue conversion, i.e. when loading a value into a register:
  • Subnormal flushing would be expressed as the implementation not supporting subnormals due to precision constraints. When an object stores a subnormal anyway (perhaps as the result of std::start_lifetime_as), the flushing would result in no change in value (because the value is zero), but would change the associated representation.
  • Similarly, canonicalization of NaNs does not change the value (because there is only one NaN value), but does change the NaN payload.
• We can define template argument equivalence in terms of associated representations and ensure that the lvalue-to-rvalue conversion taking place during template argument passing preserves the associated representation.

2.1.5. Solving the problem of quiet_Nan() and signaling_NaN()

One pre-existing problem is that numeric_limits::quiet_NaN() and numeric_limits::signaling_NaN() return NaN values by value. Ever passing signaling NaNs by value is a problem because it disallows the implementation of raising floating-point exceptions when loading a signaling NaN from memory or when passing it around in registers.

The wording of these two functions only makes sense at Level 3, not at either previously mentioned specification level:

• If value is Level 2, this actually means that both functions have the same result (a NaN), and it could even mean that quiet_NaN() returns a signaling NaN and signaling_NaN() returns a quiet NaN.
• If value is Level 4, this wording is defective because quiet NaN does not fully specify the bit pattern of the result. We should have said something about the payload being implementation-defined or unspecified, and should have said something about the sign bit.

The associated representation model lets us square that circle: we can say that both quiet_NaN() and signaling_NaN() return the same value but with different associated representations.

3. Q&A

In the following subsections, the paper tries to find a good answer to the questions above. These answers are primarily based on the existing wording and on existing implementation practice.

3.1. Do infinity and NaN exist from a core language perspective?

Yes. [basic.fundamental] mentions infinity. While NaN is not mentioned explicitly, numeric_limits::quiet_NaN() implies it.

At the very least, floating-point types may represent

• zero, for zero-initialization to make sense,
• negative zero, since both C and C++ already mention this term many times, and
• infinity, qNaN, and sNan, [numeric.limits] to make sense and to have types that adhere to ISO/IEC 60559.

3.2. Can there be an unsigned zero, infinity, or NaN value, or are all floating-point values signed?

There can be fully unsigned floating-point numbers. The C standard explicitly mentions unsigned infinity and unsigned zero, and different NaN signs are typically not distinct values. The C++ standard presumably inherits the C model because all of the <cmath> functions are stated to work like C23's <math.h>. There is no C++ restriction on fpclassify.

Furthermore, [numeric.limits.members] states that numeric_limits::is_signed is meaningful for all specializations, so there is presumably no requirement that a floating-point type or any of its values are signed.

3.3. Conversely, can there be a signed zero, infinity, or NaN value?

Presumably. The C++ standard mentions negative zero and negative infinity in a number of places. NaN with negative sign bit and NaN with positive sign bit are usually not considered distinct values (e.g. ISO/IEC 60559), but the C++ standard has no wording that would prohibit that distinction.

3.4. Can there be a negative zero and an unsigned infinity, or is the "signedness requirement" all-or-none?

It's not all-or-none. There are floating-point types that adhere to ISO/IEC 60559, and these do not have a distinct negative and positive NaN, so it cannot be all-or-none.

3.5. Is negative zero negative? What about infinity and NaN?

No. The wording uses negative in the less than zero sense ([complex.numbers] polar, complex). The C23 standard defines a negative value to values which are less than zero, so negative zero and negative NaN (NaN with negative sign bit) are not negative.

Negative infinity is negative because it compares less than zero.

3.6. Is negative NaN distinct from positive NaN?

Presumably. While ISO/IEC 60559 does not consider NaNs with different sign bit to be distinct values, the C++ standard does not prohibit such a model.

3.7. Are different NaN payloads distinct values?

Sometimes. numeric_limits::quiet_NaN() and numeric_limits::signaling_NaN() are two NaNs distinguished only by payload. However, not every implementation treats these distinctly. For example, when compiling to WASM, none of the f32 or f64 instructions handle signaling NaNs.

3.8. Can an extended floating-point type have no finite values?

No. This would make numeric_limits members such as epsilon() worded in a nonsensical way.

Furthermore, C23 §6.7.11 paragraph 14 involves initialization to zero of arithmetic types, so C23 necessitates zero. By proxy, acos(1) must return zero because zero exists, and C++'s std::acos(1) must return zero as well. This doesn't necessarily apply to an extended floating-point type that is not C-compatible, but any type that exists in practice is subject to these requirements.

It is reasonable to clarify that zero exists explicitly in the wording.

3.9. Does the core language need to distinguish between normal/subnormal numbers?

No. Subnormals either get flushed to zero, meaning that they are alternative representations of zero, numerically, or they are simply finite numbers like any other. The classification of normal and subnormal is an implementation detail of the floating-point format.

3.10. Are there values beyond finite values, infinity, and NaN?

The exotic and surprising part is that floating-point types may represent additional implementation-defined values beyond finite values, infinity, and NaN. This is backed by

• C23 §5.2.5.3.3 [Characteristics of floating types <float.h>] paragraph 8, and
• C23 §7.12 [Mathematics <math.h>] paragraph 12.

These paragraphs support the existence of such additional implementation-defined classifications. C++ inherits the behavior of fpclassify from <math.h>, so it must support the same classifications to be compatible.

While neither the C23 wording nor the C++ wording handles these extra implementation-defined classifications well, they nonetheless exist, and it seems like an unmotivated breaking change to drop support for them.

3.11. What does it mean for a type to adhere to ISO/IEC 60559?

It just means that the value representation is one of binary16, binary32, binary64, binary128, or some extended or decimal floating-point format specified in ISO/IEC 60559.

When it comes to operations, the C++ standard specifies no mapping between expressions (e.g. /) and the ISO/IEC 60559 operations (e.g. division). In fact, it seems to deliberately deviate from the ISO/IEC 60559 operations by declaring division by zero to be undefined behavior, even for floating-point types ([expr.mul]).

While it would be desirable to align the C++ operations with ISO/IEC 60559 operations, this would require significant wording effort. There exists no core wording that even attempts at doing so. Note that implementations don't always compute results correctly rounded (with the greatest available precision), while the ISO/IEC 60559 operations are usually correctly rounded. Therefore, aligning the C++ expressions with ISO/IEC 60559 may require changes to most implementations.

3.12. Is 0.0 positive or negative zero?

Presumably positive. The C++ standard does not mandate a specific sign for floating-point-literals, but no implementation makes them negative, nor does it make practical sense to do so. We should mandate that literals have no negative sign.

3.13. Are arithmetic operations required to preserve the sign of zero?

Presumably no. While the ISO/IEC 60559 multiplication operation preserves the sign bit when multiplying with 1, the C++ standard does not clearly mandate such behavior.

Especially considering that not every type adheres to ISO/IEC 60559, we should require a specific handling of sign bits, explicitly. For example, the unary - operator should be required to flip the sign bit even for zero, otherwise -0.0 may not be a spelling of negative zero, as users rely on.

3.14. How does template argument equivalence work for floating-point types?

In practice, it is based on bitwise identical values. [temp.type] states that

Two values are template-argument-equivalent if they are of the same type and

• they are of integral type and their values are the same, or
• they are of floating-point type and their values are identical, or
• […]

The distinction between same and identical is not obvious. It was added during C++20 NB comment resolution by [P1907R1], after [P1714R1] (the paper which originally added support for floating-point template parameters) was rejected. The original paper used the terminology identical value representations, which makes the intent obvious, unlike the current wording.

GCC, Clang, and MSVC implement the design of the original paper by mangling the bit-casting to an integer and mangling it into the name.

_Z1fILf7fc00000EEvv:
        ret
_Z1fILf7fc00001EEvv:
        ret

The wording should be clarified to match the design of the original paper and to match what implementations actually do. Creating a distinct instantiation for every bit pattern is the most useful behavior anyway; it would be impossible to deliberately wrap a NaN payload of choice in std::constant_wrapper otherwise, for example.

4. Impact on the standard

The clarify the floating-point specification, a bit of additional wording is required.

This paper only picks the low-hanging fruits, so to speak. In the long run, specifying the handling of NaNs and infinities by C++ expressions, documenting ISO/IEC 60559 conformance, and other large changes may be desirable. However, those would require much greater changes.

5. Impact on implementations

All proposed wording changes document the current behavior of major implementations.

6. Wording

The changes are relative to [N5032].

[lex]

Change [lex.fcon] paragraph 3 as follows:

If [CWG3129] has not yet been approved at the time of writing, add a new example attached to [lex.fcon] paragraph 3:

Otherwise, if the proposed resolution of [CWG3129] has been merged, change the existing example as follows:

[basic]

Immediately preceding [basic.fundamental] paragraph 13, insert several new paragraphs:

Immediately preceding [basic.fundamental] paragraph 14, insert a new paragraph:

[expr]

At the end of [expr.pre], insert a new paragraph:

Change [conv.lval] paragraph 3 as follows:

Change [conv.fpint] paragraph 2 as follows:

Change [expr.unary.op] paragraph 8 as follows:

Change [expr.spaceship] paragraph 4 as follows:

Change [expr.rel] paragraph 6 as follows:

Change [expr.eq] paragraph 8 as follows:

[temp]

Immediately preceding [temp.type] paragraph 2, insert a new paragraph:

Change [temp.type] paragraph 2 as follows:

[support]

Change [numeric.limits.members] as follows:

Change [cmp.alg] paragraph 2 as follows:

[meta]

Change [meta.unary.prop] paragraph 10 as follows:

7. References