[AGC] Emit v_pk_fma_f16 with exact single rounding (#420)

Completes the fused-FMA slice deferred by the VOP3P first slice (#145).
v_pk_fma_f16 previously failed emission loudly because an f32
multiply-add followed by an f16 pack rounds twice; the pinned miss is
fma(0x4100, 0x7522, 0x04EA) = 0x7A6B fused vs 0x7A6A via f32.

The f32 product of two f16 values is exact, so only the addition needs
correcting: compute sum = RN(product + addend), recover the exact
residual with Knuth 2Sum, and if the sum is inexact with an even
significand, step one ulp towards the true value. That is round-to-odd,
and rounding the f32 result to f16 with round-to-nearest-even then
matches a true fused f16 FMA exactly (24 significand bits >= 11 + 2).
Inf/NaN inputs turn the residual into NaN, the ordered compare skips the
parity fix, and IEEE special behaviour passes through unchanged. The
op_sel/op_sel_hi/neg_lo/neg_hi source modifiers apply to src2 through
the existing operand path; clamp stays rejected like the other packed
ops.

Every op in the 2Sum chain is decorated NoContraction: without it the
AMD RDNA3 Windows driver folds the sequence, collapses the residual to
zero, and the midpoint case decays to the double-rounded result. This
was caught by running the emitted shader on a real device (see below).

Verification:
- A mirror of the emitted sequence was checked against an exact
  integer reference (every finite f16 is m * 2^-24, so a*b + c is an
  exact Int128 multiple of 2^-48, rounded once to f16 RNE) across 34M
  cases: directed midpoint pins, random sweeps over all operand
  classes, tiny-addend midpoint stress, subnormal products, and
  Inf/NaN propagation. 0 mismatches.
- ShaderDump gains a pk-f16 program covering all five packed opcodes,
  both fma modifier paths, and the pinned constants; all programs
  decode and emit.
- The executable exec program now computes the pinned fma and its
  negated-addend twin (0x7A6B7A6B / 0x7A6A7A6A, straddling an f16
  midpoint) and stores them at offsets 20/24; GpuConformance checks
  both on device. All values match on an AMD Radeon RX 7700 XT.
This commit is contained in:
João Victor Amorim
2026-07-18 21:24:42 -03:00
committed by GitHub
parent 09bd4f028b
commit 3005babab8
4 changed files with 123 additions and 14 deletions
@@ -12,6 +12,11 @@
// [2] v_mul_lo_i32 -> low 32 bits of the same product
// [3] store attempted with EXEC=0 -> must NOT land (sentinel remains)
// [4] store after EXEC restored -> 1.5f (0x3FC00000)
// [5] v_pk_fma_f16 fma(2.5h, 21024h, 7.496e-5h) -> 0x7A6B packed; the exact
// sum sits just above an f16 midpoint, so a double-rounded f32
// multiply-add would give 0x7A6A instead
// [6] the same fma with the addend negated -> 0x7A6A packed (just below the
// same midpoint), pinning the opposite rounding direction
// Every other word of the buffer must still hold the sentinel afterwards.
//
// Creating the compute pipeline doubles as a driver-acceptance check for the
@@ -35,6 +40,12 @@ var expectedHi = (uint)(product >> 32);
var expectedLo = (uint)product;
var expectedRestored = BitConverter.SingleToUInt32Bits(1.5f);
// v_pk_fma_f16 of (0x4100, 0x7522, 0x04EA) per lane: the exact product
// 2.5 * 21024 = 52560 is an f16 tie (between 0x7A6A and 0x7A6B), so the tiny
// addend decides the rounding direction under a single fused rounding.
const uint ExpectedPkFma = 0x7A6B_7A6B;
const uint ExpectedPkFmaNeg = 0x7A6A_7A6A;
unsafe
{
var spvPath = args.Length > 0
@@ -352,6 +363,8 @@ unsafe
("v_mul_lo_i32 lo(0x7FFFFFFF*0x10003)", words[2], expectedLo),
("exec=0 store suppressed (offset 12 sentinel)", words[3], Sentinel),
("store after exec restore (offset 16)", words[4], expectedRestored),
("v_pk_fma_f16 fused rounds up at midpoint", words[5], ExpectedPkFma),
("v_pk_fma_f16 neg addend rounds down", words[6], ExpectedPkFmaNeg),
};
var failures = 0;
foreach (var (name, actual, expected) in results)