Home/Format/Bit Layout

Format · Specification

Bit Layout

The E2M1 element encoding, the E8M0 block scale, and the packing conventions the specification leaves to you.

The E2M1 element

An MXFP4 element is four bits: one sign, two exponent, one mantissa. The exponent bias is 1, so a stored exponent of 0b01 means an unbiased exponent of zero. Normal values follow the usual IEEE-style form; the all-zero exponent encodes subnormals.

signexponent (E2)mantissa (M1)

Written out, the value of a normal element is (-1)^S × 2^(E-1) × (1 + M/2), and a subnormal (E = 0) is (-1)^S × 2^0 × (M/2), which gives ±0.5. There are no infinities and no NaN in E2M1 — all sixteen codes are finite numbers. That is a deliberate choice: at four bits, spending two codes on non-finite values would be expensive.

The E8M0 shared scale

Each block carries one scale byte in E8M0 format: eight exponent bits, no sign bit, no mantissa. The value is 2^(byte - 127), so the scale is always an exact power of two. The reserved code 0xFF encodes NaN, and per the microscaling specification a NaN scale poisons the entire block — every element decodes to NaN regardless of its own code.

Restricting the scale to a power of two is not an accident. It means dequantization is an exponent add rather than a multiply, which is cheap in hardware and, more importantly, exact: applying the scale introduces no rounding of its own. All the error in an MXFP4 tensor comes from the 4-bit element rounding, never from the scale.

Block structure and packing

The common MX configuration uses k = 32 elements per block, laid out along the reduction axis of the matmul so that a dot product consumes whole blocks. Storage per block is 32 × 4 bits of elements plus 8 bits of scale = 136 bits, or 4.25 bits per element. Against bf16 that is a 3.76× reduction, not 4× — a distinction worth keeping straight when you are budgeting memory.

# Two 4-bit codes share one byte. Layout is a convention, not a spec mandate -
# check what your kernel expects before you write a checkpoint.
def pack_nibbles(codes):
    lo = codes[0::2] & 0x0F
    hi = codes[1::2] & 0x0F
    return (hi << 4) | lo          # element 0 in the low nibble

# Storage cost per block of 32, in bits:
#   32 elements x 4 bits            = 128
#   1 shared E8M0 scale x 8 bits    =   8
#   ------------------------------------
#   total                           = 136  ->  4.25 bits/element

Reference encode and decode

The clearest way to understand the format is to read a slow implementation of it. This one is deliberately naive — an argmin against the magnitude table rather than bit manipulation — because the point is to be obviously correct, not fast.

# mxfp4_ref.py - readable reference, not a fast kernel.
import numpy as np

# E2M1: 1 sign bit, 2 exponent bits, 1 mantissa bit.
# Eight representable magnitudes, non-uniformly spaced.
E2M1 = np.array([0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0], dtype=np.float32)
EMAX_E2M1 = 2.0          # largest power-of-two exponent in the element format

def quantize_block(x):
    '''One MX block: 32 values -> 32 x 4-bit codes + one E8M0 scale.'''
    amax = np.max(np.abs(x))
    if amax == 0:
        return np.zeros(x.shape, np.uint8), np.uint8(127)

    # Shared scale is a pure power of two: E8M0, no sign, no mantissa.
    e = np.floor(np.log2(amax)) - EMAX_E2M1
    e = np.clip(e, -127, 127)
    scale = np.exp2(e)

    # Round-to-nearest against the 8-entry magnitude grid, then re-attach sign.
    y = np.abs(x) / scale
    idx = np.argmin(np.abs(y[:, None] - E2M1[None, :]), axis=1)
    codes = idx.astype(np.uint8) | ((x < 0).astype(np.uint8) << 3)
    return codes, np.uint8(e + 127)   # E8M0 stores exponent + bias

def dequantize_block(codes, scale_e8m0):
    '''Exact inverse: no rounding happens on the way back out.'''
    if scale_e8m0 == 255:
        return np.full(codes.shape, np.nan, np.float32)   # E8M0 NaN encoding
    scale = np.exp2(np.float32(scale_e8m0) - 127)
    mag  = E2M1[codes & 0x7]
    sign = np.where(codes & 0x8, -1.0, 1.0)
    return sign * mag * scale

# Round trip on a whole weight tensor, 32 elements per block.
def roundtrip(w, k=32):
    flat = w.reshape(-1, k)          # blocks run along the reduction axis
    out  = np.empty_like(flat)
    for i, blk in enumerate(flat):
        c, sc = quantize_block(blk)
        out[i] = dequantize_block(c, sc)
    return out.reshape(w.shape)

Two details in that code are worth calling out. First, the scale exponent is derived from the block maximum minus the largest exponent representable in E2M1, which is 2 (because the top magnitude 6.0 sits in the E = 11 binade). Second, dequantization has no rounding step at all: it is a table lookup and an exponent add.