Frequently Asked Questions

No. Each element is 4 bits, but each block of 32 also carries an 8-bit E8M0 scale, so the effective cost is 4.25 bits per element. Against bf16 that is a 3.76x reduction rather than 4x.

Any storage estimate that assumes a flat 4 bits will be about 6% optimistic.

FP4 usually means a bare 4-bit float, most often E2M1, with no defined block structure. MXFP4 is E2M1 plus the microscaling block structure: a fixed block size and one shared E8M0 scale per block.

The block scale is what makes the format usable. Bare E2M1 spans only about 3.6 binades of magnitude, which is nowhere near enough for a real weight tensor.

Neither. E8M0 is eight exponent bits and nothing else. The value is 2^(byte - 127), always a positive power of two.

This is why applying the scale is exact and cheap: it is an exponent adjustment, not a multiply that introduces its own rounding. The reserved encoding 0xFF means NaN, and per the specification a NaN scale makes the whole block NaN.

No. All sixteen 4-bit codes are finite values. There is no room to spend two codes on non-finite encodings at this width, so NaN is handled at the block level through the scale instead.

It sits at the knee of the overhead curve — 8/32 = 0.25 extra bits per element — and it is the block size wired into native block-scaled matrix units. Going larger saves almost nothing; going smaller usually costs you the hardware path.

Not to produce valid MXFP4. Weight block scales are derived from the weights themselves, so a round-to-nearest pass needs no data at all.

You need calibration if you want error compensation (GPTQ-style), a principled layer sensitivity ranking, or activation quantization. For weight-only work, calibration is a refinement rather than a requirement.

Usually not as a first step. Transformer activations have systematic outlier channels that interact badly with block scaling, and full activation quantization is a substantially harder problem than weight-only.

Weight-only MXFP4 captures most of the memory benefit for inference and carries far less risk. Add activation quantization only if you have a specific compute-bound reason to.

Not by naively applying weight quantization to a training loop. Gradients have different dynamic range characteristics and 4-bit gradient noise compounds in ways weight noise does not.

Low-precision training schemes that use microscaling formats generally keep master weights and optimizer state wider, apply the narrow format selectively, and involve careful loss scaling. That is a different design problem from quantizing a finished checkpoint.

Not straightforwardly. At the same nominal bit width, a well-tuned INT4 group-quantization scheme with GPTQ or AWQ frequently matches or beats naive MXFP4 on accuracy.

MXFP4's advantages are structural: an exact power-of-two scale, no zero-point, no asymmetric range handling, and — most importantly — a path to native hardware execution that INT4 group schemes with fp16 scales generally do not have.

Often not, even when the numerics are identical. The specification defines the encoding, not a file format. Nibble order, scale storage type, scale tensor layout, transpose convention and the exclusion list are all library choices.

Round-trip a known small tensor through any converter before trusting it.

Both, sometimes, and it depends entirely on the regime. Single-stream decode is bandwidth bound, so reading a quarter of the bytes helps even with a fully emulated compute path.

Batched prefill and training are compute bound, and there the speedup requires hardware with a native block-scaled matrix instruction. Without that, MXFP4 costs accuracy and buys nothing.

Profile it. Look for a distinct block-scaled matrix instruction in the kernel trace, and check whether arithmetic throughput scales with batch the way a native path would.

Do not infer it from a configuration flag. Many runtimes accept an MXFP4 dtype and silently dequantize to bf16 before the matmul.