Format · Ecosystem
Hardware Support
Storage, emulated compute and native compute are three different claims. Only one of them makes the arithmetic faster.
Three different things called support
"Supports MXFP4" is used to mean at least three distinct claims, and conflating them is the source of most disappointing benchmark results.
- Storage support — the runtime can hold weights in MXFP4 and unpack them. This gets you the memory saving and nothing else.
- Emulated compute — blocks are dequantized to bf16 or fp16 in registers or shared memory, then fed to an existing matmul path. Memory bandwidth improves; arithmetic throughput does not.
- Native compute — the matrix unit consumes 4-bit operands and applies block scales in hardware. This is where the throughput claims come from.
| tier | memory saved | compute gain | typical use |
|---|---|---|---|
| Storage only | ~3.8x | none | checkpoint distribution |
| Emulated compute | ~3.8x | ~1x | wide hardware compatibility |
| Native compute | ~3.8x | large | recent accelerators only |
For single-stream LLM decoding, the memory saving alone is often the dominant win, because that regime is bandwidth-bound rather than compute-bound. Emulated MXFP4 can therefore be genuinely faster end to end even with no arithmetic speedup at all. For prefill, batch inference and training, the distinction matters enormously.
Telling which path you are on
Do not infer the path from a config flag. Profile it. A native path shows a distinct matrix instruction in the kernel trace and near-flat arithmetic intensity as batch grows; an emulated path shows the same bf16 MMA instructions as an unquantized run, plus unpack work in the prologue.
The current landscape
Native block-scaled 4-bit matrix units have been announced or shipped by several vendors, generally as part of the same generation that introduced narrow-precision tensor cores. The pattern across vendors is broadly similar: a fixed block size wired into the hardware, an E8M0 scale operand, and accumulation into a wider type.
We do not maintain a per-SKU support matrix here. Vendor documentation changes, firmware and driver revisions gate features, and a matrix that is wrong is worse than no matrix. Check the vendor ISA documentation for your specific part and, more usefully, profile the kernel you actually intend to run.
What is stable enough to state: the block size baked into hardware is usually 32, matching the common MX configuration; scales are consumed as E8M0 bytes rather than floats; and accumulation is into fp32 or a comparable wide type. Software that assumes a different block size will fall off the native path even when the numerics are valid.
Found an error, or a result that disagrees? This is a community reference. Corrections with a reproducible test case are the most useful thing you can send us.