Guides

Four walkthroughs, written on the assumption that you already have a checkpoint and are about to find out whether 4 bits is enough for it.

Pages in this section

Quantizing Your First Modelguide · 30 min: /guides/quantizing-your-first-model/A weight-only MXFP4 pass end to end, including the layers you should leave alone.
Calibrationguide · 20 min: /guides/calibration/What calibration does for a block-scaled format, and what it cannot fix.
Choosing Block Sizeguide · 15 min: /guides/choosing-block-size/The arithmetic behind 32, and the case for 16 on outlier-heavy tensors.
Troubleshooting Quality Lossguide · 25 min: /guides/troubleshooting-quality-loss/An ordered decision tree for a model that came out worse than expected.

Assumed background

None of these guides assume a specific library, because library APIs churn faster than the numerics do.

You know what a linear layer's reduction axis is
You can run an evaluation harness and get a number you trust
You have read the bit layout page, or are willing to take 4.25 bits/element on faith
You understand that weight-only and activation quantization are different problems
You have a bf16 baseline measured on the same harness, same seed, same prompts

If a guide does not cover it

See also

Stuck on something these do not cover? Community threads are where the awkward cases end up — unusual architectures, odd reduction axes, and models that behave badly for reasons nobody has written down yet.

Community · FAQ