docs/cli.md covers how to invoke mlx convert / mlx calibrate. This doc covers what those commands actually do: the on-disk formats, the per-tensor decision engine, the config.json provenance contract, and the places where a conversion looks right for the wrong reason.

Audience: someone adding a quantization mode, or debugging a checkpoint that converted cleanly and loads wrong.

Every quantized tensor is a group of 1–3 arrays sharing a base key: {base}.weight, {base}.scales, {base}.biases. Nine different formats reuse that same shape. Which format a group actually is comes from config.json, not from the file layout — so three stages own three different things and confusing them is the main source of drift here.

Three things the loader materializes that are not bytes on disk:

1. .biases for symmetric ggml imports (Q4_0/Q8_0) — scales * -Z, crates/mlx-core/src/engine/persistence.rs:279
2. The sym8 [K,N] contiguous int8 kernel operand — crates/mlx-core/src/models/qwen3_5/quantized_linear.rs:440
3. The bf16 weight for fp8_e4m3 — reconstructed once at load, crates/mlx-core/src/quant/fp8_weight.rs:156

Mode strings round-trip through one pair of inverses: parse_mode_str (crates/mlx-core/src/models/quant_dispatch.rs:123) and mode_to_str (:153).

                       mlx convert -i <in> -o <out> [flags]
                                     │
                     flag validation │ packages/cli/src/commands/convert.ts:210-475
                     (runs for ALL   │ 8 guards live here and ONLY here
                      paths, before  │
                      any dispatch)  │
                                     ▼
                        inputPath.endsWith('.gguf') ?      convert.ts:480
                          │                          │
                         YES                        NO
                          │                          │
   ┌──────────────────────▼──────────┐               │
   │ convertGgufToSafetensors        │               │
   │   utils/gguf.rs:2477            │               │
   │   -m / --q-mtp / sym8 NOT read  │               │
   │   → ONE model.safetensors       │               │
   │     (no index.json)             │               │
   │   + optional 2nd pass for       │               │
   │     --mmproj → vision.safetensors│              │
   └─────────────────────────────────┘               │
                                                     │
                          modelType = -m ?? auto-detect     convert.ts:601-651
                                     │                       (try/catch SWALLOWS a bad config.json)
                          ┌──────────┴───────────┐
                          │                      │
              -m ∈ {pp-lcnet-ori, uvdoc}    everything else
                          │                      │
   ┌──────────────────────▼───────┐  ┌───────────▼──────────────────────────┐
   │ convertForeignWeights        │  │ convertModel → convert_model_inner   │
   │  utils/foreign_weights.rs:50 │  │  crates/mlx-core/src/convert.rs:1884 │
   │  Paddle / PyTorch pickle     │  │                                      │
   │  no quant, no sharding,      │  │  1. weight-file probe (first match)  │
   │  synthesized config.json     │  │  2. expand_symmetric_affine_biases   │
   └──────────────────────────────┘  │  3. recipe.sanitize()   (per family) │
                                     │  4. dtype cast                       │
                                     │  5. AWQ imatrix pre-scale            │
                                     │  6. quantize_weights_inner           │
                                     │  7. save_safetensors_sharded (5 GiB) │
                                     │  8. config.json + quantization block │
                                     └──────────────────────────────────────┘

Inside step 6, the per-tensor decision is composed of up to four layers:

  key ──▶ apply_mtp_quant_policy          OUTERMOST   convert.rs:2908-2913
             (intercepts every mtp.* key; inner predicate never consulted)
            │
            ▼
          apply_mxfp_upgrade  XOR  apply_nvfp4_upgrade
             (skipped entirely when a FIXED unsloth map was selected)
                                                      convert.rs:2888-2898
            │
            ▼
          recipe predicate                            convert.rs:2883-2887
             build_official_unsloth_recipe(kind)  OR  build_predicate_for_recipe(name)
             (absent ⇒ resolve_legacy_entry ladder, the only sym8-aware path)
            │
            ▼
          should_quantize(key, embed_quantizable)     convert.rs:3317
             (the shared floor — called INSIDE each recipe)

Execution (quantize_weights_inner, crates/mlx-core/src/convert.rs:6557): Skip → leave dense; Default → re-check should_quantize, then use the top-level triple; Custom → use it verbatim. A per-tensor override lands in config.json only when the resolved triple differs from the top-level one (record_quant_override_if_non_default, crates/mlx-core/src/convert.rs:5601, called from :6455, :6528, :6677).

Nine formats. Only five are selectable by --q-mode (VALID_QUANT_MODES, crates/mlx-core/src/convert.rs:1967). fp8_e4m3 is emitted only by the fixed Unsloth DGX map. The three ggml K-quants are consume-only — the bridge's mlx_quantize throws for them by name before MLX sees them (crates/mlx-sys/src/mlx_advanced_ops.cpp:892).

Shapes are for a dense [N, K] source weight; stacked experts add a leading [E, …] to every array.

Defaults are declared in exactly three consistent places plus MLX itself: packages/cli/src/commands/convert.ts:9 (display only — see gotchas), crates/mlx-core/src/convert.rs:1995 (SafeTensors), crates/mlx-core/src/utils/gguf.rs:2884 (GGUF), crates/mlx-sys/mlx/mlx/ops.cpp:4809 (quantization_params_from_mode). The K-quant pairs q6k(16,6) / q4k(32,4) / q5k(32,5) live in the bridge (default_group_size / default_bits, crates/mlx-sys/src/mlx_kquant.h).

decode:  w = scale * q + bias,   q ∈ [0, 2^bits - 1]  (unsigned)
         multiply(w, scales) then add(biases)     ops.cpp:5462
shapes:  wq.back()     = K * bits / 32            ops.cpp:5224
         scales.back() = K / group_size           ops.cpp:5226 (both cast back to w.dtype())
legal:   group_size ∈ {32, 64, 128},  bits ∈ {2,3,4,5,6,8}    ops.cpp:5172-5185

Affine is the only mode with no pinned (group_size, bits). The float modes are pinned in fp_quantize / fp_dequantize (crates/mlx-sys/mlx/mlx/ops.cpp:5242-5257, :5491-5506), the K-quants in the bridge's params_from_mode (crates/mlx-sys/src/mlx_kquant.cpp:90).

bpw = bits + 2·(scale_dtype_bits / group_size). On [4096, 4096] = 16,777,216 weights, bf16 companions:

--dtype float32 doubles the companion cost: 4/64 becomes 4 + 2·32/64 = 5.000 bpw.

Two-array modes. validate_mode_with_type requires uint8 scales and throws "Biases must be null for quantization mode" (crates/mlx-sys/mlx/mlx/ops.cpp:4871-4882).

w = scale * decode(code)                                   fp_quantized.h:139
element codec by BITS:    bits==4 → E2M1 4-bit             fp_quantized.h:51
                          bits==8 → E4M3 byte
scale   codec by GROUP_SIZE, not by mode:                  fp_quantized.h:30
                          group_size==16 → E4M3 scale
                          otherwise      → E8M0 scale
  ⇒  mxfp4 = E2M1 × E8M0(gs 32)
     mxfp8 = E4M3 × E8M0(gs 32)
     nvfp4 = E2M1 × E4M3(gs 16)

Output dtypes {uint32, uint8} are hard-coded in fp_quantize (crates/mlx-sys/mlx/mlx/ops.cpp:5340). bpw:

Checked on [4096,4096]: mxfp4 = 8,388,608 + 524,288 = 8,912,896 B ⇒ 4.25 ✓; nvfp4 = 8,388,608 + 1,048,576 = 9,437,184 ⇒ 4.50 ✓.

Convert does not use MLX's encoder for mxfp4 or mxfp8. The layout above is the format contract — codec, group size, sidecar shapes, dtypes — and it is what convert writes and what every loader reads. The block exponent inside it is chosen in-tree, on every convert, with no flag: crates/mlx-core/src/quant/mxfp4_weight.rs evaluates both candidate E8M0 exponents per block and keeps the lower squared error, and crates/mlx-core/src/quant/mxfp8_weight.rs takes the ceiling. Since upstream 02adf7b21 MLX also rounds the E8M0 scale up, for both mxfp4 and mxfp8 (mx_scale_round_up in crates/mlx-sys/mlx/mlx/backend/metal/kernels/fp8.h; the same rule on CPU). So mxfp8 scale bytes from mlx convert equal mlx_quantize's (in a block whose scale byte is 0, MLX writes a negative-zero code for a negative element; convert writes positive zero), while mxfp4 differs wherever the search picks the lower exponent. Both load in mlx-lm and dequantize identically. MLX's rule survives as a #[cfg(test)] reference (quantize_mxfp{4,8}_mlx_round_up), pinned byte for byte against mlx_quantize by mxfp{4,8}_round_up_reference_is_bit_identical_to_mlx_quantize. nvfp4 keeps MLX's encoder, and adds a pre-quantization power-of-two lift on dense FFNs (see docs/cli.md, "Data-free encoder tuning").

The MLX nvfp4 quantizer writes no global scale. The DGX port does not carry Unsloth's calibrated global scales. The nemotron_h modelopt ingest is the one exception — it emits a .global_scale sidecar rather than folding weight_scale_2 into the per-group E4M3 scales; see the Nemotron-H NVFP4 section below.

Not a --q-mode. Explicitly distinct from MLX mxfp8 (crates/mlx-core/src/quant/fp8_weight.rs:1).

encode   dequant_scale = clip( max(|w|, axis=-1, keepdims) / 448 , min = f32::MIN_POSITIVE )
                                                       fp8_weight.rs:79   ← the floor matters:
                                                       an all-zero row gets 1.175e-38, not 0
         encoded = to_fp8( clip( w / dequant_scale, ±448 ) )       → Uint8 [..., N, K]
         scales  = dequant_scale.astype(BFloat16)                  → BF16  [..., N, 1]
decode   w ≈ from_fp8(q) * scale, in bf16, ONCE at load    fp8_weight.rs:156-187
forward  plain x.matmul(weight.T)             qwen3_5/quantized_linear.rs:763

group_size is the sentinel -1 (crates/mlx-core/src/quant/fp8_weight.rs:19), serialized as JSON null (crates/mlx-core/src/convert.rs:5589). FP8_E4M3_MAX = 448.0. Load-time rejects include any decoded magnitude > 448, because MLX's fast decoder maps the reserved 0x7f/0xff E4M3 patterns to ±480 while to_fp8 saturates at ±448 (crates/mlx-core/src/quant/fp8_weight.rs:170).

Emission gate — --q-recipe unsloth --q-mode nvfp4 alone is not enough. select_official_unsloth_recipe (crates/mlx-core/src/convert.rs:4416) returns Some(Nvfp4) only when recipe == "unsloth" and is_qwen35_hybrid and !quant_mxfp and quant_mode == "nvfp4". On any non-Qwen-hybrid input the same CLI line returns None and validate_unsloth_imatrix_after_selection (:4396) either hard-errors or falls through to the legacy affine predicate — zero fp8_e4m3 tensors.

The only mode with an unpacked weight and a 1-D scale.

encode   s[n] = max( max_k |w[n,k]| , 1e-12 ) / 127
         q    = clip(round(w/s), ±127) as int8              mlx_na_int8.cpp:1028
decode   w[n,k] ≈ scales[n] * q[n,k]
forward  M ≤ 2 → int8_w8a16_qmv ;  M ≥ 3 → int8_w8a8_matmul
         never touches mlx_quantized_matmul (no sym8 pack)  quantized_linear.rs:662
bpw      8 + 32/K  →  8.0078 @ K=4096

No group_size at all. config.json records top-level "group_size": null (crates/mlx-core/src/convert.rs:3127); the in-memory sentinel is -1 (crates/mlx-core/src/models/quant_dispatch.rs:25). parse_group_size accepts null only for sym8/fp8_e4m3 and rejects an integer for them (:621-648). The (8, 64) pair in the defaults tables is the affine-fallback group for the layers sym8 declines, not sym8's own group (crates/mlx-core/src/convert.rs:2000).

Eligibility (sym8_eligible, crates/mlx-core/src/convert.rs:5546): 2-D [N,K] and K % 16 == 0. Deliberately excludes the GPU-generation check, which is a runtime property — see gotchas.

Fallback ladder inside resolve_legacy_entry (crates/mlx-core/src/convert.rs:6233), in order:

enforce_sym8_group_coherence (crates/mlx-core/src/convert.rs:6002) then re-applies the emission gates to every co-quantized group (five tables at :5963) and forces the whole group dense if any member would not emit, because the strict loaders are all-or-none. It hard-errors instead of force-densing when any member is already sidecarred on disk (:6185).

sym8_supported() is true for Qwen35Recipe (both dense and MoE, crates/mlx-core/src/convert.rs:1170), Lfm2Recipe (:1513), Gemma4Recipe (:1736).

On-disk contract at crates/mlx-core/src/utils/gguf_kquant.rs:9, shape formulas at :113, for a row of K values with K % 256 == 0.

.biases on a K-quant holds ggml's d (and dmin) — a SCALE, not an additive bias. The name is reused only so the ~25 .scales/.biases "is quantized" sentinel sites keep working (crates/mlx-core/src/utils/gguf_kquant.rs:22).

Decode (KQScales::at, identical in Metal crates/mlx-sys/src/metal/kquant/kquant.h:678 and CPU crates/mlx-sys/src/mlx_kquant.cpp:513). For these three modes there are two branches, selected by has_min, which is true for both q4k and q5k (has_sub_min, crates/mlx-sys/src/mlx_kquant.h:44). The bridge also reads q3k and the IQ modes; their parameters are in the same header.

has_min == true    (q4k AND q5k)          super_ratio = 8
    scale =   d[2*(g >> 3)]     * sc[2*g]
    bias  = -( d[2*(g >> 3) + 1] * sc[2*g + 1] )
    q5k differs from q4k only in the fifth bit plane.

has_min == false   (q6k)                  super_ratio = 16
    scale = d[g >> 4] * (int8) sc[g]
    bias  = -32.0f * scale                 // folds ggml's (q - 32)

Why this reuses MLX's affine kernel algebra:

1. A K-quant sub-block is algebraically affine — value = scale*q + bias, both constant inside a group (kquant.h:5). get_pack_factor, load_vector, qdot, qouter, dequantize are byte copies of the affine ones and take (scale, bias) by value — they never see a K-quant.
2. The importer, not the kernel, absorbs ggml's swizzle: q6k_code/q4k_code/q5k_code de-interleave ggml's ql/qh/nibble planes (gguf_kquant.rs:257) and BitPacker re-emits a plain LSB-first n-bit stream — bit j of code i at absolute bit i*bits + j (:184).
3. Sub-scales are stored unpacked rather than in ggml's 6-bit fields, which "keeps the affine kernel's per-group pointer walk intact" (gguf_kquant.rs:432) and costs exactly +0.125 bpw.
4. Every row is a whole number of 256-value super-blocks, so a flat group index stays aligned as the cursor runs off one row into the next (kquant.h:657).

It is a copied kernel family, not the same binary: crates/mlx-sys/src/metal/kquant/kquant.h is mlx-node's own source, prebuilt into paged_attn.metallib (kquant.metal) and dispatched by mlx_kquant_metal.cpp, with two extra template params (super_ratio, has_min). Its QuantizedBlockLoader generalises the affine loader's static_assert(BCOLS <= group_size) (crates/mlx-sys/mlx/mlx/backend/metal/kernels/quantized.h:582) into group_steps/scale_step because q6k's group of 16 is narrower than the BK=32 tile (kquant.h:730).

bpw = bits + 8/scales_per_value + 16·per_group/256:

Exactness vs llama.cpp: decode is float32 on both levels, so q4k and q5k are bitwise identical. Q6_K is identical up to the sign of zero — ggml subtracts 32 in integer arithmetic, the contract folds it into bias = -32*scale, and IEEE-754 gives x + (-x) = +0.0 where ggml writes -0.0 (crates/mlx-sys/src/mlx_kquant.cpp:525). A second divergence exists for a non-finite d (whole super-block NaN here vs signed infinities in ggml) but no real GGUF holds one.

Consume-only is enforced by name: the bridge's mlx_quantize throws "can be read but not produced" for every K-quant mode (crates/mlx-sys/src/mlx_advanced_ops.cpp:892); dequantize routes to the bridge's KQuantDequantize primitive (crates/mlx-sys/src/mlx_kquant.cpp:362); there is no K-quant quantize kernel.

ggml     Q4_0:  w = d*(q-8)          Q8_0: w = d*(q-128)   (after the importer's sign-bit flip)
MLX      w = scale*q + bias   ⇒  bias is the CONSTANT -Z*scale, so it need not be stored.

CONVERTER writes  .weight u32, .scales f16, NO .biases,
                  config.json symmetric_zero_point = 8 (Q4_0) / 128 (Q8_0)   gguf.rs:891
LOADER   rebuilds biases = scales.mul_scalar(-Z)                    persistence.rs:279

Bitwise exactness rests on two independent facts:

• mlx_array_mul_scalar builds the scalar in the array's own dtype when the array is floating (crates/mlx-sys/src/mlx_array_ops.cpp:357), so an f16 .scales yields an f16 .biases with no f32 promotion and no double-rounding.
• Z ∈ {8 = 2³, 128 = 2⁷} is a power of two, so -Z*s only shifts the exponent field. The mantissa is untouched and the product is exactly representable in f16 for every finite non-overflowing s, including subnormals (2^-24 · 8 = 2^-21).

The historical writer is kept in-tree as derived_symmetric_bias_bits (crates/mlx-core/src/utils/gguf.rs:748) purely as a test oracle; symmetric_bias_parity.rs asserts raw IEEE-754 bit equality with no tolerance over an adversarial scale set.

bpw at K=4096 rows:

Before the change, Q4_0 landed at 5.0 and Q8_0 at 9.0 — larger than the GGUF they came from. 0.5 bpw = 0.0625 bytes/weight; the 681 MB saving on Gemma-4-12B-QAT (crates/mlx-core/src/engine/persistence.rs:206) therefore implies ≈ 1.09 × 10¹⁰ quantized weights, i.e. essentially every matrix in the model.

Legal zero points are exactly 1 << (bits-1) (crates/mlx-core/src/models/quant_dispatch.rs:700). Any other value is rejected, not clamped — it would rebuild every bias at the wrong offset while the shapes still match.

Related but distinct: repack_symmetric_to_mlx_affine (crates/mlx-core/src/utils/gemma_quant_repack.rs:17) maps the same algebra for Google gemma-QAT weights but does write a real .biases array (every entry -2^(bits-1)·s) and involves no symmetric_zero_point. It is a parallel implementation — changing one does not change the other.

Because all nine formats share the triplet shape, dtypes are the only discriminator. Every guard runs before dispatching on plq.mode.

ensure_affine_biases_present is live in four families: gemma4 (6 non-test call sites — crates/mlx-core/src/models/gemma4/persistence.rs:1361, :1390, :1586, :1850, :2140, :2205), qwen3_5, qwen3_5_moe, lfm2. Without it the failure surfaces as MLX C++'s anonymous "Biases must be provided for affine quantization".

The converter mirrors these before writing: validate_existing_quantized_entry / validate_existing_kquant_entry refuse to relabel already-packed bytes (crates/mlx-core/src/convert.rs:5620). Dtype-cast preservation is content-keyed, not suffix-keyed: kquant_biases_to_preserve freezes an f16 .biases only when its sibling .scales is int8/u8 (:5455), and sym8_scales_cast_action normalizes an f16/bf16 [N] scale next to an int8 weight up to f32 (:5485).

enum QuantDecision { Skip, Default, Custom { bits: i32, group_size: i32, mode: String } }
// crates/mlx-core/src/convert.rs:3909

There are six predicate-builder functions; build_predicate_for_recipe (crates/mlx-core/src/convert.rs:5207) is a dispatcher, not a builder:

Seven --q-recipe names are accepted (crates/mlx-core/src/convert.rs:2122); nine dispatch paths exist once the fixed map and privacy-filter are counted; resolve_legacy_entry (:6233) is the tenth path and the only sym8-aware one.

B = --q-bits, G = --q-group-size. Under --q-mode nvfp4 the recipe builder is handed recipe_gs = 64 instead of the top-level 16, because affine only accepts {32,64,128} (crates/mlx-core/src/convert.rs:2878).

* unless embed_quantizable (lfm2/lfm2_moe) — but see the note below: no recipe ever receives it.

The last row is the sharpest structural split: mixed_* / qwen3_5 / unsloth-legacy are allow-by-default; build_official_unsloth_recipe and build_nvidia_recipe are deny-by-default (QuantDecision::Skip fall-through at crates/mlx-core/src/convert.rs:4573 and :4716).

Bits arithmetic for unsloth (snap_bits maps 7 → 8, crates/mlx-core/src/convert.rs:4176):

The d=3 column reproduces the documented ladder (packages/cli/src/commands/convert.ts:87). d=3 is a TS-only injection at packages/cli/src/commands/convert.ts:383 — see gotchas.

Returns false for (crates/mlx-core/src/convert.rs:3317-3396):

Recipes that quantize lm_head / embed_tokens branch before calling it: mixed_* (:3988), unsloth (:4198, :4205), fixed-Unsloth (:4534), nvidia (:4648). qwen3_5 has no such branch, so --q-recipe qwen3_5 always leaves both bf16.

Every recipe passes embed_quantizable = false hard-coded — :3997, :4058, :4213, :4538, :4658. The real flag is read only on the no-recipe ladder (:6246) and in quantize_weights_inner's Default arm (:6561). So lfm2's packed-embedding support is unreachable through any recipe except build_unsloth_recipe's explicit pre-should_quantize branch at :4198 — which fires for every family, not just lfm2.

(a) Fixed-map selection. When select_official_unsloth_recipe (crates/mlx-core/src/convert.rs:4416) returns Some(kind), no upgrade wrapper runs at all (:2889/:2891 both require official_unsloth_kind.is_none()), and --q-bits/--q-group-size have zero effect — build_official_unsloth_recipe takes no bits arguments. is_qwen35_hybrid requires three independent agreements (:4329): the config's own model_type collapses to a qwen3_5 family, the caller's --model-type collapses to the SAME family, and the sanitized keys carry all four of self_attn.q_proj + linear_attn.in_proj_qkv + in_proj_z + out_proj (:4308).

The map's only depth rule: final_eight_start = num_layers.saturating_sub(8), with num_layers computed after filtering MTP / vision_tower / visual. / vision_embedder keys (:4476) — the only builder that does this filtering.

(b) Upgrade wrapper. Branch order is load-bearing (crates/mlx-core/src/convert.rs:4815-4887):

original = inner(key)
 1. is_affine_only_key(key)                        → return original UNCHANGED     :4820
    (lm_head | router.proj | embed_tokens* | embedding_projection)
 2. is_bitexact_affine_proj(key) AND original == Custom{8, *, "affine"}
                                                    → return original UNCHANGED     :4836
 3. is_router_gate(key)  → Skip stays Skip; anything else → Custom{8, 64, affine}   :4849
 4. match original: Skip              → Skip
                    Default + default_bits==8 → mxfp8 8/32
                    Default + default_bits==4 → mxfp4 4/32
                    Default + ANYTHING ELSE   → Default   (SILENT pass-through)     :4863
                    Custom{bits:8,..}         → mxfp8 8/32
                    Custom{bits:4,..}         → mxfp4 4/32
                    other                     → unchanged

apply_nvfp4_upgrade (crates/mlx-core/src/convert.rs:5139) is not the same shape:

Trap A — is_bitexact_affine_proj must be decision-scoped. It is (crates/mlx-core/src/convert.rs:4836):

if is_bitexact_affine_proj(key)
    && matches!(&original, QuantDecision::Custom { bits: 8, mode, .. } if mode == "affine")
{ return original; }

Key-only would break mixed_* + --q-mxfp: mixed_4_6 assigns o_proj / out_proj / in_proj_a / in_proj_b its low bits — Custom{4, gs, affine} (:4026), not 8. A key-only guard returns that untouched, so --q-mxfp becomes a silent partial no-op on exactly four tensor classes. Regression test: apply_mxfp_upgrade_promotes_non_pinned_low_bit_projections (:8394), paired with apply_mxfp_upgrade_preserves_bitexact_affine_projections (:8312).

It must also not be folded into is_affine_only_key (:3886), because that helper is also consulted by the no-recipe ladder (:6311), where listing o_proj would stop a uniform --q-mode mxfp8 from upgrading it.

Trap B — sym8_eligible must not be asked about a packed tensor. It reads the ARRAY: ndim == 2 && K % 16 == 0 (:5546). A packed affine weight is Uint32 [N, K/8] — for [8, 128] that is 2-D with 128 % 16 == 0, so it answers "eligible" while describing the packing. resolve_legacy_entry therefore takes a for_existing flag and swaps the question for a dtype witness (:6345):

let stays_sym8 = match weights.get(key) {
    Some(array) if for_existing => array.dtype()? == DType::Int8,   // stored-format witness
    Some(array) => sym8_eligible(array)?,                            // fresh float weight
    None => false,
};

Named mutation at crates/mlx-core/src/convert.rs:10310.

CLI accepts off | cyankiwi | all | split | drafter (packages/cli/src/commands/convert.ts:265); Rust normalizes drafter → split at crates/mlx-core/src/convert.rs:1960. Non-off, non-split requires --quantize and --q-recipe (:1987).

mtp_quant_decision(key)                                   convert.rs:3790
  policy == "off"        → None            (delegate entirely)
  !is_mtp_key(key)       → None            (delegate)
  key lacks ".weight"    → Some(Skip)
  prefix = normalize_mtp_prefix(key minus ".weight")
    is_mtp_layer_quantizable_prefix(prefix)       → Custom{4, 32, "affine"}
    OR (policy == "all" AND prefix == "mtp.fc")   → Custom{4, 32, "affine"}
    else                                          → Some(Skip)

MTP_QUANT_BITS = 4, MTP_QUANT_GROUP_SIZE = 32 are hard-coded (crates/mlx-core/src/convert.rs:3303). Because apply_mtp_quant_policy is the outermost wrapper, MTP linears are never touched by either upgrade wrapper — 4/32 affine even under --q-mxfp. The GGUF entry point never wires the policy at all (crates/mlx-core/src/utils/gguf.rs:2974).

// crates/mlx-core/src/convert.rs:3966
let num_layers     = infer_num_layers(weight_keys);   // max layers.N index + 1, NO filtering
let first_boundary = num_layers / 8;
let last_boundary  = num_layers - num_layers / 8;
use_more_bits[i] = i < first_boundary || i >= last_boundary || (i % 3 == 0);

# mlx-lm/mlx_lm/convert.py:61
use_more_bits = (index < num_layers // 8
                 or index >= 7 * num_layers // 8
                 or (index - num_layers // 8) % 3 == 2)

num_layers = 48:

Same count, different layers — the middle-band phase is offset by 2. num_layers = 36: mlx-node's last boundary is 32, mlx-lm's is 7*36//8 = 31, so layer 31 is high-bit there and low-bit here.

Three further deltas: mlx-lm also gives high bits to v_a_proj/v_b_proj which key.contains("v_proj") does not match (:4016); mlx-node adds a router-gate 8/64 pin mlx-lm has no equivalent of (:4002); and infer_num_layers (:3932) does no vision/MTP filtering, so on a VLM whose vision tower is deeper than the LM every boundary is wrong. The doc comment at :3945 calls this "Logic (from mlx-lm)" — it is a paraphrase, not a port.

Reachable only through #[napi] convert_gguf_to_safetensors (crates/mlx-core/src/utils/gguf.rs:2477), plus the K-quant repacker (crates/mlx-core/src/utils/gguf_kquant.rs).

1. HEADER, no payload read
   parse_gguf  gguf.rs:459   magic 0x46554747, version >= 3 (no upper bound)
     alignment = general.alignment ?? 32, .max(1)                 :504
     tensor infos — UNKNOWN TYPE ⇒ HARD REJECT, whole file        :530
     data_offset = align(stream_pos)                              :550

2. HEADER-ONLY GUARDS  (all run before the destructive File::create)
     :2591  --gguf-kquant + requantize     (only if the file HOLDS K-quants)
     :2633  --quantize over Q4_0/Q8_0
     :2678  secondary output carries symmetric or K-quant tensors
     :2733  gemma4 K=V detected + synthesized config + head_count_kv can't state both

3. PAYLOAD   load_gguf_tensors  :1058
     k_quant_format().is_some()  ─┬─ import_k_quants → load_kquant_repack   :970 (4 MiB chunks)
                                  ├─ off + Q6_K + arch=="gemma4"
                                  │    + name=="token_embd.weight"
                                  │        → load_q6k_tensor_bf16 (DEQUANT) :633
                                  └─ else → reject                          :1101
     is_mlx_affine_quantized()   → load_quantized_tensor  :759  (WHOLE tensor into RAM)
     otherwise                   → load_unquantized_tensor :578

4. POST, in this exact order
     remap_keys :1365 (collision = HARD ERROR) → fixup_gemma4_mmproj_layout :1408
     → fixup_shapes :1477 → fixup_qwen35_linear_attn :1565 → dtype cast :2804
     → AWQ imatrix :2842 → optional re-quantize :2941 → vlm_key_prefix :3061

5. WRITE
     save_safetensors :3096   ONE file, no index.json   ◀ DESTRUCTIVE from here
     if is_primary_model (filename == "model.safetensors"):
        config.json  (authoritative or extract_config :1753)
        gemma4 attention_k_eq_v / layer_types :3133
        quantization block :3155 / preserved_source_quantization :3166
        12-file runtime asset allowlist :3193

Guards 2 are hoisted deliberately: File::create truncates an existing checkpoint the instant it runs (crates/mlx-core/src/utils/gguf.rs:2651). This is not universal — three refusals still run after the save: --config-dir without a config.json (:3109), the source-quant profile collision inside preserved_source_quantization (only called from :3166/:3182), and the asset-copy / config-write ? at :3215 / :3178.

Nine types are recognized (GgufTensorType, crates/mlx-core/src/utils/gguf.rs:42). Anything else is a hard error at header parse — the whole file is refused even if one tensor uses an unlisted type.

block_size() (:99), k_quant_format() (:127) and SourceQuantProfile::for_gguf_type (:1902) are all _-free exhaustive matches, so adding a type forces every routing site to be updated.

Memory behaviour differs per route: K-quant streams in 4 MiB chunks (:903); the affine repack reads the whole tensor into a Vec<u8> at once (:769); the Q6_K→BF16 fallback streams one 210-byte block but allocates the destination in full.

Dispatch is by metadata only (gguf_name_to_hf_for_metadata, crates/mlx-core/src/utils/gguf.rs:1252):

Where the two LLM maps disagree — the whole reason the gemma4 map exists:

Under the generic map ffn_norm and post_attention_norm both land on post_attention_layernorm — a silent one-of-two loss when collected into a HashMap. remap_keys (:1365) now makes that a hard error ("GGUF key remap collision").

Generic-only Qwen3.5 GDN rules the gemma4 map does not carry (:1328) — note every destination carries the .linear_attn. segment, which is load-bearing because fixup_qwen35_linear_attn triggers on k.contains("linear_attn.") (:1570):

.attn_qkv.   → .linear_attn.in_proj_qkv.      .ssm_conv1d. → .linear_attn.conv1d.
.attn_gate.  → .linear_attn.in_proj_z.        .ssm_norm.   → .linear_attn.norm.
.ssm_beta.   → .linear_attn.in_proj_b.        .ssm_dt.bias → .dt_bias
.ssm_alpha.  → .linear_attn.in_proj_a.        .ssm_a       → .A_log
.ssm_out.    → .linear_attn.out_proj.

Global-tensor quant-group rename. Per-layer rules are infix replace(), so they ride along on any suffix. The three global tensors are whole-string matches and need rename_global_quant_group (:1167), which requires the remainder to be exactly one of QUANT_GROUP_SUFFIXES = [".weight", ".scales", ".biases"] (:1154):

Moving only .weight was the original bug: loaders probe embed_tokens.scales as the "this tensor is quantized" sentinel, so a stranded sidecar silently degrades the group to a bare packed weight (:1147). Pinned by global_renames_do_not_over_generalize (:5940).

Exactly one tensor, and only for gemma4:

// crates/mlx-core/src/utils/gguf.rs:1179  (gemma4_name_to_hf)
if name == "rope_freqs.weight" { return None; }   // RoPE freqs are derived from config at runtime

The generic map has no such rule: gguf_name_to_hf("rope_freqs.weight") matches nothing and returns the string unchanged (:1299), so a non-gemma4 GGUF carries the precomputed RoPE table into model.safetensors under its raw GGUF name.

Is the dropped data recoverable?

So for gemma4 with a synthesized config, parse_rope_parameters(None) returns the hard-coded triple (1_000_000.0, 10_000.0, 0.25) (crates/mlx-core/src/models/gemma4/persistence.rs:446), and the flat rope_theta in the emitted config.json is dead. partial_rotary_factor has no GGUF source at all.

asset_dir  = config_source_dir  OR  input_path.parent()  OR  "."      gguf.rs:2711
src_config = asset_dir/config.json
synthesized_config = !src_config.exists()

--config-dir given, no config.json inside  → HARD ERROR   :3109  (AFTER the save)
--config-dir absent, no config.json beside → SILENT fall-through to extract_config  :3117

extract_config (:1753) can produce at most 12 flat keys: _name_or_path, model_type (the GGUF arch string verbatim — qwen3, never qwen3_5), hidden_size, num_hidden_layers, intermediate_size, num_attention_heads, num_key_value_heads (scalar only), max_position_embeddings, rope_theta, rms_norm_eps, vocab_size, head_dim.

It cannot produce: vision_config, audio_config, rope_parameters, partial_rotary_factor, tie_word_embeddings, layer_types, sliding_window, PLE fields, architectures, or any tokenizer artifact. Its value match at :1790 has a _ => {} arm, so any array-valued metadata is silently dropped — which is exactly why gemma4's array-spelled head_count_kv needs apply_gemma4_attention_geometry (:2257), and why that helper runs only when synthesized_config (:3150).

The runtime-asset allowlist is exactly 12 filenames (:3193): tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, special_tokens_map.json, added_tokens.json, chat_template.jinja, generation_config.json, preprocessor_config.json, video_preprocessor_config.json, processor_config.json, viterbi_calibration.json. Copy semantics are asymmetric (:3207): with --config-dir a failed fs::copy is a hard error; without it the same failure is only a warn!.

Gemma4's global layers reuse the key projection as V, so llama.cpp never writes blk.N.attn_v. No metadata key records this — the tensor list is the only witness (gemma4_layer_types_from_missing_v, crates/mlx-core/src/utils/gguf.rs:2064):

collect blk.N indices                                              :2065
missing_v = layers with no blk.N.attn_v*                           :2075
  ├─ empty OR == all layers        → Ok(None)   NOT an error       :2084  ◀ load-bearing guard
  ├─ blocks not 0..N contiguous, or gemma4.block_count disagrees → Err     :2104
  ├─ head_count_kv NOT spelled as an array → Ok(Some(types))       :2130  (silence ≠ contradiction)
  ├─ array len != layer count → Err                                :2135
  ├─ argmin(head_count_kv) set != missing_v set → Err              :2158
  └─ else → Ok(Some(types))

The all-present/all-absent guard matters: a truncated download is also missing attn_v, and reading that as K=V suppresses the loader's own missing-weight check and feeds attention the keys as values (:2031). An absent header is tolerated (:2048) only because the caller closes the hole it creates — the :2733 refusal. Results are written non-destructively (:3133): attention_k_eq_v and layer_types are set only if the config does not already state them, probed via gemma4_config_states (:2188) which checks text_config first then top level.

Qwen3.5 hybrid detection (is_qwen35_hybrid_gguf, :1738) requires two witnesses: architecture ∈ {qwen35, qwen35moe, qwen3} and has_qwen35_hybrid_weight_shape (crates/mlx-core/src/convert.rs:4308). Ordinary Qwen3 fails the shape check.

fixup_qwen35_linear_attn (:1565) is architecture-independent — it triggers purely on the presence of any linear_attn. key. Geometry comes from metadata with hard-coded fallbacks: {arch}.ssm.state_size defaults to 128, {arch}.ssm.inner_size to 4096 (:1584), qk_dim to 4096 (:1598). If n_value_heads < 2 the whole fixup silently returns Ok(()) (:1613). A_log also becomes log(-x) because GGUF stores -exp(A_log) (:1626).

GgufConversionOptions.dtype accepts float32|f32, float16|f16, bfloat16|bf16 (crates/mlx-core/src/utils/gguf.rs:3363). The napi doc says "default: keep original" (:2904) — but the CLI never passes None: const dtype = args.dtype || 'bfloat16' (packages/cli/src/commands/convert.ts:625, the GGUF branch).

The cast loop (:3376) skips a key if any of four tests hit:

preserve_dtype_keys (:3319) is built from the source tensor list mapped through the same gguf_name_to_hf_for_metadata that remap_keys uses, then expanded by k_quant_output_keys (:947). Tests 2-4 only happen to cover those keys today, which is why the producer and the do-not-cast set are named through one helper and cannot drift (:3307).

model.safetensors             exists? → single lazy load            convert.rs:2361
weights.safetensors           exists? → single lazy load                     :2380
model.safetensors.index.json  exists? → sharded                              :2394
else → Err "No model weights found"                                          :2441

The index is used only to derive the set of shard filenames (let shard_files: HashSet<String> = index.weight_map.values().cloned().collect();, :2403), and shards merge with all_tensors.extend(...) (:2427). Tensor names in weight_map are never cross-checked in either direction.

Output sharding is fixed and does not depend on the input layout (crates/mlx-core/src/utils/safetensors.rs:862):

MAX_SHARD_SIZE = 5 << 30 = 5 × 1,073,741,824 = 5,368,709,120 bytes = 5 GiB   safetensors.rs:330
1. sort tensor names lexicographically                (deterministic; mlx-lm uses insertion order)
2. byte_size = array.size() * dtype.byte_size()
3. greedy bin-pack, close when current + next > 5 GiB, guarded by !current_shard.is_empty()
   (that guard prevents mlx-lm's empty leading shard when tensor 0 exceeds the budget)
naming: 1 shard → model.safetensors ; N>1 → model-{i+1:05}-of-{N:05}.safetensors
model.safetensors.index.json is written ALWAYS, even for one shard        :967

config.json ──▶ TS auto-detect ──▶ modelType: string | undefined
                convert.ts:602-651        │
                                          ├─ pp-lcnet-ori | uvdoc → convert_foreign_weights()
                                          │
                                          └─ convertModel({modelType, …})
                                                 recipe_for(model_type)     convert.rs:2783
                                                   Some(recipe)  → recipe.sanitize(...)
                                                   None + Some(mt) → Err "Unknown model type"  :4007
                                                   None (mt ABSENT) → tensors pass through
                                                                       UNSANITIZED, no error   :4012

TS auto-detect arms (packages/cli/src/commands/convert.ts:602-651): paddleocr_vl → paddleocr-vl; qwen3_5_moe/qwen3_5 passthrough; gemma4/gemma4_text → gemma4; gemma4_unified kept raw; architectures[] containing Gemma4UnifiedForConditionalGeneration → gemma4_unified; lfm2_moe/lfm2 passthrough; openai_privacy_filter → privacy-filter. There is no arm for qianfan-ocr (its HF model_type is internvl_chat, crates/mlx-core/src/models/qianfan_ocr/config.rs:109), pp-lcnet-ori, or uvdoc.

gemma4_unified is deliberately not collapsed: the E2B prequantized importer gates on the exact string "gemma4" (crates/mlx-core/src/convert.rs:2266), so collapsing would route an audio-carrying unified QAT into an importer that drops audio.

Native registry: 12 model_type strings → 9 recipe impls (recipe_for, crates/mlx-core/src/convert.rs:2783).

Five behaviour flags on the trait (crates/mlx-core/src/convert.rs:153), all defaulting to the conservative value:

owns_dtype_cast = true bypasses the generic dtype loop entirely (:3878), so the hard Err("Unsupported target dtype") at :3966 is unreachable for qwen3_5 / qwen3_5_moe / lfm2 / lfm2_moe / nemotron_h. qwen3_5 and lfm2 still warn! and default to bfloat16 in their own arms (:686, :1360); nemotron_h does neither — its sanitize takes _target_dtype_str and ignores it outright (:2731), because the ingest reproduces the source's NVFP4/FP8 layout rather than casting to a requested float dtype. --dtype is therefore a no-op on that family, silently.

fn model_types() (:161) is #[allow(dead_code)] — it exists only for the registry-consistency test.

qwen3_5 / qwen3_5_moe (crates/mlx-core/src/convert.rs:486-1165). Two preflights on the untouched source map first (:499): qwen_vision_quantization_preflight (:218) — the vision runtime is dense-only, so only one uniform packed mode is allowed — and reject_prequantized_qwen_individual_experts (:442). Then five steps: key remap (:542) → dequantize pre-quantized vision groups (:616) → FP8 E4M3 dequant (:695) → expert stacking (:846) → mlx-vlm sanitize (:1008: 5-D patch_embed.proj.weight transpose, 3-D conv1d.weight transpose when dim2 <= 16, RMSNorm +1.0 on five suffixes).

lfm2 / lfm2_moe (:1196-1503). Drop tied lm_head; *.conv.conv.weight 3-D transpose when shape[2] > shape[1]; w1/w2/w3 → gate_proj/down_proj/up_proj renaming all three suffixes; reject any feed_forward.experts.* sidecar (:1306); stack experts (:1323); float-only cast excluding .expert_bias (:1396) — the repo's analogue of mlx-lm's cast_predicate (mlx-lm/mlx_lm/models/lfm2_moe.py:389 excludes exactly expert_bias); final invariant pass (:1454).

gemma4 / gemma4_unified (:1596-1734). Longest-first prefix strip (:1613); drop rotary_emb; drop .input_max/.input_min/.output_max/.output_min only for language keys (:1628) since mlx-vlm's ClippableLinear needs them on the multimodal side; drop tied lm_head; multimodal keys keep the bare stripped key with two conv transposes (:1649); .experts.gate_up_proj split on axis 1 at shape[1]/2 → switch_glu.{gate,up}_proj (:1688); everything else re-prefixed language_model.model.{stripped} (:1721).

paddleocr-vl (crates/mlx-core/src/models/paddleocr_vl/persistence.rs:88). Identity-detect first (any language_model. prefix ⇒ already MLX); otherwise merge visual q/k/v into a single .qkv. via concatenate_many on axis 0 (:114), patch_embedding.weight 4-D transpose, key rewrites (:14).

qianfan-ocr (crates/mlx-core/src/models/qianfan_ocr/persistence.rs:111). Pure key rename plus one Conv2d transpose gated on the heuristic shape[1] < shape[2] (:85). Unmatched keys fall through unchanged.

privacy-filter (crates/mlx-core/src/convert.rs:1567). Identity pass — Ok(weights). Quantization is owned by a dedicated predicate block (:2813) that re-derives a complete per-layer override map from the resulting .scales keys.

convert_foreign_weights (crates/mlx-core/src/utils/foreign_weights.rs:49) is a separate NAPI export. It never touches convert_model, ConversionRecipe, quantization, or sharding. It always writes exactly model.safetensors plus a synthesized config.json — the source directory's config is not read or copied.

model_type dispatch                          foreign_weights.rs:71
  "pp-lcnet-ori" → convert_pp_lcnet_ori      :118
  "uvdoc"        → convert_uvdoc             :168
  other          → Err "Unknown foreign model type"

per-type auto-detect
  dir + inference.pdiparams + inference.json → load_paddle_inference_params  :849
  dir fallback                               → *.pdparams / *.pkl|*.pt|*.pth
  file .pdiparams                            → needs sibling .json, else Err
  file else                                  → pytorch ZIP-pickle           :363

.pdiparams stream layout per tensor (:879-987): u32 version → u64 lod_count → LoD levels → u32 desc version → u32 proto_size → TensorDesc protobuf → raw payload of prod(dims) * elem_size bytes.

Name↔tensor pairing (:992) is purely positional:

names  = extract_param_names_from_json()   // ops with "#":"p", name at A[3]     :1014
ONLY CHECK: names.len() == tensors.len()                                          :992
names.sort_by_key(strip_deepcopy_suffix)                                          :1002
zip(names, tensors_ordered)                //  dims DISCARDED: `(_dims, array)`   :1005

Dtype table (paddle_dtype_to_str, :1192): 2→i32, 3→i64, 4→f16, 5→f32, 6→f64, _ → "f32". Paddle's VarType also defines BOOL=0, INT16=1, UINT8=20, INT8=21, BF16=22 — none mapped. numpy_dtype_size also defaults to 4 (:776), and parse_tensor_desc defaults dtype = 5 when the field is absent (:1095).

Format A (individual)                     Format B (pre-stacked fused)
 …experts.{i}.gate_proj.weight            …experts.gate_up_proj   [E, 2*I, H]
 …experts.{i}.up_proj.weight              …experts.down_proj      [E, H, I]
 …experts.{i}.down_proj.weight
         │ mx.stack(axis=0)                        │ slice axis 1 at I
         ▼                                         ▼
     switch_mlp.{gate,up,down}_proj.weight  [E, out, in]   (qwen, lfm2)
     switch_glu.{gate,up,down}_proj.weight  [E, out, in]   (gemma4)

Detection is by key probe and A wins if both are present (crates/mlx-core/src/convert.rs:858, warning at :866).

Why FP8 dequant must run BEFORE stacking. Inside Qwen35Recipe::sanitize, Step 2 (FP8 dequant, :695) consumes every *weight_scale_inv* pair; Step 3 (expert stacking, :846) consumes only *.weight. dequant_fp8 (:6801) is a 2-D block-wise op — it reads shape[0]/shape[1], pads both to 128, reshapes to [m/128, 128, n/128, 128], and broadcasts scale_inv[m_blocks, 1, n_blocks, 1]. A [E, N, K] stacked tensor has no valid interpretation under that reshape. Stacking first would also orphan every per-expert weight_scale_inv, because Step 3's cleanup only deletes .weight keys (:947). The per-expert case is additionally rejected outright before any mutation (:442), and lfm2 rejects the same shape by name and by dtype (:1306, :1345).

3-D stacked experts are NOT excluded on the recipe paths — quant_entry_emits only requires ndim >= 2 (:5899), and quantize_with_optional_tiling splits axis 0 into 32-expert tiles once E >= 32 to dodge the ~5 s macOS GPU watchdog (:5235). They are forced to 8-bit affine only under --q-mode sym8, via sym8_eligible's ndim != 2 → false.

convert_model never writes vision.safetensors. Vision/audio tensors stay in the main sharded output under their family prefix.

The vision.safetensors sidecar is a GGUF-path artifact, written only by the --mmproj second pass (packages/cli/src/commands/convert.ts:567). At load it is never required:

append_vision_safetensors(dir, load_vision, params)     engine/persistence.rs:176
   if !load_vision           → Ok(())        :181
   if !vision_path.exists()  → Ok(())        :185   ◀ SILENTLY OPTIONAL
   else → params.extend(load_safetensors_lazy(...))

It is appended after expand_symmetric_affine_biases in both branches (crates/mlx-core/src/engine/persistence.rs:120, :162) deliberately, so a Q4_0 body + Q8_0 mmproj cannot inherit each other's zero point.

is_mtp_key(k)         = strip_wrapper_prefix(k).starts_with("mtp."|"mtp_") || k.contains(".mtp.")
                        crates/mlx-core/src/convert.rs:3399
normalize_mtp_prefix  → models::mtp_drafter::strip_wrapper_prefix (shared longest-first chain)  :3416

The delegation is load-bearing: a triple-wrapped model.language_model.model.mtp.… must collapse before the bare-prefix test, or it falls into the language-model remap branch (:530).

Carry policy → emission:

MtpPolicy::Sidecar (dense qwen3_5)  → extract to mtp.safetensors    convert.rs:2979, :3255
MtpPolicy::Inline  (qwen3_5_moe)    → quantized in place, main shards
MtpPolicy::None    (all others)     → nothing
quant_mtp == "split"                → mtp-drafter/ dir, EXCLUDED from Sidecar     :2977

write_mtp_drafter_dir (:3552) emits model.safetensors, a config.json with model_type: "qwen3_5_mtp", block_size = mtp_num_hidden_layers + 2 (:3624), tie_word_embeddings defaulting to true when absent (:3645), and copies tokenizer assets from the source dir. It first removes stale legacy sidecars (:3050), because the dense loader probes mtp.safetensors before mtp-drafter/.

MTP sanitize is deliberately split from the body: the main +1.0 norm loop skips mtp.*, but an independent probe (:1069-1083) samples the mean of mtp.layers.0.input_layernorm.weight and shifts the seven MTP norms when mean < 0.5 (:1146). The comment at :1141 records that a previous revision skipped mtp.* entirely and produced zero MTP acceptance.

Pre-quantized MTP sources are refused when do_quantize && has_custom_sanitizer (crates/mlx-core/src/convert.rs:2511, predicate is_pre_quantized_mtp_key at :3409): re-quantizing the body rewrites the global quantization block, through which mtp.rs::apply_weights resolves missing per-tensor overrides, so an affine-8 MTP head would silently load as NVFP4-4/g16.

is_gemma_qat_source = config.quantization_config.quant_method == "gemma"     convert.rs:2258
is_gemma_qat_family = nvidia_recipe_family(model_type) == Some("gemma4")
                      && is_gemma_qat_source                                          :2264
   │
   ├─ is_gemma_qat_family + any of -q / --q-recipe / --imatrix-path / --q-mtp → Err    :2267
   ├─ is_gemma_qat_source + unified (model_type or architectures[0])          → Err    :2284
   └─ is_gemma_e2b_import (EXACT --model-type "gemma4")                                :2266
          validate_e2b_qat_schedule(&config)                                           :2306
          import_gemma_prequantized(tensors, &config, dtype)                           :2560

All three gates are pure config reads, evaluated before the process-wide convert mutex and before CpuConvertGuard::enter_cpu() (:2252) — a bad source is rejected without touching MLX. validate_e2b_qat_schedule (crates/mlx-core/src/convert_gemma_import.rs:92) compares module_quant_configs against a hard-coded 6-entry regex→bits table (:77); nothing is derived from the checkpoint, so any other gemma4 QAT variant is rejected rather than mis-repacked.

Routing (crates/mlx-core/src/convert_gemma_import.rs:483):

Bit routing: lm_head/embed_tokens → 2; attention q/k/v/o → 4; MLP → 4 for layer <= 14, else 2 (:292). Dropped (:235): .input_activation_scale, .output_activation_scale, .k_cache_scale, .v_cache_scale — the entire a8o8 half of "wNa8o8". Skipped (:243): all audio, relative_k_proj, .per_dim_scale, rotary_emb.

The lossless mapping (crates/mlx-core/src/utils/gemma_quant_repack.rs:1):

Google:  w[o,c] = (q_unsigned[o,c] - 2^(bits-1)) * weight_scale[o]
MLX:     w[o,c] =  q_unsigned[o,c] * scales[o,g] + biases[o,g],  g = c/group_size
⇒  pack the RAW nibble/crumb (no subtraction)
   scales[o,g] = weight_scale[o]
   biases[o,g] = -(2^(bits-1)) * weight_scale[o]

Size cost at LINEAR_GROUP_SIZE = 128, in = 2048:

Lossless but ~+0.48 bpw of pure redundancy, since every group in a row carries the identical source scale. 128 is MLX affine's largest legal group, which halves that overhead versus 64.

A convert-time tripwire, verify_override_coverage (crates/mlx-core/src/convert_gemma_import.rs:268), fails the conversion if any .scales-bearing output lacks a per-layer override. It exists only here — see gotchas.

nemotron_h (source model_type: "nemotron_h", quantization_config.quant_method == "modelopt", producer modelopt) is an ingest of an already-quantized checkpoint, not a quantizer. The hf_quant_config.json declares MIXED_PRECISION: W4A16_NVFP4 (group_size 16) on every expert, shared_experts, and lm_head, plus plain FP8 on the Mamba-2 mixer.in_proj / mixer.out_proj, with kv_cache_quant_algo: "FP8". The convert pass repacks those formats into MLX storage and drops what v1 cannot use:

• NVFP4 experts / shared experts / lm_head — source weight is U8 with two E2M1 4-bit codes per byte, weight_scale is per-16-group E4M3 [N, K/16], and weight_scale_2 is an F32 scalar per tensor (not a power of two). Ingest is byte-preserving on both halves of the NVFP4 encoding:

  • the E2M1 codes are packed into MLX nvfp4 u32 weight storage bit-exact, and
  • the per-group E4M3 scale bytes are copied verbatim into .scales — byte-identical to the source weight_scale.

  weight_scale_2 is not folded into those scales. It is carried out-of-band as a separate Float32 .global_scale key and applied at runtime as a scalar multiply on the projection output. This is vLLM's convention (nvfp4_marlin_process_global_scale in vllm/model_executor/layers/quantization/utils/marlin_utils_fp4.py; modelopt.py renames weight_scale_2 → weight_global_scale and never folds it — the only thing vLLM ever multiplies into an E4M3 group scale is a power of two, which is exact by construction).

  Shapes, because weight_scale_2 is per expert, not per layer: for the stacked [E, N, K] switch-linear pairs .global_scale is an [E] F32 vector gathered by the routing indices (measured on the real checkpoint, layer 1: up_proj 5.658e-5 … 2.124e-4, a 3.75× spread over 80 distinct values across 128 experts; down_proj 4.18× over 94 values — a single scalar would mis-scale 127 of 128 experts). shared_experts gets a per-tensor scalar. lm_head gets none (see below).

  Why the fold was removed. weight_scale_2 ≈ 8.6e-5, so weight_scale[o,g] * weight_scale_2 lands in E4M3's subnormal band for 99.77% of groups and the re-encode loses a median 6.67% / mean 8.15% of each group scale. Measured weight relative Frobenius error against the exact NVIDIA weight on three real tensors: 8.99% / 8.04% / 8.21%. Carrying the global scale separately is exact instead — the matmul is linear in the weight, so (code · decode_e4m3(weight_scale)) @ xᵀ · weight_scale_2 reproduces NVIDIA's intended weight with 0.0 max-abs weight error (measured activation error 2.2e-13%, i.e. f64 round-off).

  Consequences, stated plainly:

  • Reconstruction is now exact rather than ~8% off on every MoE expert projection.
  • The output is no longer a plain mlx-lm-loadable nvfp4 checkpoint — mlx-lm has no .global_scale concept. (Precedent: the symmetric Q4_0/Q8_0 derived-bias drop.)
  • .global_scale must stay Float32; bf16 would round weight_scale_2 with ~0.4% error and silently reintroduce a smaller version of the same bug. NemotronH owns its dtype cast (owns_dtype_cast() == true), so the key bypasses convert's BF16 pass.
  • The loader is fail-closed: a missing, wrong-dtype, or wrong-length .global_scale on an nvfp4 NemotronH prefix is a hard error. It is never defaulted to 1.0 — that would leave the projection ~1.15e4× too large with no diagnostic.
  • Existing converted checkpoints must be regenerated. Anything produced by the folding ingest carries pre-multiplied .scales and no .global_scale; the fail-closed check turns such a checkpoint into a loud load error rather than a silently-8%-wrong model.

• lm_head — NVFP4 in the source, dequantized to bf16 at ingest (a single dense matmul; keeping the widest matrix in bf16 avoids a 4-bit gather on it). The dequant uses the exact scale: dequantize with the unmodified weight_scale into F32, multiply by weight_scale_2, then cast to bf16. It therefore needs no runtime hook and emits no .global_scale key.

• FP8 Mamba-2 projections (mixer.in_proj / mixer.out_proj) — source weight is raw E4M3 [N,K] with an F32 scalar weight_scale and a static input_scale [1]. Ingest re-quantizes them to affine 8-bit group-32 (fp8_to_affine8: reconstruct from_fp8(w) * weight_scale, cast to BF16, then mlx_quantize("affine", 32, 8), emitting the mandatory .biases sidecar) and threads the checkpoint's input_scale as a static input_amax on each per-layer override, so at load the projection fake-quantizes its activation (from_fp8(to_fp8(x·448/amax))·amax/448). Divergence: modelopt's runtime scales activations per-token dynamically, while the exported input_scale is a fixed value — v1 keeps it as a static amax (same mechanics as mlx calibrate's sites, minus the calibration pass). These were mxfp8 8/32 until the quantization-accuracy pass. At the time MLX rounded the E8M0 block exponent to nearest and lost 6.1191% relative RMS (20.96% max/amax) on a per-tensor-E4M3 source. Today convert and MLX (upstream 02adf7b21) both round it up, and mxfp8 still loses 2.964% relative RMS (3.733% max/amax) versus 0.6366% (0.8669%) for affine 8/32 — about 4.7x better, measured on a seeded Gaussian [256, 2688] fixture in nemotron_fp8_to_affine8_requant_error_within_tolerance (both values print with --nocapture). What is left is E4M3's own element grid. The load-side gate below makes the choice two-sided, not a one-line swap. The on-disk consequences: .scales U8 -> BF16, a NEW BF16 .biases, .weight unchanged, and the per-layer mode string "mxfp8" -> "affine". A pre-pass checkpoint is rejected at load with a regenerate hint (reject_legacy_mxfp8_mamba / require_affine_sidecars) rather than silently running 6% off, because the .weight bytes are identical between the two encodings and only the declared mode can tell them apart. Both gates are exercised end to end through load_inner by load_inner_rejects_a_pre_affine_checkpoint_end_to_end. Every stale-format rejection in this family's loader — mxfp8 mamba, a missing affine sidecar, a missing or malformed .global_scale, a BF16 router bias — ends in one shared REGENERATE_HINT sentence naming mlx convert -m nemotron_h.

• k_scale / v_scale (attention FP8 KV-cache scales, F32 [1]) — dropped. v1 keeps a bf16 KV cache, so the FP8 KV scales have no consumer and are discarded, not carried.

• mtp.* — already bf16 in the source; retained bf16 verbatim.

• Everything else (norms, embeddings, A_log/D/dt_bias, conv1d, attention q/k/v/o projections, router gates) is bf16 and passes through as-is.

Because the source is already quantized, the re-quantization flags (-q/--quantize, --q-recipe, --q-mxfp, --imatrix-path, --q-mtp) are rejected upfront on a modelopt nemotron_h source — repacking quantized weights through the recipe decision engine would double-quantize them.

The SafeTensors converter writes one key, quantization, and deletes any quantization_config the source carried:

output_config["quantization"] = quant_obj;

That delete is load-bearing. quantization_config is NVIDIA modelopt's and gemma-QAT's name for the input encoding, so leaving it beside the block just written gives the loader two aliases that disagree — which select_quantization_block rejects outright.

The reader still accepts quantization_config; dropping it would make an NVIDIA source un-ingestable. The GGUF converter still writes both keys.

Built by one function so the frontends cannot diverge (build_quantization_object, crates/mlx-core/src/convert.rs):

{ "group_size": <int|null>, "bits": <int>, "mode": "<str>",
  "<normalized per-tensor key>": { "bits": …, "group_size": …, "mode": "…" }, … }

Per-tensor keys go through normalize_override_key (crates/mlx-core/src/utils/mod.rs:27), which forces the mlx-lm/mlx-vlm language_model.model.* prefix — except for privacy-filter, which keeps raw keys (crates/mlx-core/src/convert.rs:6740). The loader re-strips wrappers with normalize_per_layer_key (crates/mlx-core/src/models/quant_dispatch.rs:528).

Top-level group_size is JSON null for sym8 (crates/mlx-core/src/convert.rs:3132). It can never be fp8_e4m3 — that mode is not in VALID_QUANT_MODES (:1967), so it only ever appears in per-tensor children via serialized_quant_override (:5589).

MTP is deliberately excluded (skip_mtp = true). Quantized MTP linears are described instead by a separate mtplx_mtp_quantization object pinned to {4, 32, affine} (:3155).

There is no mixed field in any emitted config — grep -rn '"mixed"' --include=*.rs crates/ returns nothing. mixed is a local bool in the GGUF importer.

GGUF modal derivation (crates/mlx-core/src/utils/gguf.rs:2344):

default_profile = argmax over profiles by (count, profile)   // modal, ties by Ord
mixed           = any tensor profile != default_profile
requires_explicit_entry() = self.mode != "affine"            // ⇒ every K-quant tensor is named

SourceQuantProfile identity is the full 4-tuple (bits, group_size, mode, symmetric_zero_point) (:1872) — deliberately not bits alone, because Q4_0 and Q4_K are both 4-bit through different kernels, and Q4_0 vs Q4_1 differ only in derived-vs-stored offset.

The gemma-QAT importer is the only path that derives an honest top-level block from its own overrides (crates/mlx-core/src/convert_gemma_import.rs:139): it takes the modal bits and hard-errors if the overrides mix mode or group_size ("no honest top-level quantization.mode exists").

load_with_thread(path)                              crates/mlx-core/src/engine/persistence.rs
  └─ load_all_safetensors(dir, load_vision)         :98
       single-file branch                           :102-123
         1. load_safetensors_lazy                   :112
         2. expand_symmetric_affine_biases   ◀ FIRST :120
         3. append_vision_safetensors               :121
       sharded branch                               :125-167
         1. load every shard, extend                :155
         2. expand_symmetric_affine_biases   ◀ SAME ORDER, deliberate  :164
         3. append_vision_safetensors               :165
  ├─ prewarm_checkpoint_pages
  ├─ dequant_fp8_weights
  ├─ sanitize_weights
  ├─ validate_required_weights
  ├─ MoE gate/up fusion
  └─ apply_weights  ◀ LAYER CONSTRUCTION; ensure_affine_biases_present fires here

Bias expansion must precede the sidecar join (:113): SymmetricZeroPoints::for_key falls back to the top-level default for any key with no entry (quant_dispatch.rs:806), so a Q4_0 main model paired with a Q8_0 mmproj would otherwise derive every vision bias at the wrong offset. "Running afterwards turns a loud missing-.biases failure into silent corruption."

The missing-weight message is deliberately bare and uninterpolated (crates/mlx-core/src/models/gemma4/persistence.rs:502-807): Missing required weight: layers.7.self_attn.q_proj.weight. Rationale at :500: without it "the branch found no .weight to load, and the model silently kept its constructor-RANDOM weights."

There is no dequantize/requantize path. Re-converting keeps the packed bytes, so pass 2 must both prove the bytes match the new request and re-emit any override that moves the tensor off the top-level triple. Phase 1 of quantize_weights_inner is read-only against the pristine input map (crates/mlx-core/src/convert.rs:6404):

for {base}.weight whose {base}.scales or {base}.weight_scale_inv already exists     :6418
  ── recipe active ────────────────────────────────────────────────
     Skip    → Err "recipe requires this tensor dense"
     Default → !should_quantize ⇒ Err ; else default entry
     Custom  → entry from the recipe decision
     then validate_existing_quantized_entry  :6454
          record_quant_override_if_non_default  :6455
  ── no recipe, has_scales ────────────────────────────────────────
     Uint8 weight + float scales (plain FP8) → Err                 :6475
     else resolve_legacy_entry(for_existing = true)                :6499
          None ⇒ Err "default predicate does not quantize"
     validate (unless deferred to sym8 group coherence)
     record_quant_override_if_non_default                          :6528

Validation by resolved mode (validate_existing_quantized_entry, :5620): affine needs u32 weight + float scales + mandatory float .biases; mxfp/nvfp needs u32 + u8 scales and .biases absent; sym8 needs int8 + float scales; fp8_e4m3 needs rank 2/3 and no .biases; K-quants get a full ggml geometry check.

The recording half is the regression, spelled out at :6493:

"Record: a key the ladder moves off the top-level triple (every router gate, pinned to 8-bit affine) needs its per-layer override re-emitted. Without it the config writer stamps the top-level triple over 8-bit bytes and the same nameless decode failure comes back — validating alone would only have proved the bytes were fine."

The nameless failure (:6488): group-32 data relabelled group-64, or K-quant bytes relabelled affine, survive the write and surface at first decode as null handle returned: quantized_matmul, naming neither the layer nor the shape.

Second round-trip hazard: the derived-bias claim. expand_symmetric_affine_biases runs at the converter's input boundary (:2447), so the output is a stored-bias checkpoint. strip_symmetric_zero_point (:1930) therefore removes symmetric_zero_point from both aliases and from every child object before writing. Without it, a plain mlx convert of an imported model produces an unloadable model — the loader hits the "derived or stored, never both" contradiction.

A = { prefix : "{prefix}.scales" ∈ safetensors index }        # what is actually packed
B = { normalize_per_layer_key(k) : k ∈ config.quantization ,  # what is described
                                   value is an object }
A \ B must be ∅.  Anything in A \ B decodes with the top-level triple —
which for several recipes describes zero tensors on disk.

A standard GGUF v3 whose tensors come in pairs, all F32 — any other type is a hard error (crates/mlx-core/src/utils/imatrix.rs:243):

{name}.in_sum2   [input_channels]   sum of squared activations over calibration
{name}.counts    [1]                number of calibration tokens
importance = in_sum2 / counts

Keys are mapped blk.N.ffn_gate.weight → model.layers.N.mlp.gate_proj.weight via gguf_name_to_hf — imported from utils/gguf.rs (crates/mlx-core/src/utils/imatrix.rs:20), so name rules are shared even though the two modules have independent GGUF header parsers.

s_j = max(importance_j, 1e-8) ^ ratio                    convert.rs:7187
s   = s / max( sqrt(max(s) * min(s)), 1e-8 )
ratio = 0.5, hardcoded at BOTH entry points:
  crates/mlx-core/src/convert.rs:2724          (SafeTensors)
  crates/mlx-core/src/utils/gguf.rs:2472       (GGUF, reached from gguf.rs:2842)

Fold groups (crates/mlx-core/src/convert.rs:6885, impl :6920):

self_attn.o_proj and linear_attn.out_proj are not covered by design — their inputs come from attention/GDN compute, not a norm (:6891).

Which norm is correct — the code says pre_feedforward_layernorm for gemma4. :6932 probes {prefix}.pre_feedforward_layernorm.weight first and only falls back. Rationale at :6926: gemma4 sandwich layers feed the MLP from pre_feedforward_layernorm; their post_attention_layernorm normalizes the attention output into the residual, so folding 1/s there rescales the attention branch and leaves gate/up columns uncompensated. Verified against the model: gemma4's decoder layer is h = x + post_attention_layernorm(r); r = mlp(pre_feedforward_layernorm(h)) (crates/mlx-core/src/models/gemma4/decoder_layer.rs:79, forward at :421). Two-norm families (qwen3_5, lfm2) have no pre_feedforward_layernorm and keep post_attention_layernorm.

Group A takes the element-wise max of gate/up importance and accepts a single present key (:7096); Groups C/D require ALL keys or skip with a warning, because "partial AWQ correction is worse than none" (:7126).

Recipe acceptance:

Pre-quantized bodies are rejected before any mutation (reject_awq_for_prequantized_body, :6861): "Packed weights cannot be safely AWQ-scaled and this converter has no dequantize/requantize path; refusing before mutating tensors."

State layout (crates/mlx-core/src/calibration/activation_amax.rs:12) is split on purpose:

CALIBRATING  thread_local Cell<bool>                   :36   arm flag, per model thread
AMAX         LazyLock<Mutex<HashMap<String,f32>>>      :47   running max, process-global
calib_guard  tokio Mutex, try_lock                 napi.rs:40   serializes RUNS (2nd run fails fast)

Arming is RAII (CalibrationArmGuard, :131) so every exit path disarms. record (:91) folds a non-finite-preserving running max: once a key sees inf/NaN it stays non-finite, because plain f32::max ignores NaN and a later finite sample would erase the evidence.

Exactly six calibrated sites (is_activation_fp8_site, :162): .self_attn.{q,k,v,o}_proj, .linear_attn.in_proj_qkvz, .linear_attn.out_proj. in_proj_ba (affine 8/64) is deliberately absent. The forward tap additionally requires self.mode == MXFP8_MODE (crates/mlx-core/src/models/qwen3_5/quantized_linear.rs:723).

fake-quant           crates/mlx-core/src/quant/fp8_activation.rs:14
  s_in = 448 / amax ;  s_out = amax / 448
  xq   = from_fp8( to_fp8( x.astype(f32) * s_in ) , f32 ) * s_out   → astype(x.dtype)
  amax <= 0.0 ⇒ return x unchanged
  (the f32 upcast before the pre-scale is required for bit-exactness with modelopt)

gate at use          quantized_linear.rs:746
  requires input_amax > 0 AND mode == MXFP8_MODE,
  and is SUPPRESSED entirely while calibrating (so re-calibration measures raw bf16)

Pipeline and failure ladder (crates/mlx-core/src/calibration/napi.rs:230):

readCalibTexts(dataset, calibSize)        calibrate.ts:33   first N rows IN FILE ORDER, no shuffle
        │  0 rows → throw, never reaches native
        ▼
calibrateActivationAmaxRaw(modelPath, rows, calibSeq)
  1. calib_guard().try_lock()                                        :242
  2. read_model_type(config.json) — qwen3_5 | qwen3_5_moe ONLY       :251 / Err at :293
  3. ActivationAmaxCollector::take()   (clear residue from a panic)  :257
  4. per row: tokenize raw text → truncate → PREFILL ONLY
       (no chat template, no generated token; caches reset per row)
  5. rows_prefilled == 0    → Err              config UNTOUCHED      :104
  6. amax map empty         → Ok(0)            config BYTE-UNTOUCHED :154
  7. any amax non-finite    → Err              config UNTOUCHED      :163
  8. write_amax_into_config()                  activation_amax.rs:201
  9. written < expected     → Err                                    :176

write_amax_into_config iterates config entries, not the collected map, because config keys are raw/wrapped and store the GDN input projection split (in_proj_qkv, in_proj_z) while the collector uses stripped keys with it merged (in_proj_qkvz); the merged amax fans out to both (:343). Only per-layer object entries with a parseable integer bits gain input_amax (:332), mirroring the loader's materialization gate. The write is atomic: serialize to .{name}.tmp.{pid}.{nanos} in the same directory, then rename(2) (:264). If homed < expected it returns early without writing (:257).

Stated three times in code plus docs/cli.md:257-259: Apple GPUs have no FP8 matmul hardware, so this is numeric parity only, not a speedup (crates/mlx-core/src/quant/fp8_activation.rs:7, crates/mlx-core/src/models/qwen3_5/quantized_linear.rs:744).

The highest-value section. Everything here is confirmed in code.

8 guards live only in packages/cli/src/commands/convert.ts and 15 live only in Rust. Of the TS-only ones, only two are genuinely reachable by a direct NAPI caller (--mmproj and --imatrix-path path/extension checks) — the rest are structurally impossible or fail later anyway. Of the Rust-only ones the load-bearing one is the nvidia family gate (crates/mlx-core/src/convert.rs:5027, :5042, :5054): it reads the input config.json's own model_type, not -m, precisely because the CLI forwards an explicit -m verbatim and skips auto-detection.

Because all three NAPI entry points (convertGgufToSafetensors, convertForeignWeights, convertModel) are public exports of @mlx-node/core (packages/core/index.cjs:784), every TS-only guard is bypassable.

Two consequences worth naming:

• --q-recipe unsloth's documented 3-bit base is a TS-only default (packages/cli/src/commands/convert.ts:383). Rust has none — default_bits comes from --q-mode alone. A direct convertModel({quantRecipe:'unsloth', quantize:true, imatrixPath:'…'}) NAPI call silently produces the 4/5/6/8/6 ladder instead of the documented 3/4/5/6/5. (Without imatrixPath the call hard-errors at crates/mlx-core/src/convert.rs:2148 — it does not silently produce anything.)
• The nvfp4 arm of effectiveQuantBits is not redundant. It is a ternary chain: quantMode === 'nvfp4' ? 4 : quantRecipe === 'unsloth' ? 3 : undefined. For the documented --q-recipe unsloth --q-mode nvfp4 pair the nvfp4 arm preempts the unsloth 3-bit arm; delete it and the TS nvfp4 invariant check at convert.ts:430 rejects the invocation with "nvfp4 requires bits=4 and group_size=16". The code comment at convert.ts:379 says this verbatim.

Asymmetries where the code is right but the quality rationale does not survive scrutiny:

• linear_attn.out_proj is protected asymmetrically. validate_nvfp4_recipe (crates/mlx-core/src/convert.rs:4930) rejects --q-mode nvfp4 --q-recipe mixed_* precisely because out_proj (KLD ~6.0, "worst tensor") would be promoted to a 4-bit float format with no affine fallback. But --q-mxfp --q-recipe mixed_4_6 promotes the same tensor to mxfp4 4/32 with no gate, and the behaviour is locked in by a passing test (:8394).
• build_official_unsloth_recipe fails silently in two opposite directions where mixed_* fails loudly: FFN tensors with no parseable layer index go to bf16 (None => Skip, :4566), and num_layers == 0 makes final_eight_start = 0.saturating_sub(8) = 0, so EVERY layer is "final eight" and the whole model gets the high format (:4488). build_recipe_predicate errors when it cannot infer num_layers (:3966).
• mlx calibrate supports only qwen3_5 / qwen3_5_moe, but mlx convert --q-recipe nvidia now also produces gemma4 / gemma4_unified checkpoints — whose mxfp8 attention sites therefore keep bf16 activations permanently, with no path to modelopt W8A8 parity. Neither the CLI help nor docs/cli.md mentions the restriction (crates/mlx-core/src/calibration/napi.rs:259, crates/mlx-core/src/convert.rs:4952).
• Pre-K-quant Q6_K fallback is still live and narrowly scoped. With --gguf-kquant OFF, a Q6_K tensor is accepted only if general.architecture == "gemma4" and the tensor is exactly token_embd.weight, in which case it is dequantized to dense BF16. Every other Q6_K position, and all Q4_K/Q5_K, hard-error. The same source tensor therefore lands in two completely different on-disk formats depending on one flag — 6.5625 bpw packed vs 16 bpw dense. A size or quality comparison that does not pin the flag is meaningless (crates/mlx-core/src/utils/gguf.rs:1085, :633).