hf_mirrors/Ling-3.1-flash-1019-oQ4e-mtp
Model Introduction
Files & versions
Analysis

Ling-3.1-flash-1019-oQ4e-mtp

This model was quantized using oQ (oMLX v0.7.3rc1) mixed-precision quantization.

A mixed-precision MLX checkpoint of inclusionAI/Ling-3.1-flash-1019, an upgraded version of Ling-3.1-flash with improved reasoning and agentic capabilities. It is a hybrid-linear mixture-of-experts model with 560B total and 25B active parameters, built from 63 KDA linear-attention layers and 9 gated MLA layers, with 512 routed experts of which 8 are active per token. It includes the language model and the MTP (NextN) layer, which oMLX runs as Lightning MTP speculative decoding.

The effective average is 4.615 bits/weight for the complete checkpoint.

Quantization and Bit Distribution

oQ4e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (256 samples x 512 tokens, collected layer by layer from the original FP8 weights) and a weighted least-squares refit of its scale and bias.

Storage formatLogical weightsShare of totalTensor storageEffective bits/weight
Affine 4-bit, group 64555.661B97.123%291.094 GiB4.50
Affine 8-bit, group 1281.085B0.190%1.042 GiB8.25
Affine 8-bit, group 6415.333B2.680%15.173 GiB8.50
BF160.041B0.007%0.076 GiB16.00
FP320.000B0.000%0.000 GiB32.00
Total572.121B100%307.385 GiB4.615

Storage Breakdown

ComponentLogical weightsStorage
Language backbone563.946B325.363 GB / 303.017 GiB
MTP head8.174B4.690 GB / 4.367 GiB
Total572.121B330.052 GB / 307.385 GiB

License

MIT, following the source model.