Ling-2.6-1T is a large-scale language model based on the bailing_hybrid architecture, featuring a hybrid attention mechanism combining Multi-Head Latent Attention (MLA) and Linear Attention, with a Mixture-of-Experts (MoE) design.
| Attribute | Value |
|---|---|
| Model Type | bailing_hybrid |
| Architecture | BailingMoeV2_5ForCausalLM |
| Hidden Size | 8192 |
| Layers | 80 |
| Attention Heads | 64 |
| Experts | 256 (routed) + 1 (shared) |
| Context Length | 262,144 |
| Quantization | compressed-tensors FP8 dynamic |
| License | MIT |
This model is available on ModelScope.
Please refer to the model card on HuggingFace or ModelScope for citation information.