b
bcdhuss/Ling-2.6-1T
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

Ling-2.6-1T

Model Description

Ling-2.6-1T is a large-scale language model based on the bailing_hybrid architecture, featuring a hybrid attention mechanism combining Multi-Head Latent Attention (MLA) and Linear Attention, with a Mixture-of-Experts (MoE) design.

Model Details

AttributeValue
Model Typebailing_hybrid
ArchitectureBailingMoeV2_5ForCausalLM
Hidden Size8192
Layers80
Attention Heads64
Experts256 (routed) + 1 (shared)
Context Length262,144
Quantizationcompressed-tensors FP8 dynamic
LicenseMIT

Usage

This model is available on ModelScope.

Citation

Please refer to the model card on HuggingFace or ModelScope for citation information.