Ming-omni-tts-0.5B NPU

# 基础框架
pip install torch==2.9.0
pip install torch_npu==2.9.0.post1

# 推理依赖
pip install transformers>=4.45.0
pip install safetensors

# 可选：端到端 TTS 推理
pip install modelscope

# 从 ModelScope 下载
pip install modelscope
modelscope download --model inclusionAI/Ming-omni-tts-0.5B --local_dir ./Ming-omni-tts-0.5B

# 克隆 NPU 适配代码
git clone https://gitcode.com/your-org/Ming-omni-tts-0.5B-NPU.git
cd Ming-omni-tts-0.5B-NPU

import torch
import torch_npu
from configuration_bailingmm import BailingMMConfig
from modeling_bailingmm import BailingMMNativeForConditionalGeneration

# 加载模型
config = BailingMMConfig.from_pretrained(".")
model = BailingMMNativeForConditionalGeneration.from_pretrained(
    ".", config=config, torch_dtype=torch.bfloat16,
)

# 移至 NPU
model = model.to("npu:0")
model.eval()

# 推理
input_ids = torch.randint(0, 151936, (1, 32)).to("npu:0")
with torch.no_grad():
    output = model(input_ids=input_ids)
print(f"Output shape: {output.shape}")

python inference.py --mode benchmark --device-id 0 --seq-len 32

Ming-omni-tts-0.5B (BailingMM)
├── model (Qwen2ForCausalLM)        # 文本骨干网络，24层
├── audio                           # 音频 VAE
│   ├── encoder (AudioVAEEncoder)   # Mel → 64维潜在空间
│   │   ├── encoder (Qwen2Model)    #   24层编码器
│   │   └── aggregator (Qwen2Model) #   4层聚合器 + CLS Token
│   └── decoder (AudioVAEDecoder)   # 潜在空间 → 波形
│       ├── decoder (Qwen2Model)    #   24层解码器
│       └── head (ISTFTHead)        #   ISTFT 波形合成 (n_fft=3528)
├── linear_proj_audio (DiTModel)    # DiT 音频细化，8层
├── spk_head (Linear)               # 说话人编码投影
├── stop_head (Linear)              # 停止预测
└── flowloss (FlowLoss)             # CFM 训练模型（仅用于权重加载）

======================================================================
  SUMMARY - FP32 Accuracy (NPU vs CPU)
======================================================================
  LLM_Backbone         | FP32 mean rel diff: 0.00040307 (0.0403%) | BF16 speedup: 43.64x | PASS
  AudioEncoder         | FP32 mean rel diff: 0.00025325 (0.0253%) | BF16 speedup: 54.78x | PASS
  DiT_Head             | FP32 mean rel diff: 0.00000472 (0.0005%) | BF16 speedup: 30.74x | PASS

Overall FP32 accuracy: ALL PASSED
Accuracy tolerance: FP32 mean relative difference < 1%

Ming-omni-tts-0.5B-NPU/
├── inference.py                    # NPU 推理与精度验证脚本
├── modeling_bailingmm.py           # 模型架构实现
├── configuration_bailingmm.py      # 模型配置类
├── config.json                     # 模型配置参数
├── model.safetensors               # 模型权重（软链接）
├── tokenizer.json                  # 分词器
├── vocab.json                      # 词表
├── merges.txt                      # BPE 合并规则
├── special_tokens_map.json         # 特殊 token 映射
├── tokenizer_config.json           # 分词器配置
├── chat_template.jinja             # 对话模板
├── npu_evaluation_report.json      # 精度评测报告
├── npu_benchmark.log               # 运行日志
└── README.md                       # 本文档