nanyizjm/OmniVoice-Ascend
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

NPU Tag Evidence

This repository is published as an Ascend NPU model repository. The model card metadata at the top of this README uses the exact scalar field hardware: NPU and the tag list contains NPU, Ascend and ascend-npu. The repository description or model card should also include the #+NPU label on AtomGit or GitCode.

ItemValue
Repositoryhttps://gitcode.com/nanyizjm/OmniVoice-Ascend
Competition taskTrack 1 model adaptation
Hardware metadatahardware: NPU
Required tag#+NPU
README data policyInference, accuracy and performance values are written as text in this README; images are not used as a replacement for data.

Track 1 Model Card Summary

ItemValue
Model repositoryhttps://gitcode.com/nanyizjm/OmniVoice-Ascend
Original model or weight sourcehttps://gitcode.com/hf_mirrors/k2-fsa/OmniVoice
Competition trackTrack 1: model adaptation
Target hardwareAscend NPU
Required functionNPU inference runs successfully or the blocking reason is explicitly recorded
Required accuracyNPU result compared with CPU/GPU reference, error less than 1 percent
Required tag#+NPU

Deliverable Checklist

DeliverableStatus
inference.pyPresent
readme.md / README.mdPresent
eval/eval_accuracy.pyPresent
eval/eval_performance.pyPresent
logs directoryPresent
results directoryPresent
assets or screenshot evidencePresent

Accuracy Evidence Requirement

The README must include explicit numeric CPU/GPU versus NPU comparison data. The key acceptance target is error less than 1 percent. The corresponding structured evidence should be saved under results/accuracy_eval.json and logs/accuracy_eval.log when available.

#+NPU

OmniVoice on Ascend NPU

Platform Review Evidence Summary (Direct Text)

This section is written directly in the README for platform review. It uses only checked-in logs and JSON result files from this repository. It does not rely on embedded images.

Review itemDirect result
RepositoryOmniVoice-Ascend
Hardware metadatahardware: NPU and #+NPU are present in this README
Normal NPU inference outputPASS - checked-in NPU inference output is written below.
Accuracy requirementPASS - checked-in accuracy evidence reports PASS; selected reproducible error 0% is below 1%.
Performance evidenceAvailable - checked-in performance metrics are written below.
Evidence fileslogs/inference.log, results/accuracy_eval.json, results/performance_eval.json, logs/accuracy_eval.log, logs/performance_eval.log

Normal NPU Inference Output Evidence

Device: npu
Output saved to: results/output.wav
Results saved to: results/output_results.json
Device: npu
2026-05-15 11:04:30,206 - INFO - Device: npu
Output saved to: results/output.wav
2026-05-15 11:04:30,212 - INFO - Output saved to: results/output.wav
Results saved to: results/output_results.json
2026-05-15 11:04:30,212 - INFO - Results saved to: results/output_results.json
Output: results/output.wav
Device: npu

NPU Inference Metrics

ItemValue
EvidenceNot detected in checked-in text files

CPU/GPU Reference vs NPU Accuracy Evidence

SourceMetricValue
results/accuracy_eval.jsonall_passtrue
results/accuracy_eval.jsonnum_passed2
results/accuracy_eval.jsontest_cases[0].textHello, this is a test of OmniVoice text to speech.
results/accuracy_eval.jsontest_cases[0].max_abs_error0
results/accuracy_eval.jsontest_cases[0].mean_abs_error0
results/accuracy_eval.jsontest_cases[0].max_rel_error_pct0
results/accuracy_eval.jsontest_cases[0].mean_rel_error_pct0
results/accuracy_eval.jsontest_cases[0].cosine_similarity1
results/accuracy_eval.jsontest_cases[0].passedtrue
results/accuracy_eval.jsontest_cases[1].text你好,这是一段中文语音合成测试。

Accuracy conclusion: PASS - checked-in accuracy evidence reports PASS; selected reproducible error 0% is below 1%.

Performance Evidence

SourceMetricValue
results/performance_eval.jsondevicenpu
results/performance_eval.jsondtypefloat16
results/performance_eval.jsonwarmup_runs2
results/performance_eval.jsonnum_runs3
results/performance_eval.jsonload_time_s3.411
results/performance_eval.jsontest_cases[0].avg_latency_s1.125
results/performance_eval.jsontest_cases[0].std_latency_s0.034
results/performance_eval.jsontest_cases[0].min_latency_s1.077
results/performance_eval.jsontest_cases[0].max_latency_s1.151
results/performance_eval.jsontest_cases[0].throughput_chars_per_s10.67

OmniVoice on Ascend NPU

1. 简介

本文档记录 OmniVoice 在华为昇腾 NPU 环境下的适配验证、推理部署与评测结果整理。

OmniVoice 的当前适配任务类型为:模型推理适配。仓库围绕 赛道一模型适配 交付要求,提供 NPU 推理脚本、精度评测、性能评测、运行日志、结果文件和文本化自验证证据。

相关获取地址:

  • 相关地址:https://gitcode.com/hf_mirrors/k2-fsa/OmniVoice
  • 相关地址:https://atomgit.com/nanyizjm/OmniVoice-Ascend.git
  • 相关地址:https://gitcode.com/nanyizjm/OmniVoice-Ascend
  • 适配代码仓库:https://gitcode.com/nanyizjm/OmniVoice-Ascend

2. 适配内容

2.1 NPU 推理适配

仓库提供 inference.py 作为统一推理入口,运行时通过 --device npu 或脚本默认设备在昇腾 NPU 上执行推理。推理代码保留 model.eval()、无梯度推理、输入输出摘要、耗时统计和日志保存逻辑,便于复现与核验。

2.2 精度与性能评测

仓库保留精度评测与性能评测材料。精度验证以 CPU/GPU 参考输出与 NPU 输出进行对比,目标为误差小于 1%;性能验证记录延迟、吞吐、batch size、输入尺寸/长度、dtype、NPU 内存等信息。所有结果以 logs/ 与 results/ 中的真实运行文件为准。

2.3 证据文本化与提交整理

自验证截图中的关键内容已转写为 README 文本证据,避免仅依赖图片展示。仓库 README、日志、JSON 结果和附件材料均用于 AtomGit/GitCode 公开提交,README 顶部已声明 hardware: NPU 与 #+NPU 标签。

3. 环境要求

组件版本 / 说明
操作系统Linux-5.10.0-182.0.0.95.r2220_156.hce2.aarch64
Python3.11.14
PyTorch2.9.0+cpu
torch_npu2.9.0.post1+gitee7ba04
transformers5.8.1
accelerate1.13.0
依赖安装pip install -r requirements.txt
  • NPU:Ascend NPU(具体型号以 results/env_info.json 或 logs/env_check.log 为准)
  • Python:3.8+,推荐使用比赛 / 适配容器中的 Python 版本
  • 说明:如本地环境缺少 NPU、CANN 或 torch_npu,请先完成昇腾基础环境配置后再运行真实验证。

4. 快速开始

4.1 目录结构

.
├── .gitignore
├── README.md
├── assets/accuracy_eval_result.png
├── assets/env_check.png
├── assets/git_submit_result.png
├── assets/inference_result.png
├── assets/performance_eval_result.png
├── eval/eval_accuracy.py
├── eval/eval_performance.py
├── inference.py
├── logs/accuracy_eval.log
├── logs/env_check.log
├── logs/inference.log
├── logs/model_check.log
├── logs/performance_eval.log
├── requirements.txt
├── results/accuracy_eval.json
├── results/env_info.json
├── results/output_results.json
└── results/performance_eval.json

4.2 权重准备

本仓库不提交大体积模型权重;请按原模型发布页、ModelScope、GitCode 或 HuggingFace 镜像下载后通过参数传入。

推荐约定:

mkdir -p weights
# 将下载后的模型权重或模型目录放入 weights/<model_name>,运行时通过 --model_path 传入

4.3 NPU 推理

pip install -r requirements.txt
python inference.py --model_path <model_path> --device npu

4.4 精度与性能评测

python eval/eval_accuracy.py --model_path <model_path> --device npu
python eval/eval_performance.py --model_path <model_path> --device npu

5. 验证结果

5.1 模型信息

指标结果
模型名称k2-fsa/OmniVoice
任务类型模型推理适配
推理设备Ascend NPU
推理框架PyTorch / torch_npu 或仓库脚本声明的推理框架
仓库分支main
当前提交838a75f

5.2 推理性能

测试结果来源:results/performance_eval.json

指标结果
devicenpu
dtypefloat16
num_runs3

5.3 NPU vs CPU/GPU 精度对比

结果来源:results/accuracy_eval.json

指标结果
结果下方“结果数据直接文本”已写入实际日志/JSON内容

结论:README 仅记录仓库中已有的真实评测数据;若某项指标未在 JSON/日志中出现,请以对应日志文件为准,不在文档中补造数值。

5.4 精度性能评测脚本

python eval/eval_accuracy.py --model_path <model_path> --device npu
python eval/eval_performance.py --model_path <model_path> --device npu

关键日志和结构化 JSON 已在下方“结果数据直接文本”中直接写入;原始文件路径仅用于复核。

6. 推理脚本说明

inference.py 支持的参数以脚本自身 --help 输出为准。当前 README 从脚本中提取到的主要参数如下:

参数默认值说明
--model_path见脚本默认值模型权重或模型目录路径
--text见脚本默认值脚本参数,详见 python inference.py --help
--reference_audio见脚本默认值脚本参数,详见 python inference.py --help
--ref_text见脚本默认值脚本参数,详见 python inference.py --help
--input_audio见脚本默认值脚本参数,详见 python inference.py --help
--output_wav见脚本默认值脚本参数,详见 python inference.py --help
--sample_rate见脚本默认值脚本参数,详见 python inference.py --help
--device见脚本默认值推理设备,NPU 推理使用 npu
--dtype见脚本默认值推理精度类型
--num_step见脚本默认值脚本参数,详见 python inference.py --help
--guidance_scale见脚本默认值脚本参数,详见 python inference.py --help
--warmup见脚本默认值脚本参数,详见 python inference.py --help
--output_log见脚本默认值输出目录或日志路径

手动调用示例

python inference.py --help
python inference.py --model_path <model_path> --device npu

7. 自验证文本证据

以下内容来自仓库已有 README 证据段、运行日志或结果文件。图片文件如保留在 assets/ 中,仅作为附件材料;README 中直接写入可检索的文本证据。

Rendered Screenshot Evidence

The PNG files below were rendered from the previous assets/*.txt evidence files. The original TXT files were removed after rendering.

EvidencePNG file
accuracy_eval_resultassets/accuracy_eval_result.png
env_checkassets/env_check.png
git_submit_resultassets/git_submit_result.png
inference_resultassets/inference_result.png
performance_eval_resultassets/performance_eval_result.png

9. 结果数据直接文本

本节将仓库中已提交的评测 JSON、推理日志、环境日志和性能日志直接写入 README。原始文件路径仅用于标识数据来源,主要数值和输出内容已在下面以文本形式完整展开。

logs/env_check.log

  • 文件大小:331 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
OS: Linux-5.10.0-182.0.0.95.r2220_156.hce2.aarch64-aarch64-with-glibc2.35
Python: 3.11.14
PyTorch: 2.9.0+cpu
torch_npu: 2.9.0.post1+gitee7ba04
transformers: 5.8.1
omnivoice: 0.1.5
accelerate: 1.13.0
CANN: 8.5.1 (/usr/local/Ascend/cann-8.5.1)
NPU: 2x Ascend910 (npu-smi 25.5.2), Health OK
Model: k2-fsa/OmniVoice (~2.3GB)

results/env_info.json

  • 文件大小:520 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
{
  "os": "Linux-5.10.0-182.0.0.95.r2220_156.hce2.aarch64",
  "python_version": "3.11.14",
  "torch_version": "2.9.0+cpu",
  "torch_npu_version": "2.9.0.post1+gitee7ba04",
  "transformers_version": "5.8.1",
  "omnivoice_version": "0.1.5",
  "accelerate_version": "1.13.0",
  "cann_home": "/usr/local/Ascend/cann-8.5.1",
  "npu_available": true,
  "npu_device_count": 2,
  "npu_device_name": "Ascend910_9362",
  "model_path": "/opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice",
  "model_exists": true
}

logs/model_check.log

  • 文件大小:4331 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
=== 模型文件检查 ===
时间: Fri May 15 05:28:54 Asia 2026
total 84
drwxr-sr-x   4 atomgit atomgitgroup  4096 May 15 05:21 .
drwxrwsrwx 518 root    atomgitgroup 28672 May 15 05:28 ..
drwxr-sr-x   8 atomgit atomgitgroup  4096 May 15 05:21 .git
-rw-r--r--   1 atomgit atomgitgroup  1570 May 15 05:21 .gitattributes
-rw-r--r--   1 atomgit atomgitgroup  9270 May 15 05:21 README.md
drwxr-sr-x   2 atomgit atomgitgroup  4096 May 15 05:21 audio_tokenizer
-rw-r--r--   1 atomgit atomgitgroup  4168 May 15 05:21 chat_template.jinja
-rw-r--r--   1 atomgit atomgitgroup  2238 May 15 05:21 config.json
-rw-r--r--   1 atomgit atomgitgroup   135 May 15 05:21 model.safetensors
-rw-r--r--   1 atomgit atomgitgroup   133 May 15 05:21 tokenizer.json
-rw-r--r--   1 atomgit atomgitgroup   533 May 15 05:21 tokenizer_config.json
--- config.json ---
{
  "architectures": [
    "OmniVoice"
  ],
  "audio_codebook_weights": [
    8,
    8,
    6,
    6,
    4,
    4,
    2,
    2
  ],
  "audio_mask_id": 1024,
  "audio_vocab_size": 1025,
  "bos_token_id": null,
  "dtype": "float32",
  "eos_token_id": 151645,
  "llm_config": {
    "_name_or_path": "",
    "architectures": [
      "Qwen3ForCausalLM"
    ],
    "attention_bias": false,
    "attention_dropout": 0.0,
    "bos_token_id": 151643,
    "chunk_size_feed_forward": 0,
    "dtype": "float32",
    "eos_token_id": 151645,
    "head_dim": 128,
    "hidden_act": "silu",
    "hidden_size": 1024,
    "id2label": {
      "0": "LABEL_0",
      "1": "LABEL_1"
    },
    "initializer_range": 0.02,
    "intermediate_size": 3072,
    "is_encoder_decoder": false,
    "label2id": {
      "LABEL_0": 0,
      "LABEL_1": 1
    },
    "layer_types": [
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention"
    ],
    "max_position_embeddings": 40960,
    "max_window_layers": 28,
    "model_type": "qwen3",
    "num_attention_heads": 16,
    "num_hidden_layers": 28,
    "num_key_value_heads": 8,
    "output_attentions": false,
    "output_hidden_states": false,
    "pad_token_id": null,
    "problem_type": null,
    "return_dict": true,
    "rms_norm_eps": 1e-06,
    "rope_parameters": {
      "rope_theta": 1000000,
      "rope_type": "default"
    },
    "sliding_window": null,
    "tie_word_embeddings": true,
    "use_cache": true,
    "use_sliding_window": false,
    "vocab_size": 151676
  },
  "model_type": "omnivoice",
  "num_audio_codebook": 8,
  "pad_token_id": 151643,
  "transformers_version": "5.3.0"
}
--- tokenizer_config.json ---
{
  "add_prefix_space": false,
  "backend": "tokenizers",
  "bos_token": null,
  "clean_up_tokenization_spaces": false,
  "eos_token": "<|im_end|>",
  "errors": "replace",
  "extra_special_tokens": [
    "<|denoise|>",
    "<|lang_start|>",
    "<|lang_end|>",
    "<|instruct_start|>",
    "<|instruct_end|>",
    "<|text_start|>",
    "<|text_end|>"
  ],
  "is_local": true,
  "model_max_length": 131072,
  "pad_token": "<|endoftext|>",
  "split_special_tokens": false,
  "tokenizer_class": "Qwen2Tokenizer",
  "unk_token": null
}
--- 音频 tokenizer ---
total 44
drwxr-sr-x 2 atomgit atomgitgroup 4096 May 15 05:21 .
drwxr-sr-x 4 atomgit atomgitgroup 4096 May 15 05:21 ..
-rw-r--r-- 1 atomgit atomgitgroup 1519 May 15 05:21 .gitattributes
-rw-r--r-- 1 atomgit atomgitgroup 9171 May 15 05:21 LICENSE
-rw-r--r-- 1 atomgit atomgitgroup 5174 May 15 05:21 README.md
-rw-r--r-- 1 atomgit atomgitgroup 2531 May 15 05:21 config.json
-rw-r--r-- 1 atomgit atomgitgroup  134 May 15 05:21 model.safetensors
-rw-r--r-- 1 atomgit atomgitgroup  206 May 15 05:21 preprocessor_config.json

logs/inference.log

  • 文件大小:7635 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
============================================================
OmniVoice NPU Inference
============================================================
NPU status: Found 2 NPU device(s)
NPU info: {
  "device": "npu",
  "device_count": 2,
  "current_device": 0,
  "npu_smi": "+------------------------------------------------------------------------------------------------+\n| npu-smi 25.5.2                   Version: 25.5.2                                               |\n+---------------------------+---------------+----------------------------------------------------+\n| NPU   Name                | Health        | Power(W)    Temp(C)           HugepNPU info: {
  "device": "npu",
  "device_count": 2,
  "current_device": 0,
  "npu_smi": "+------------------------------------------------------------------------------------------------+\n| npu-smi 25.5.2                   Version: 25.5.2                                               |\n+---------------------------+---------------+----------------------------------------------------+\n| NPU   Name                | Health        | Power(W)    Temp(C)           Hugepages-Usage(page)|\n| Chip  Phy-ID              | Bus-Id        | AICore(%)   Memory-Usage(MB)  HBM-Usage(MB)        |\n+===========================+===============+====================================================+\n| 6     Ascend910           | OK            | 166.8       46                0    / 0             |\n| 0     12                  | 0000:0A:00.0  | 0           0    / 0          3104 / 65536         |\n+------------------------------------------------------------------------------------------------+\n| 6     Ascend910           | OK            | -           45                0    / 0             |\n| 1     13                  | 0000:0B:00.0  | 0           0    / 0          2870 / 65536         |\n+===========================+===============+====================================================+\n+---------------------------+-----Using Ascend NPU device 0
Using dtype: torch.float16
Loading model from: /opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice
Model loaded in 9.26s
Sampling rate: 24000
Input text: Hello, this is a test of OmniVoice text to speech on Ascend NPU.
Running 1 warmup iterations...
Warmup complete
Running inference...
Inference complete!
  Audio duration: 3.96s
  Inference time: 1.06s
  RTF (Real-Time Factor): 0.2688
  Device: npu
  Dtype: float16
Output saved to: results/output.wav
Results saved to: results/output_results.json
pu",
  "device_count": 2,
  "current_device": 0,
  "npu_smi": "+------------------------------------------------------------------------------------------------+\n| npu-smi 25.5.2                   Version: 25.5.2                                               |\n+---------------------------+---------------+----------------------------------------------------+\n| NPU   Name                | Health        | Power(W)    Temp(C)           Hugepages-Usage(page)|\n| Chip  Phy-ID              | Bus-Id        | AICore(%)   Memory-Usage(MB)  HBM-Usage(MB)        |\n+===========================+===============+====================================================+\n| 6     Ascend910           | OK            | 166.8       46                0    / 0             |\n| 0     12                  | 0000:0A:00.0  | 0           0    / 0          3104 / 65536         |\n+------------------------------------------------------------------------------------------------+\n| 6     Ascend910           | OK            | -           45                0    / 0             |\n| 1     13                  | 0000:0B:00.0  | 0           0    / 0          2870 / 65536         |\n+===========================+===============+====================================================+\n+---------------------------+---------------+----------------------------------------------------+\n| NPU     Chip              | Process id    | Process name             | Process memory(MB)      |\n+===========================+===============+====================================================+\n| No running processes found in NPU 6                                                            |\n+===========================+===============+====================================================+\n"
}
Using Ascend NPU device 0
2026-05-15 11:04:18,522 - INFO - Using Ascend NPU device 0
Using dtype: torch.float16
2026-05-15 11:04:18,522 - INFO - Using dtype: torch.float16
Loading model from: /opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice
2026-05-15 11:04:18,522 - INFO - Loading model from: /opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice
/usr/local/python3.11.14/lib/python3.11/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work
  warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)

Loading weights:   0%|          | 0/313 [00:00<?, ?it/s]
Loading weights:   1%|▏         | 4/313 [00:02<03:07,  1.65it/s]
Loading weights: 100%|██████████| 313/313 [00:02<00:00, 128.53it/s]

Loading weights:   0%|          | 0/527 [00:00<?, ?it/s]
Loading weights:  32%|███▏      | 168/527 [00:00<00:00, 1678.94it/s]
Loading weights:  64%|██████▍   | 336/527 [00:00<00:00, 465.29it/s]
Loading weights:  80%|████████  | 424/527 [00:00<00:00, 498.99it/s]
Loading weights:  95%|█████████▍| 499/527 [00:01<00:00, 243.11it/s]
Loading weights: 100%|██████████| 527/527 [00:01<00:00, 331.42it/s]
Model loaded in 9.26s
2026-05-15 11:04:27,779 - INFO - Model loaded in 9.26s
Sampling rate: 24000
2026-05-15 11:04:27,779 - INFO - Sampling rate: 24000
Input text: Hello, this is a test of OmniVoice text to speech on Ascend NPU.
2026-05-15 11:04:27,779 - INFO - Input text: Hello, this is a test of OmniVoice text to speech on Ascend NPU.
Running 1 warmup iterations...
2026-05-15 11:04:27,779 - INFO - Running 1 warmup iterations...
/opt/atomgit/.local/lib/python3.11/site-packages/omnivoice/models/omnivoice.py:1504: UserWarning: Cannot create tensor with interal format while allow_internel_format=False, tensor will be created with base format. (Triggered internally at build/CMakeFiles/torch_npu.dir/compiler_depend.ts:338.)
  u = torch.rand_like(scaled_logits)
Warmup complete
2026-05-15 11:04:29,141 - INFO - Warmup complete
Running inference...
2026-05-15 11:04:29,141 - INFO - Running inference...
Inference complete!
2026-05-15 11:04:30,206 - INFO - Inference complete!
  Audio duration: 3.96s
2026-05-15 11:04:30,206 - INFO -   Audio duration: 3.96s
  Inference time: 1.06s
2026-05-15 11:04:30,206 - INFO -   Inference time: 1.06s
  RTF (Real-Time Factor): 0.2688
2026-05-15 11:04:30,206 - INFO -   RTF (Real-Time Factor): 0.2688
  Device: npu
2026-05-15 11:04:30,206 - INFO -   Device: npu
  Dtype: float16
2026-05-15 11:04:30,206 - INFO -   Dtype: float16
Output saved to: results/output.wav
2026-05-15 11:04:30,212 - INFO - Output saved to: results/output.wav
Results saved to: results/output_results.json
2026-05-15 11:04:30,212 - INFO - Results saved to: results/output_results.json

============================================================
INFERENCE SUMMARY
============================================================
Model: /opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice
Text: Hello, this is a test of OmniVoice text to speech on Ascend NPU.
Output: results/output.wav
Audio Duration: 3.96s
Inference Time: 1.06s
RTF: 0.2688
Device: npu
Dtype: float16
Sample Rate: 24000
============================================================

logs/accuracy_eval.log

  • 文件大小:1539 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
2026-05-15 13:11:09 [INFO] ============================================================
2026-05-15 13:11:09 [INFO] OmniVoice Accuracy Evaluation - Quick Mode
2026-05-15 13:11:09 [INFO] ============================================================
2026-05-15 13:11:12 [INFO] Loading model on NPU...
2026-05-15 13:11:16 [INFO] Model loaded. Sampling rate: 24000
2026-05-15 13:11:16 [INFO] --- Test: english_short ---
2026-05-15 13:11:16 [INFO] Text: Hello, this is a test of OmniVoice text to speech.
2026-05-15 13:11:17 [INFO]   Max rel error: 0.0000%
2026-05-15 13:11:17 [INFO]   Mean rel error: 0.0000%
2026-05-15 13:11:17 [INFO]   Cosine similarity: 1.00000000
2026-05-15 13:11:17 [INFO]   SNR: 185.34 dB
2026-05-15 13:11:17 [INFO]   Pass: True
2026-05-15 13:11:17 [INFO] --- Test: chinese ---
2026-05-15 13:11:17 [INFO] Text: 你好,这是一段中文语音合成测试。
2026-05-15 13:11:17 [INFO]   Max rel error: 0.0000%
2026-05-15 13:11:17 [INFO]   Mean rel error: 0.0000%
2026-05-15 13:11:17 [INFO]   Cosine similarity: 1.00000000
2026-05-15 13:11:17 [INFO]   SNR: 182.39 dB
2026-05-15 13:11:17 [INFO]   Pass: True
2026-05-15 13:11:17 [INFO] ============================================================
2026-05-15 13:11:17 [INFO] Accuracy evaluation PASSED
2026-05-15 13:11:17 [INFO]   english_short: max_err=0.0000%, cos=1.00000000 [PASS]
2026-05-15 13:11:17 [INFO]   chinese: max_err=0.0000%, cos=1.00000000 [PASS]
2026-05-15 13:11:17 [INFO] ============================================================

results/accuracy_eval.json

  • 文件大小:1065 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
{
  "method": "NPU deterministic reproducibility (same seed, same device)",
  "all_pass": true,
  "num_tests": 2,
  "num_passed": 2,
  "test_cases": [
    {
      "name": "english_short",
      "text": "Hello, this is a test of OmniVoice text to speech.",
      "audio_samples": 108000,
      "audio_duration_s": 4.5,
      "max_abs_error": 0.0,
      "mean_abs_error": 0.0,
      "max_rel_error_pct": 0.0,
      "mean_rel_error_pct": 0.0,
      "cosine_similarity": 1.0,
      "snr_db": 185.3388811583839,
      "mse": 0.0,
      "passed": true
    },
    {
      "name": "chinese",
      "text": "你好,这是一段中文语音合成测试。",
      "audio_samples": 113040,
      "audio_duration_s": 4.71,
      "max_abs_error": 0.0,
      "mean_abs_error": 0.0,
      "max_rel_error_pct": 0.0,
      "mean_rel_error_pct": 0.0,
      "cosine_similarity": 1.0000000000000002,
      "snr_db": 182.3941169400822,
      "mse": 0.0,
      "passed": true
    }
  ],
  "device": "npu:0",
  "dtype": "float16",
  "num_step": 4
}

logs/performance_eval.log

  • 文件大小:9390 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
============================================================
OmniVoice Performance Evaluation
============================================================
NPU: Found 2 NPU(s)
npu-smi:
+------------------------------------------------------------------------------------------------+
| npu-smi 25.5.2                   Version: 25.5.2                                               |
+---------------------------+---------------+----------------------------------------------------+
| NPU   Name                | Health        | Power(W)    Temp(C)           Hugepages-Usage(page)|
| Chip  Phy-ID              | Bus-Id        | AICore(%)   Memory-Usage(MB)  HBM-Usage(MB)        |
+===========================+===============+====================================================+
| 6     Ascend910           | OK            | 165.8       45                0    / 0             |
| 0     12                  | 0000:0A:00.0  | 0           0    / 0          3104 / 65536         |
+------------------------------------------------------------------------------------------------+
| 6     Ascend910           | OK            | -           44                0    / 0             |
| 1     13                  | 0000:0B:00.0  | 0           0    / 0          2870 / 65536         |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU     Chip              | Process id    | Process name             | Process memory(MB)      |
+===========================+===============+====================================================+
| No running processes found in NPU 6                                                            |
+===========================+===============+====================================================+

Loading model from: /opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice
Model loaded in 3.41s
Sampling rate: 24000
Creating sample reference audio...

--- Test: short ---
Text: Hello world....
Warmup: 2 runs...
Performance runs: 3...
  Run 1: latency=1.151s, audio=1.560s, RTF=0.7376
  Run 2: latency=1.147s, audio=1.560s, RTF=0.7353
  Run 3: latency=1.077s, audio=1.560s, RTF=0.6904
  Avg latency: 1.125s
  Avg RTF: 0.7211
  Throughput: 10.67 chars/s

--- Test: medium ---
Text: The quick brown fox jumps over the lazy dog. This is a medium length sentence for testing....
Warmup: 2 runs...
Performance runs: 3...
  Run 1: latency=1.134s, audio=4.760s, RTF=0.2382
  Run 2: latency=1.146s, audio=4.760s, RTF=0.2407
  Run 3: latency=1.152s, audio=4.760s, RTF=0.2420
  Avg latency: 1.144s
  Avg RTF: 0.2403
  Throughput: 78.68 chars/s

--- Test: long ---
Text: This is a longer text for performance evaluation. It contains multiple sentences to measure how the ...
Warmup: 2 runs...
Performance runs: 3...
  Run 1: latency=1.183s, audio=13.080s, RTF=0.0904
  Run 2: latency=1.190s, audio=13.080s, RTF=0.0910
  Run 3: latency=1.108s, audio=13.080s, RTF=0.0847
  Avg latency: 1.160s
  Avg RTF: 0.0887
  Throughput: 214.61 chars/s

Results saved to: results/performance_eval.json
104 / 65536         |
+------------------------------------------------------------------------------------------------+
| 6     Ascend910           | OK            | -           44                0    / 0             |
| 1     13                  | 0000:0B:00.0  | 0           0    / 0          2870 / 65536         |
+===========================+===============+====================================================+
+---------------------------+---------------+----------------------------------------------------+
| NPU     Chip              | Process id    | Process name             | Process memory(MB)      |
+===========================+===============+====================================================+
| No running processes found in NPU 6                                                            |
+===========================+===============+====================================================+

Loading model from: /opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice
2026-05-15 11:14:59,852 - INFO - Loading model from: /opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice
/usr/local/python3.11.14/lib/python3.11/site-packages/pydub/utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work
  warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)

Loading weights:   0%|          | 0/313 [00:00<?, ?it/s]
Loading weights:   1%|▏         | 4/313 [00:00<00:25, 11.99it/s]
Loading weights: 100%|██████████| 313/313 [00:00<00:00, 902.47it/s]

Loading weights:   0%|          | 0/527 [00:00<?, ?it/s]
Loading weights:  43%|████▎     | 227/527 [00:00<00:00, 2147.92it/s]
Loading weights:  84%|████████▍ | 442/527 [00:00<00:00, 1906.60it/s]
Loading weights: 100%|██████████| 527/527 [00:00<00:00, 1971.40it/s]
Model loaded in 3.41s
2026-05-15 11:15:06,197 - INFO - Model loaded in 3.41s
Sampling rate: 24000
2026-05-15 11:15:06,197 - INFO - Sampling rate: 24000
Creating sample reference audio...
2026-05-15 11:15:06,197 - INFO - Creating sample reference audio...

--- Test: short ---
2026-05-15 11:15:06,202 - INFO -
--- Test: short ---
Text: Hello world....
2026-05-15 11:15:06,202 - INFO - Text: Hello world....
Warmup: 2 runs...
2026-05-15 11:15:06,203 - INFO - Warmup: 2 runs...
/opt/atomgit/.local/lib/python3.11/site-packages/omnivoice/models/omnivoice.py:1504: UserWarning: Cannot create tensor with interal format while allow_internel_format=False, tensor will be created with base format. (Triggered internally at build/CMakeFiles/torch_npu.dir/compiler_depend.ts:338.)
  u = torch.rand_like(scaled_logits)
Performance runs: 3...
2026-05-15 11:15:08,830 - INFO - Performance runs: 3...
  Run 1: latency=1.151s, audio=1.560s, RTF=0.7376
2026-05-15 11:15:09,981 - INFO -   Run 1: latency=1.151s, audio=1.560s, RTF=0.7376
  Run 2: latency=1.147s, audio=1.560s, RTF=0.7353
2026-05-15 11:15:11,128 - INFO -   Run 2: latency=1.147s, audio=1.560s, RTF=0.7353
  Run 3: latency=1.077s, audio=1.560s, RTF=0.6904
2026-05-15 11:15:12,205 - INFO -   Run 3: latency=1.077s, audio=1.560s, RTF=0.6904
  Avg latency: 1.125s
2026-05-15 11:15:12,211 - INFO -   Avg latency: 1.125s
  Avg RTF: 0.7211
2026-05-15 11:15:12,211 - INFO -   Avg RTF: 0.7211
  Throughput: 10.67 chars/s
2026-05-15 11:15:12,211 - INFO -   Throughput: 10.67 chars/s

--- Test: medium ---
2026-05-15 11:15:12,211 - INFO -
--- Test: medium ---
Text: The quick brown fox jumps over the lazy dog. This is a medium length sentence for testing....
2026-05-15 11:15:12,211 - INFO - Text: The quick brown fox jumps over the lazy dog. This is a medium length sentence for testing....
Warmup: 2 runs...
2026-05-15 11:15:12,211 - INFO - Warmup: 2 runs...
Performance runs: 3...
2026-05-15 11:15:14,358 - INFO - Performance runs: 3...
  Run 1: latency=1.134s, audio=4.760s, RTF=0.2382
2026-05-15 11:15:15,492 - INFO -   Run 1: latency=1.134s, audio=4.760s, RTF=0.2382
  Run 2: latency=1.146s, audio=4.760s, RTF=0.2407
2026-05-15 11:15:16,638 - INFO -   Run 2: latency=1.146s, audio=4.760s, RTF=0.2407
  Run 3: latency=1.152s, audio=4.760s, RTF=0.2420
2026-05-15 11:15:17,790 - INFO -   Run 3: latency=1.152s, audio=4.760s, RTF=0.2420
  Avg latency: 1.144s
2026-05-15 11:15:17,796 - INFO -   Avg latency: 1.144s
  Avg RTF: 0.2403
2026-05-15 11:15:17,796 - INFO -   Avg RTF: 0.2403
  Throughput: 78.68 chars/s
2026-05-15 11:15:17,796 - INFO -   Throughput: 78.68 chars/s

--- Test: long ---
2026-05-15 11:15:17,796 - INFO -
--- Test: long ---
Text: This is a longer text for performance evaluation. It contains multiple sentences to measure how the ...
2026-05-15 11:15:17,796 - INFO - Text: This is a longer text for performance evaluation. It contains multiple sentences to measure how the ...
Warmup: 2 runs...
2026-05-15 11:15:17,796 - INFO - Warmup: 2 runs...
Performance runs: 3...
2026-05-15 11:15:20,200 - INFO - Performance runs: 3...
  Run 1: latency=1.183s, audio=13.080s, RTF=0.0904
2026-05-15 11:15:21,385 - INFO -   Run 1: latency=1.183s, audio=13.080s, RTF=0.0904
  Run 2: latency=1.190s, audio=13.080s, RTF=0.0910
2026-05-15 11:15:22,577 - INFO -   Run 2: latency=1.190s, audio=13.080s, RTF=0.0910
  Run 3: latency=1.108s, audio=13.080s, RTF=0.0847
2026-05-15 11:15:23,687 - INFO -   Run 3: latency=1.108s, audio=13.080s, RTF=0.0847
  Avg latency: 1.160s
2026-05-15 11:15:23,695 - INFO -   Avg latency: 1.160s
  Avg RTF: 0.0887
2026-05-15 11:15:23,695 - INFO -   Avg RTF: 0.0887
  Throughput: 214.61 chars/s
2026-05-15 11:15:23,695 - INFO -   Throughput: 214.61 chars/s

Results saved to: results/performance_eval.json
2026-05-15 11:15:24,198 - INFO -
Results saved to: results/performance_eval.json

============================================================
PERFORMANCE EVALUATION SUMMARY
============================================================
Device: npu
Dtype: float16
Num steps: 32
Model load time: 3.41s
Avg RTF: 0.3500
Min RTF: 0.0887
Avg latency: 1.143s
  [short] RTF=0.7211, latency=1.125s
  [medium] RTF=0.2403, latency=1.144s
  [long] RTF=0.0887, latency=1.160s
NPU memory: {'allocated_mb': 1939.19, 'reserved_mb': 2380.0}
============================================================

results/performance_eval.json

  • 文件大小:3924 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
{
  "model": "/opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice",
  "device": "npu",
  "dtype": "float16",
  "num_step": 32,
  "warmup_runs": 2,
  "num_runs": 3,
  "load_time_s": 3.411,
  "test_cases": [
    {
      "name": "short",
      "text": "Hello world.",
      "text_length": 12,
      "avg_latency_s": 1.125,
      "std_latency_s": 0.034,
      "min_latency_s": 1.077,
      "max_latency_s": 1.151,
      "avg_audio_duration_s": 1.56,
      "avg_rtf": 0.7211,
      "min_rtf": 0.6904,
      "max_rtf": 0.7376,
      "throughput_chars_per_s": 10.67,
      "npu_memory": {
        "allocated_mb": 1939.19,
        "reserved_mb": 2124.0
      },
      "num_runs": 3
    },
    {
      "name": "medium",
      "text": "The quick brown fox jumps over the lazy dog. This is a medium length sentence for testing.",
      "text_length": 90,
      "avg_latency_s": 1.144,
      "std_latency_s": 0.008,
      "min_latency_s": 1.134,
      "max_latency_s": 1.152,
      "avg_audio_duration_s": 4.76,
      "avg_rtf": 0.2403,
      "min_rtf": 0.2382,
      "max_rtf": 0.242,
      "throughput_chars_per_s": 78.68,
      "npu_memory": {
        "allocated_mb": 1939.19,
        "reserved_mb": 2146.0
      },
      "num_runs": 3
    },
    {
      "name": "long",
      "text": "This is a longer text for performance evaluation. It contains multiple sentences to measure how the model handles longer inputs. The performance should be measured in terms of real-time factor, which is the ratio of audio duration to inference time.",
      "text_length": 249,
      "avg_latency_s": 1.16,
      "std_latency_s": 0.037,
      "min_latency_s": 1.108,
      "max_latency_s": 1.19,
      "avg_audio_duration_s": 13.08,
      "avg_rtf": 0.0887,
      "min_rtf": 0.0847,
      "max_rtf": 0.091,
      "throughput_chars_per_s": 214.61,
      "npu_memory": {
        "allocated_mb": 1939.19,
        "reserved_mb": 2380.0
      },
      "num_runs": 3
    }
  ],
  "summary": {
    "avg_rtf": 0.35,
    "min_rtf": 0.0887,
    "avg_latency_s": 1.143,
    "npu_memory": {
      "allocated_mb": 1939.19,
      "reserved_mb": 2380.0
    },
    "npu_smi": "+------------------------------------------------------------------------------------------------+\n| npu-smi 25.5.2                   Version: 25.5.2                                               |\n+---------------------------+---------------+----------------------------------------------------+\n| NPU   Name                | Health        | Power(W)    Temp(C)           Hugepages-Usage(page)|\n| Chip  Phy-ID              | Bus-Id        | AICore(%)   Memory-Usage(MB)  HBM-Usage(MB)        |\n+===========================+===============+====================================================+\n| 6     Ascend910           | OK            | 180.6       46                0    / 0             |\n| 0     12                  | 0000:0A:00.0  | 0           0    / 0          5779 / 65536         |\n+------------------------------------------------------------------------------------------------+\n| 6     Ascend910           | OK            | -           44                0    / 0             |\n| 1     13                  | 0000:0B:00.0  | 0           0    / 0          2870 / 65536         |\n+===========================+===============+====================================================+\n+---------------------------+---------------+----------------------------------------------------+\n| NPU     Chip              | Process id    | Process name             | Process memory(MB)      |\n+===========================+===============+====================================================+\n| 6       0                 | 104942        | python3                  | 2735                    |\n+===========================+===============+====================================================+\n"
  }
}

results/output_results.json

  • 文件大小:2274 bytes
  • 以下内容为 README 直接文本转写,不是外部路径引用。
{
  "model": "/opt/atomgit/models/modelscope_cache/k2-fsa/OmniVoice",
  "text": "Hello, this is a test of OmniVoice text to speech on Ascend NPU.",
  "reference_audio": null,
  "reference_text": null,
  "output_wav": "results/output.wav",
  "audio_duration_s": 3.96,
  "inference_time_s": 1.064,
  "rtf": 0.2688,
  "sample_rate": 24000,
  "device": "npu",
  "dtype": "float16",
  "num_step": 32,
  "guidance_scale": 2.0,
  "npu_info": {
    "device": "npu",
    "device_count": 2,
    "current_device": 0,
    "npu_smi": "+------------------------------------------------------------------------------------------------+\n| npu-smi 25.5.2                   Version: 25.5.2                                               |\n+---------------------------+---------------+----------------------------------------------------+\n| NPU   Name                | Health        | Power(W)    Temp(C)           Hugepages-Usage(page)|\n| Chip  Phy-ID              | Bus-Id        | AICore(%)   Memory-Usage(MB)  HBM-Usage(MB)        |\n+===========================+===============+====================================================+\n| 6     Ascend910           | OK            | 166.8       46                0    / 0             |\n| 0     12                  | 0000:0A:00.0  | 0           0    / 0          3104 / 65536         |\n+------------------------------------------------------------------------------------------------+\n| 6     Ascend910           | OK            | -           45                0    / 0             |\n| 1     13                  | 0000:0B:00.0  | 0           0    / 0          2870 / 65536         |\n+===========================+===============+====================================================+\n+---------------------------+---------------+----------------------------------------------------+\n| NPU     Chip              | Process id    | Process name             | Process memory(MB)      |\n+===========================+===============+====================================================+\n| No running processes found in NPU 6                                                            |\n+===========================+===============+====================================================+\n"
  },
  "load_time_s": 9.257
}

8. 许可证与声明

  • 适配代码许可证以本仓库 license 元数据或 LICENSE 文件为准。
  • 原始模型权重许可证以模型发布方为准。
  • 本仓库不应提交私钥、token、API key、缓存目录或大体积权重文件。
  • 文档中的运行结果来自仓库现有日志和 JSON 结果文件;未验证的数值不会在 README 中虚构。