panhg/speech_sanm_kws_phone-xiaoyun-commands-offline
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

SANM KWS (Keyword Spotting) - Ascend NPU Adaptation

Model Overview

This is the NPU-adapted version of iic/speech_sanm_kws_phone-xiaoyun-commands-offline, a mobile-grade Chinese keyword spotting model based on the SANM (Self-Attention Network with Memory) architecture from the FunASR project.

The model supports 38 Chinese command keywords across multiple scenarios (wake words, music, navigation, smart home, and general commands) and has been verified to run on Huawei Ascend NPU with accuracy within 1% of CPU reference.

Key Specifications

ItemValue
Model ArchitectureSANM (6-layer encoder + CTC)
Parameters3.28M
Input16kHz mono WAV
Feature40-dim Fbank + LFR (m=7, n=6)
Output2602-token CTC log-probabilities
DecodingCTC greedy decode
Keywords Supported38 (wake words + multi-scene commands)

Supported Keywords

  • Wake words: 小云小云, 你好小云, 你好问问, 嗨小问
  • Music: 播放音乐, 增大音量, 减小音量, 继续播放, 暂停播放, 上一首, 下一首, 单曲循环, 随机模式, 列表循环
  • Navigation: 取消导航, 退出导航, 放大地图, 查看全程, 缩小地图, 不走高速, 躲避拥堵, 避免收费, 高速优先
  • Smart Home: 返回桌面, 睡眠模式, 蓝牙模式, 打开灯光, 关闭灯光, 打开空调, 关闭空调, 拍照拍照, 我要拍照
  • General: 上一页, 下一页, 上一个, 下一个, 换一批, 打开录音, 关闭录音

Ascend NPU Adaptation

Environment

ComponentVersion
CANN8.5.1
PyTorch2.9.0+cpu
torch_npu2.9.0.post1
FunASR1.3.1
Python3.11
NPU DeviceAscend910_9362 (Atlas 800 A2)

Adaptation Strategy

The model is a standard PyTorch model (FunASR SanmKWS). NPU adaptation leverages torch_npu to move compute to the Ascend device:

  1. Model Loading: Load the SANM KWS model via FunASR AutoModel on CPU
  2. NPU Migration: Call model.to("npu:0") to move all parameters and buffers to Ascend NPU
  3. Feature Extraction: Audio frontend (Fbank + LFR + CMVN) runs on CPU
  4. Inference: Encoder + CTC classifier execute on NPU with torch.npu.synchronize() for timing
  5. Decoding: CTC greedy decode runs on CPU after transferring log-probabilities back

No custom operators or graph modifications were required — the model runs directly on Ascend NPU via torch_npu's PyTorch operator mapping.


Accuracy Verification

Methodology

CPU reference inference is compared against NPU inference using the following metrics:

  • Decode Text Match: Whether CTC greedy decoded text is identical
  • Keyword Detection Match: Whether keyword presence detection agrees
  • Cosine Similarity: Directional similarity of output tensors
  • Angular Error: 1 - cosine_similarity as percentage (<0.01% threshold)
  • Mean Relative Error: Per-element relative difference

Results (Test Audio: "小云小云" wake word)

MetricValueStatus
Decode Text CPU你好我小云小云-
Decode Text NPU你好我小云小云-
Decode Text MatchTruePASS
Keyword MatchTrue (小云小云)PASS
CTC Cosine Similarity0.99998474PASS
CTC Angular Error0.0015%PASS (<0.01%)
Encoder Cosine Similarity0.99952179-
Encoder Angular Error0.0478%-

Overall Accuracy: PASS (CTC angular error 0.0015% < 0.01% threshold)


Performance Benchmark

Benchmark configuration: 5 warmup runs + 100 timing runs, measuring encoder+CTC inference.

MetricCPUNPUSpeedup
Mean Latency17.58 ms6.42 ms2.74x
Std Dev0.17 ms0.05 ms-
Min Latency17.31 ms6.34 ms2.73x
Max Latency18.21 ms6.54 ms2.78x

Quick Start

Installation

# Install dependencies
pip install torch torch_npu funasr modelscope soundfile scipy

# The model will be auto-downloaded from ModelScope on first use

Inference

# Basic NPU inference
python inference.py --audio test_audio.wav --keywords "小云小云" --device npu

# With benchmark
python inference.py --audio test_audio.wav --keywords "小云小云,你好小云" --device npu --benchmark

# CPU mode
python inference.py --audio test_audio.wav --keywords "小云小云" --device cpu

Accuracy Evaluation

# Compare NPU vs CPU accuracy + benchmark
python evaluate.py --audio test_audio.wav --keywords "小云小云" --benchmark

Python API

from inference import KWSInference

# Initialize on NPU
engine = KWSInference(keywords="小云小云,播放音乐", device="npu")

# Run inference
result = engine.infer("audio.wav")
print(result["decoded_text"])       # "你好我小云小云"
print(result["keyword_matches"])    # {"小云小云": True, "播放音乐": False}

Files

FileDescription
inference.pyNPU inference script with Python API
evaluate.pyAccuracy evaluation (CPU vs NPU) and benchmark
outputs/evaluation_result.jsonAccuracy comparison results
outputs/benchmark_result.jsonPerformance benchmark results
logs/Runtime log directory
test_audio.wavSample audio ("小云小云" wake word, 4.53s)

Limitations

  • The SANM encoder uses grouped Conv1d (depthwise) and multi-head attention — while these run correctly on Ascend NPU via torch_npu, intermediate activations may exhibit floating-point differences compared to CPU (as seen in the encoder angular error of 0.05%). The CTC output layer compresses these differences to negligible levels (0.0015% angular error).
  • Feature extraction (Fbank + LFR + CMVN) runs on CPU as the kaldi-native-fbank backend does not have an NPU implementation.
  • Tested with single audio files; batch inference throughput is not benchmarked.

Citation

@inproceedings{Gao2020SANMME,
  title={SAN-M: Memory Equipped Self-Attention for End-to-End Speech Recognition},
  author={Zhifu Gao and Shiliang Zhang and Ming Lei and Ian Mcloughlin},
  booktitle={Interspeech},
  year={2020},
}