This is the NPU-adapted version of iic/speech_sanm_kws_phone-xiaoyun-commands-offline, a mobile-grade Chinese keyword spotting model based on the SANM (Self-Attention Network with Memory) architecture from the FunASR project.
The model supports 38 Chinese command keywords across multiple scenarios (wake words, music, navigation, smart home, and general commands) and has been verified to run on Huawei Ascend NPU with accuracy within 1% of CPU reference.
| Item | Value |
|---|---|
| Model Architecture | SANM (6-layer encoder + CTC) |
| Parameters | 3.28M |
| Input | 16kHz mono WAV |
| Feature | 40-dim Fbank + LFR (m=7, n=6) |
| Output | 2602-token CTC log-probabilities |
| Decoding | CTC greedy decode |
| Keywords Supported | 38 (wake words + multi-scene commands) |
| Component | Version |
|---|---|
| CANN | 8.5.1 |
| PyTorch | 2.9.0+cpu |
| torch_npu | 2.9.0.post1 |
| FunASR | 1.3.1 |
| Python | 3.11 |
| NPU Device | Ascend910_9362 (Atlas 800 A2) |
The model is a standard PyTorch model (FunASR SanmKWS). NPU adaptation leverages torch_npu to move compute to the Ascend device:
AutoModel on CPUmodel.to("npu:0") to move all parameters and buffers to Ascend NPUtorch.npu.synchronize() for timingNo custom operators or graph modifications were required — the model runs directly on Ascend NPU via torch_npu's PyTorch operator mapping.
CPU reference inference is compared against NPU inference using the following metrics:
1 - cosine_similarity as percentage (<0.01% threshold)| Metric | Value | Status |
|---|---|---|
| Decode Text CPU | 你好我小云小云 | - |
| Decode Text NPU | 你好我小云小云 | - |
| Decode Text Match | True | PASS |
| Keyword Match | True (小云小云) | PASS |
| CTC Cosine Similarity | 0.99998474 | PASS |
| CTC Angular Error | 0.0015% | PASS (<0.01%) |
| Encoder Cosine Similarity | 0.99952179 | - |
| Encoder Angular Error | 0.0478% | - |
Benchmark configuration: 5 warmup runs + 100 timing runs, measuring encoder+CTC inference.
| Metric | CPU | NPU | Speedup |
|---|---|---|---|
| Mean Latency | 17.58 ms | 6.42 ms | 2.74x |
| Std Dev | 0.17 ms | 0.05 ms | - |
| Min Latency | 17.31 ms | 6.34 ms | 2.73x |
| Max Latency | 18.21 ms | 6.54 ms | 2.78x |
# Install dependencies
pip install torch torch_npu funasr modelscope soundfile scipy
# The model will be auto-downloaded from ModelScope on first use# Basic NPU inference
python inference.py --audio test_audio.wav --keywords "小云小云" --device npu
# With benchmark
python inference.py --audio test_audio.wav --keywords "小云小云,你好小云" --device npu --benchmark
# CPU mode
python inference.py --audio test_audio.wav --keywords "小云小云" --device cpu# Compare NPU vs CPU accuracy + benchmark
python evaluate.py --audio test_audio.wav --keywords "小云小云" --benchmarkfrom inference import KWSInference
# Initialize on NPU
engine = KWSInference(keywords="小云小云,播放音乐", device="npu")
# Run inference
result = engine.infer("audio.wav")
print(result["decoded_text"]) # "你好我小云小云"
print(result["keyword_matches"]) # {"小云小云": True, "播放音乐": False}| File | Description |
|---|---|
inference.py | NPU inference script with Python API |
evaluate.py | Accuracy evaluation (CPU vs NPU) and benchmark |
outputs/evaluation_result.json | Accuracy comparison results |
outputs/benchmark_result.json | Performance benchmark results |
logs/ | Runtime log directory |
test_audio.wav | Sample audio ("小云小云" wake word, 4.53s) |
@inproceedings{Gao2020SANMME,
title={SAN-M: Memory Equipped Self-Attention for End-to-End Speech Recognition},
author={Zhifu Gao and Shiliang Zhang and Ming Lei and Ian Mcloughlin},
booktitle={Interspeech},
year={2020},
}