weixin_72661020/Qwen-7B-Chat-Int4
Model Introduction
Files & versions
Pull Requests
Discussion
Analysis

Qwen-7B-Chat-Int4 Ascend NPU Deployment

Overview

Qwen-7B-Chat-Int4 is a 7 billion parameter conversational model optimized with 4-bit quantization, specifically designed for efficient deployment on Huawei Ascend NPU infrastructure.

Key Features

  • Model Size: 7B parameters
  • Quantization: INT4 (Group Size: 128)
  • Architecture: QWenLMHeadModel
  • Context Length: 32,768 tokens
  • Optimized For: Ascend 910B series NPUs

Model Information

  • Base Model: Qwen/Qwen-7B-Chat
  • Quantization Method: GPTQ
  • Quantization Bits: 4
  • Quantization Group Size: 128
  • Library: Transformers

Deployment Guide

See Qwen-7B-Chat-Int4-detailed.md for comprehensive deployment instructions.

Quick Start

# Clone this repository
git clone https://gitcode.com/weixin_72661020/Qwen-7B-Chat-Int4.git

# Navigate to the project directory  
cd Qwen-7B-Chat-Int4

# Check out the detailed documentation
cat Qwen-7B-Chat-Int4-detailed.md

Files Included

  • README.md - This file (project overview)
  • Qwen-7B-Chat-Int4-detailed.md - Complete deployment guide
  • SKILL.md - Claude Code skill specification
  • inference.py - Inference script with NPU support
  • evaluation.md - Test results and compatibility report
  • model_info.json - Model specifications

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Related Links

  • Hugging Face Hub: Qwen Models Collection
  • Ascend Documentation: Ascend NPU Programming Guide
  • Transformers: Qwen Model Card

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference