HuggingFace镜像/Llama2-Chinese-13b-Chat-ms
Model Introduction
Files & versions
Analysis

Llama2


Llama2中文微调参数

由于Llama2本身的中文对齐较弱,我们采用中文指令集,对meta-llama/Llama-2-13b-chat-hf进行LoRA微调,使其具备较强的中文对话能力。

🎯 该版本为LoRA中文微调参数FlagAlpha/Llama2-Chinese-13b-Chat-LoRA和meta-llama/Llama-2-13b-chat-hf参数结合后的版本,可直接使用


示例代码

Use with openMind

environment variable

# source environment variable
source /usr/local/Ascend/ascend-toolkit/set_env.sh
export OPENMIND_FRAMEWORK=pt

pip install openMind Library

OpenMind Library can be installed through pip, please select the corresponding command according to the actual environment for installation.

It should be noted that since torch npu depends on torch, which can be directly installed through pip in aarch64, but requires a specific URL to download the CPU version in x86, the installation commands in the two environments are different. The specific installation code has been distinguished and presented in the following text.

# aarch64
pip install openmind[all]
# x86
pip install openmind[all] --extra-index-url https://download.pytorch.org/whl/cpu

Inference

from openmind import AutoTokenizer, AutoModelForCausalLM
import torch
import torch_npu

model_dir = "HangZhou_Ascend/Llama2-Chinese-13b-Chat-ms"
tokenizer = AutoTokenizer.from_pretrained(model_dir, device_map="auto", trust_remote_code=True)
# Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and might cause OOM Error.
model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto",  trust_remote_code=True, torch_dtype=torch.float16)
model = model.eval()
response, history = model.chat(tokenizer, "1+1=", history=[], meta_instruction="")
print(response)