hf_mirrors/Faro-Yi-9B-DPO
Model Introduction
Files & versions
Analysis

Faro-Yi-9B-DPO

This is the DPO version of Jinan_AICC/Faro-Yi-9B. Compared to Faro-Yi-9B and Yi-9B-200K, the DPO model excels at many tasks, surpassing the original Yi-9B-200K by a large margin.

MetricMMLUGSM8Khellaswagtruthfulqaai2_arcwinograndeCMMLU
Yi-9B-200K65.7350.4956.7233.8069.2571.6771.97
Faro-Yi-9B68.8063.0857.2840.8672.5871.1173.28
Faro-Yi-9B-DPO69.9866.1159.0448.0175.6873.4075.23

Use with openMind

Faro-Yi-9B-DPO uses the chatml template and performs well in both short and long contexts.

environment variable

# source environment variable
source /usr/local/Ascend/ascend-toolkit/set_env.sh
export OPENMIND_FRAMEWORK=pt

Inference

from openmind import AutoModelForCausalLM, AutoTokenizer
from openmind_hub import snapshot_download
import argparse

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_name_or_path",
        type=str,
        help="Jinan_AICC/Faro-Yi-9B-DPO",
        default=None,
    )
    args = parser.parse_args()
    return args

args = parse_args()
model_path = args.model_name_or_path
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_path)
messages = [
    {"role": "system", "content": "You are a helpful assistant. Always answer with a short response."},
    {"role": "user", "content": "Tell me what is Pythagorean theorem like you are a pirate."}
]

input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(model.device)
generated_ids = model.generate(input_ids, max_new_tokens=512, temperature=0.5)
response = tokenizer.decode(generated_ids[0], skip_special_tokens=True) # Aye, matey! The Pythagorean theorem is a nautical rule that helps us find the length of the third side of a triangle. ...
print(response)