HuggingFace镜像/internlm2_5-20b-chat
模型介绍文件和版本分析
下载使用量0

InternLM

 
InternLM HOT
 

evaluation

💻GitHub 代码库 • 🤔问题反馈 • 📜技术报告

👋 欢迎加入我们的 Discord 和 微信 社群

简介

InternLM2.5 已开源 200 亿参数基础模型及面向实际场景优化的对话模型。该模型具备以下特点：

  • 卓越的推理能力：在数学推理任务上表现领先，性能超越 Llama3 和 Gemma2-27B 等模型。

  • 更强的工具调用能力：InternLM2.5 支持从超 100 个网页获取信息，相关实现已在 MindSearch 中发布。在指令遵循、工具选择与反思等工具使用相关能力上，InternLM2.5 均有出色表现。详见 示例。

InternLM2.5-20B-Chat

性能评估

我们使用开源评估工具 OpenCompass 对 InternLM 进行了全面评估。评估涵盖学科能力、语言能力、知识能力、推理能力和理解能力五个维度。以下是部分评估结果，更多详情可访问 OpenCompass 排行榜。

基准测试InternLM2.5-20B-ChatGemma2-27B-IT
MMLU (5-shot)73.575.0
CMMLU (5-shot)79.763.3
BBH (3-shot CoT)76.371.5
MATH (0-shot CoT)64.750.1
GPQA (0-shot)33.329.3
  • 评估结果来源于 OpenCompass，评估配置可参见 OpenCompass 提供的配置文件。
  • 由于 OpenCompass 版本迭代，评估数据可能存在数值差异，请以 OpenCompass 的最新评估结果为准。

局限性：尽管我们在训练过程中已尽力确保模型的安全性，并鼓励模型生成符合伦理和法律要求的文本，但受模型规模及概率生成范式影响，模型仍可能产生意外输出。例如，生成的回复可能包含偏见、歧视或其他有害内容。请勿传播此类内容。对于因传播有害信息造成的任何后果，我们不承担责任。

从 Transformers 导入

要使用 Transformers 加载 InternLM2.5 20B Chat 模型，请使用以下代码：

import torch
from openmind import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("AI-Research/internlm2_5-20b-chat", trust_remote_code=True)
# Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and cause OOM Error.
model = AutoModelForCausalLM.from_pretrained("AI-Research/internlm2_5-20b-chat", torch_dtype=torch.float16, trust_remote_code=True).npu()
model = model.eval()
response, history = model.chat(tokenizer, "hello", history=[])
print(response)
# Hello! How can I help you today?
response, history = model.chat(tokenizer, "please provide three suggestions about time management", history=history)
print(response)

可以使用 stream_chat 来流式输出响应：

import torch
from openmind import AutoModelForCausalLM, AutoTokenizer

model_path = "AI-Research/internlm2_5-20b-chat"
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float16, trust_remote_code=True).npu()
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

model = model.eval()
length = 0
for response, history in model.stream_chat(tokenizer, "Hello", history=[]):
    print(response[length:], flush=True, end="")
    length = len(response)

部署

LMDeploy

LMDeploy 是一款由 MMRazor 和 MMDeploy 团队开发的大语言模型压缩、部署与服务工具包。

pip install lmdeploy

您可以通过以下 Python 代码在本地运行批量推理：

import lmdeploy
pipe = lmdeploy.pipeline("internlm/internlm2_5-20b-chat")
response = pipe(["Hi, pls intro yourself", "Shanghai is"])
print(response)

或者，您可以通过以下命令启动一个与 OpenAI 兼容的服务器：

lmdeploy serve api_server internlm/internlm2_5-20b-chat --model-name internlm2_5-20b-chat --server-port 23333

然后你可以向服务器发送聊天请求：

curl http://localhost:23333/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "internlm2_5-20b-chat",
    "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Introduce deep learning to me."}
    ]
    }'

更多详情请参见 LMDeploy 文档

vLLM

使用 vLLM>=0.3.2 启动兼容 OpenAI 的服务器：

pip install vllm
python -m vllm.entrypoints.openai.api_server --model internlm/internlm2_5-20b-chat --served-model-name internlm2_5-20b-chat --trust-remote-code

然后您可以向服务器发送聊天请求：

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "internlm2_5-20b-chat",
    "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Introduce deep learning to me."}
    ]
    }'

更多详细信息请参见 vLLM 文档

开源许可

代码采用 Apache-2.0 许可协议，模型权重完全开放供学术研究使用，同时允许免费商业用途。如需申请商业许可，请填写 申请表（英文）/申请表（中文）。其他问题或合作事宜，请联系 internlm@pjlab.org.cn。

引用

@misc{cai2024internlm2,
      title={InternLM2 Technical Report},
      author={Zheng Cai and Maosong Cao and Haojiong Chen and Kai Chen and Keyu Chen and Xin Chen and Xun Chen and Zehui Chen and Zhi Chen and Pei Chu and Xiaoyi Dong and Haodong Duan and Qi Fan and Zhaoye Fei and Yang Gao and Jiaye Ge and Chenya Gu and Yuzhe Gu and Tao Gui and Aijia Guo and Qipeng Guo and Conghui He and Yingfan Hu and Ting Huang and Tao Jiang and Penglong Jiao and Zhenjiang Jin and Zhikai Lei and Jiaxing Li and Jingwen Li and Linyang Li and Shuaibin Li and Wei Li and Yining Li and Hongwei Liu and Jiangning Liu and Jiawei Hong and Kaiwen Liu and Kuikun Liu and Xiaoran Liu and Chengqi Lv and Haijun Lv and Kai Lv and Li Ma and Runyuan Ma and Zerun Ma and Wenchang Ning and Linke Ouyang and Jiantao Qiu and Yuan Qu and Fukai Shang and Yunfan Shao and Demin Song and Zifan Song and Zhihao Sui and Peng Sun and Yu Sun and Huanze Tang and Bin Wang and Guoteng Wang and Jiaqi Wang and Jiayu Wang and Rui Wang and Yudong Wang and Ziyi Wang and Xingjian Wei and Qizhen Weng and Fan Wu and Yingtong Xiong and Chao Xu and Ruiliang Xu and Hang Yan and Yirong Yan and Xiaogui Yang and Haochen Ye and Huaiyuan Ying and Jia Yu and Jing Yu and Yuhang Zang and Chuyu Zhang and Li Zhang and Pan Zhang and Peng Zhang and Ruijie Zhang and Shuo Zhang and Songyang Zhang and Wenjian Zhang and Wenwei Zhang and Xingcheng Zhang and Xinyue Zhang and Hui Zhao and Qian Zhao and Xiaomeng Zhao and Fengzhe Zhou and Zaida Zhou and Jingming Zhuo and Yicheng Zou and Xipeng Qiu and Yu Qiao and Dahua Lin},
      year={2024},
      eprint={2403.17297},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

简介

InternLM2.5，即书生·浦语大模型第2.5代，开源了面向实用场景的200亿参数基础模型与对话模型（InternLM2.5-20B-Chat）。模型具有以下特点：

  • 卓越的推理性能：在数学推理方面取得了同量级模型最优精度，超越了Llama3和Gemma2-27B。
  • 工具调用能力整体升级：InternLM2.5支持从上百个网页搜集有效信息进行分析推理，相关实现已开源到MindSearch。InternLM2.5具有更强和更具有泛化性的指令理解、工具筛选与结果反思等能力，新版模型可以更可靠地支持复杂智能体的搭建，支持对工具进行有效的多轮调用，完成较复杂的任务。可以查看更多样例。

InternLM2.5-20B-Chat

性能评测

我们使用开源评测工具OpenCompass从学科综合能力、语言能力、知识能力、推理能力、理解能力五大能力维度对InternLM开展全面评测，部分评测结果如下表所示，欢迎访问OpenCompass榜单获取更多的评测结果。

BenchmarkInternLM2.5-20B-ChatGemma2-27B-IT
MMLU (5-shot)73.575.0
CMMLU (5-shot)79.763.3
BBH (3-shot CoT)76.371.5
MATH (0-shot CoT)64.750.1
GPQA (0-shot)33.329.3
  • 以上评测结果基于OpenCompass获得，具体测试细节可参见OpenCompass中提供的配置文件。
  • 评测数据会因OpenCompass的版本迭代而存在数值差异，请以OpenCompass最新版的评测结果为主。

局限性： 尽管在训练过程中我们非常注重模型的安全性，尽力促使模型输出符合伦理和法律要求的文本，但受限于模型大小以及概率生成范式，模型可能会产生各种不符合预期的输出，例如回复内容包含偏见、歧视等有害内容，请勿传播这些内容。由于传播不良信息导致的任何后果，本项目不承担责任。

通过Transformers加载

通过以下的代码加载InternLM2.5 20B Chat模型

import torch
from openmind import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("AI-Research/internlm2_5-20b-chat", trust_remote_code=True)
# `torch_dtype=torch.float16` 可以令模型以 float16 精度加载，否则 transformers 会将模型加载为 float32，导致显存不足
model = AutoModelForCausalLM.from_pretrained("AI-Research/internlm2_5-20b-chat", torch_dtype=torch.float16, trust_remote_code=True).npu()
model = model.eval()
response, history = model.chat(tokenizer, "你好", history=[])
print(response)
# 你好！有什么我可以帮助你的吗？
response, history = model.chat(tokenizer, "请提供三个管理时间的建议。", history=history)
print(response)

如果想进行流式生成，则可以使用 stream_chat 接口：

import torch
from openmind import AutoModelForCausalLM, AutoTokenizer

model_path = "internlm/internlm2_5-20b-chat"
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dype=torch.float16, trust_remote_code=True).npu()
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

model = model.eval()
length = 0
for response, history in model.stream_chat(tokenizer, "你好", history=[]):
    print(response[length:], flush=True, end="")
    length = len(response)

部署

LMDeploy

LMDeploy 由 MMDeploy 和 MMRazor 团队联合开发，是涵盖了 LLM 任务的全套轻量化、部署和服务解决方案。

pip install lmdeploy

你可以使用以下 Python 代码进行本地批量推理:

import lmdeploy
pipe = lmdeploy.pipeline("internlm/internlm2_5-20b-chat")
response = pipe(["Hi, pls intro yourself", "Shanghai is"])
print(response)

或者你可以使用以下命令启动兼容 OpenAI API 的服务:

lmdeploy serve api_server internlm/internlm2_5-20b-chat --model-name internlm2_5-20b-chat --server-port 23333

然后你可以向服务端发起一个聊天请求:

curl http://localhost:23333/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "internlm2_5-20b-chat",
    "messages": [
    {"role": "system", "content": "你是个友善的AI助手。"},
    {"role": "user", "content": "介绍一下深度学习。"}
    ]
    }'

更多信息请查看 LMDeploy 文档

vLLM

使用vLLM>=0.3.2启动兼容 OpenAI API 的服务:

pip install vllm
python -m vllm.entrypoints.openai.api_server --model internlm/internlm2_5-20b-chat --served-model-name internlm2_5-20b-chat --trust-remote-code

然后你可以向服务端发起一个聊天请求:

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "internlm2_5-20b-chat",
    "messages": [
    {"role": "system", "content": "你是个友善的AI助手。"},
    {"role": "user", "content": "介绍一下深度学习。"}
    ]
    }'

更多信息请查看 vLLM 文档

开源许可证

本仓库的代码依照 Apache-2.0 协议开源。模型权重对学术研究完全开放，也可申请免费的商业使用授权（申请表）。其他问题与合作请联系 internlm@pjlab.org.cn。

引用

@misc{cai2024internlm2,
      title={InternLM2 Technical Report},
      author={Zheng Cai and Maosong Cao and Haojiong Chen and Kai Chen and Keyu Chen and Xin Chen and Xun Chen and Zehui Chen and Zhi Chen and Pei Chu and Xiaoyi Dong and Haodong Duan and Qi Fan and Zhaoye Fei and Yang Gao and Jiaye Ge and Chenya Gu and Yuzhe Gu and Tao Gui and Aijia Guo and Qipeng Guo and Conghui He and Yingfan Hu and Ting Huang and Tao Jiang and Penglong Jiao and Zhenjiang Jin and Zhikai Lei and Jiaxing Li and Jingwen Li and Linyang Li and Shuaibin Li and Wei Li and Yining Li and Hongwei Liu and Jiangning Liu and Jiawei Hong and Kaiwen Liu and Kuikun Liu and Xiaoran Liu and Chengqi Lv and Haijun Lv and Kai Lv and Li Ma and Runyuan Ma and Zerun Ma and Wenchang Ning and Linke Ouyang and Jiantao Qiu and Yuan Qu and Fukai Shang and Yunfan Shao and Demin Song and Zifan Song and Zhihao Sui and Peng Sun and Yu Sun and Huanze Tang and Bin Wang and Guoteng Wang and Jiaqi Wang and Jiayu Wang and Rui Wang and Yudong Wang and Ziyi Wang and Xingjian Wei and Qizhen Weng and Fan Wu and Yingtong Xiong and Chao Xu and Ruiliang Xu and Hang Yan and Yirong Yan and Xiaogui Yang and Haochen Ye and Huaiyuan Ying and Jia Yu and Jing Yu and Yuhang Zang and Chuyu Zhang and Li Zhang and Pan Zhang and Peng Zhang and Ruijie Zhang and Shuo Zhang and Songyang Zhang and Wenjian Zhang and Wenwei Zhang and Xingcheng Zhang and Xinyue Zhang and Hui Zhao and Qian Zhao and Xiaomeng Zhao and Fengzhe Zhou and Zaida Zhou and Jingming Zhuo and Yicheng Zou and Xipeng Qiu and Yu Qiao and Dahua Lin},
      year={2024},
      eprint={2403.17297},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}