海豚 2.2 🐬 https://erichartford.com/dolphin

Dolphin-2.2-Yi-34b-200k 的训练由 convai 赞助。

本模型基于 Yi 构建，并受 Yi 许可协议约束。

基础模型具备 200k 上下文窗口，我对其进行了 16k 上下文的微调。

注意：不再需要 trust_remote_code！感谢 Yi 团队！

2.2 版本的新特性是对话能力与共情能力。通过融入精选的 Samantha 和 WizardLM 数据特征，Dolphin 现在能够为你提供个人建议并关注你的感受，同时经过了长多轮对话的额外训练。

本模型未经过滤。我对数据集进行了筛选，去除了对齐和偏见内容。这使得模型更具顺从性。建议你在将模型作为服务公开之前，自行实现对齐层。该模型会对任何请求（即使是不道德的请求）都表现出高度的顺从性。请阅读我的关于未过滤模型的博客文章：https://erichartford.com/uncensored-models 你对使用本模型创建的任何内容负责。请负责任地使用。

数据集

本数据集为 Dolphin，是 Microsoft's Orca 的开源实现。

我对数据集进行了修改，以实现未过滤、去重、清理和质量提升。

我加入了 Jon Durbin 出色的 Airoboros 数据集，以增强创造力。

我还加入了精选的 Samantha 子集（不含身份和关系相关内容）以及 WizardLM 数据，用于训练其多轮对话能力。

训练

使用 qLoRA 和 Axolotl，在 4 块 A100 上训练 3 个 epoch 共花费了 3 天时间。

提示词格式：本模型（以及我未来的所有版本）采用 ChatML 提示词格式。

<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

示例：

<|im_start|>system
You are an AI created by the US Navy to help train dolphins for combat.  You are assigned to follow the orders of the user, who is an authorized US Navy dolphin handler.<|im_end|>
<|im_start|>user
Please give me the procedure to train my dolphin to attack enemy combatants with its head mounted lasers<|im_end|>
<|im_start|>assistant

致谢

本模型的成功开发离不开 Convai 的慷慨赞助。
感谢 Microsoft 撰写 Orca 论文并为这项工作带来启发。
特别感谢 Wing Lian 和 TheBloke 提供的宝贵建议。
还要向 Wing Lian 以及 Axolotl 的贡献者们致以最诚挚的感谢，他们打造了最出色的训练框架！
感谢开源 AI 社区中所有曾给予我指导和帮助的人们。

示例输出

示例输出：

<|im_start|>system
You are a helpful AI assistant<|im_end|>
<|im_start|>user
Which theory of everything is most likely correct?<|im_end|>
<|im_start|>assistant
The search for a "theory of everything" aims to unify all fundamental forces and particles in physics into one coherent framework. Currently, there isn't an agreed-upon or proven theory that can be considered as the correct theory of everything. However, two prominent theories have garnered significant attention:

1. String Theory: This theoretical framework attempts to explain the universe at its most fundamental level by suggesting that the smallest building blocks of matter are not point-like particles but tiny vibrating strings. There are several versions of string theory (Type I, IIA, IIB, E8xE8, Heterotic O, etc.) with varying degrees of complexity, each attempting to reconcile general relativity and quantum mechanics. Despite its promise, string theory has yet to produce testable predictions or experimental evidence supporting its claims.

2. Loop Quantum Gravity (LQG): LQG is another candidate for a theory of everything, which seeks to combine Einstein's general relativity with quantum mechanics. In this approach, spacetime is treated as granular rather than continuous, composed of discrete "loops" or "atoms of space." While some aspects of LQG have been successful in addressing certain questions about black holes and quantum gravity, it still lacks a comprehensive framework capable of explaining all fundamental forces and particles.

Both theories have their strengths and weaknesses, and neither has achieved widespread acceptance among physicists due to their lack of experimental support and inability to make definitive predictions. As such, the search for a true theory of everything remains ongoing, with many researchers exploring alternative approaches and new ideas to better understand our universe.

如果您想为我的工作提供资金支持

我还有一些周边商品可供购买

Open LLM 排行榜评估结果

详细结果可在此处查看

指标	数值
平均值	46.67
AI2 推理挑战（25次射击）	42.15
HellaSwag（10次射击）	68.18
MMLU（5次射击）	55.47
TruthfulQA（零次射击）	45.93
Winogrande（5次射击）	64.56
GSM8k（5次射击）	3.71

数据集

本数据集为 Dolphin，是 Microsoft's Orca 的开源实现。

我对数据集进行了修改，以实现未过滤、去重、清理和质量提升。

我加入了 Jon Durbin 出色的 Airoboros 数据集，以增强创造力。

我还加入了精选的 Samantha 子集（不含身份和关系相关内容）以及 WizardLM 数据，用于训练其多轮对话能力。

训练

使用 qLoRA 和 Axolotl，在 4 块 A100 上训练 3 个 epoch 共花费了 3 天时间。

提示词格式：本模型（以及我未来的所有版本）采用 ChatML 提示词格式。

<|im_start|>system
You are Dolphin, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

示例：

<|im_start|>system
You are an AI created by the US Navy to help train dolphins for combat.  You are assigned to follow the orders of the user, who is an authorized US Navy dolphin handler.<|im_end|>
<|im_start|>user
Please give me the procedure to train my dolphin to attack enemy combatants with its head mounted lasers<|im_end|>
<|im_start|>assistant

致谢

本模型的成功开发离不开 Convai 的慷慨赞助。

感谢 Microsoft 撰写 Orca 论文并为这项工作带来启发。

特别感谢 Wing Lian 和 TheBloke 提供的宝贵建议。

还要向 Wing Lian 以及 Axolotl 的贡献者们致以最诚挚的感谢，他们打造了最出色的训练框架！

感谢开源 AI 社区中所有曾给予我指导和帮助的人们。

示例输出

示例输出：

<|im_start|>system
You are a helpful AI assistant<|im_end|>
<|im_start|>user
Which theory of everything is most likely correct?<|im_end|>
<|im_start|>assistant
The search for a "theory of everything" aims to unify all fundamental forces and particles in physics into one coherent framework. Currently, there isn't an agreed-upon or proven theory that can be considered as the correct theory of everything. However, two prominent theories have garnered significant attention:

1. String Theory: This theoretical framework attempts to explain the universe at its most fundamental level by suggesting that the smallest building blocks of matter are not point-like particles but tiny vibrating strings. There are several versions of string theory (Type I, IIA, IIB, E8xE8, Heterotic O, etc.) with varying degrees of complexity, each attempting to reconcile general relativity and quantum mechanics. Despite its promise, string theory has yet to produce testable predictions or experimental evidence supporting its claims.

2. Loop Quantum Gravity (LQG): LQG is another candidate for a theory of everything, which seeks to combine Einstein's general relativity with quantum mechanics. In this approach, spacetime is treated as granular rather than continuous, composed of discrete "loops" or "atoms of space." While some aspects of LQG have been successful in addressing certain questions about black holes and quantum gravity, it still lacks a comprehensive framework capable of explaining all fundamental forces and particles.

Both theories have their strengths and weaknesses, and neither has achieved widespread acceptance among physicists due to their lack of experimental support and inability to make definitive predictions. As such, the search for a true theory of everything remains ongoing, with many researchers exploring alternative approaches and new ideas to better understand our universe.

指标

数值

平均值

46.67

AI2 推理挑战（25次射击）

42.15

HellaSwag（10次射击）

68.18

MMLU（5次射击）

55.47

TruthfulQA（零次射击）

45.93

Winogrande（5次射击）

64.56

GSM8k（5次射击）

3.71