该模型是专为中文打造的DeBERTa-v2版本，拥有1.86亿参数，采用sentence piece分词技术，擅长处理各类自然语言理解（NLU）任务。
Good at solving NLU tasks, adopting sentence piece, Chinese DeBERTa-v2 with 186M parameters.
| 需求 Demand | 任务 Task | 系列 Series | 模型 Model | 参数 Parameter | 额外 Extra |
|---|---|---|---|---|---|
| 通用 General | 自然语言理解 NLU | 二郎神 Erlangshen | DeBERTa-v2 | 186M | 中文-分句 Chinese-SentencePiece |
为了研发这款中文版的DeBERTa-v2（186M参数），我们采用悟道语料库（180G版本）进行预训练。分词方面，我们选用了Sentence Piece方案（词表规模约为128,000）。在预训练过程中，我们借助封神框架，使用8张3090TI（24G）显卡，历时约21天完成训练。
To get a Chinese DeBERTa-v2 (186M), we use WuDao Corpora (180 GB version) for pre-training. We employ the sentence piece as the tokenizer (vocabulary size: around 128,000). Specifically, we use the fengshen framework in the pre-training phase which cost about 21 days with 8 3090TI（24G） GPUs.
以下为模型在各下游任务开发集（dev集）上的表现结果：
We present the results (dev set) on the following tasks:
| Model | OCNLI | CMNLI |
|---|---|---|
| RoBERTa-base | 0.743 | 0.7973 |
| Erlangshen-DeBERTa-v2-186M-Chinese-SentencePiece | 0.7625 | 0.8100 |
from transformers import AutoModelForMaskedLM, AutoTokenizer, FillMaskPipeline
import torch
tokenizer=AutoTokenizer.from_pretrained('IDEA-CCNL/Erlangshen-DeBERTa-v2-186M-Chinese-SentencePiece', use_fast=False)
model=AutoModelForMaskedLM.from_pretrained('IDEA-CCNL/Erlangshen-DeBERTa-v2-186M-Chinese-SentencePiece')
text = '中国首都位于<mask>。'
fillmask_pipe = FillMaskPipeline(model, tokenizer)
print(fillmask_pipe(text, top_k=10))如果您在您的工作中使用了我们的模型，可以引用我们的论文：
如果您将本资源用于您的工作，请引用我们的论文：
@article{fengshenbang,
author = {Jiaxing Zhang and Ruyi Gan and Junjie Wang and Yuxiang Zhang and Lin Zhang and Ping Yang and Xinyu Gao and Ziwei Wu and Xiaoqun Dong and Junqing He and Jianheng Zhuo and Qi Yang and Yongfeng Huang and Xiayu Li and Yanghan Wu and Junyu Lu and Xinyu Zhu and Weifeng Chen and Ting Han and Kunhao Pan and Rui Wang and Hao Wang and Xiaojun Wu and Zhongshen Zeng and Chongpei Chen},
title = {Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence},
journal = {CoRR},
volume = {abs/2209.02970},
year = {2022}
}也可以引用我们的网站：
您也可以引用我们的网站：
@misc{Fengshenbang-LM,
title={Fengshenbang-LM},
author={IDEA-CCNL},
year={2021},
howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
}