Erlangshen-ZEN1-224M-Chinese

主页：封神榜
代码库：Fengshenbang-LM

简介 Brief Introduction

这是一款擅长处理自然语言理解（NLU）任务的ZEN1模型，它采用N-gram编码技术增强文本语义理解，参数量为2.24亿。

ZEN1 model, which uses N-gram to enhance text semantic and has 224M parameters, is adept at NLU tasks.

模型分类 Model Taxonomy

需求 Demand	任务 Task	系列 Series	模型 Model	参数 Parameter	额外 Extra
通用 General	自然语言理解 NLU	二郎神 Erlangshen	ZEN1	224M	中文-Chinese

模型信息 Model Information

我们与ZEN团队合作，基于封神框架开源发布了ZEN1模型。具体来说，ZEN通过整合无监督学习提取的知识，借助N-gram方法学习不同层级的文本粒度信息。ZEN1仅在单一小型语料库（低资源场景）上训练即可获得显著的性能提升。接下来，我们将继续与ZEN团队携手，探索预训练语言模型（PLM）的优化方向，以提升下游任务的性能表现。

We open source and publicly release ZEN1 using our Fengshen Framework in collaboration with the ZEN team. More precisely, by bringing together knowledge extracted by unsupervised learning, ZEN learns different textual granularity information through N-gram methods. ZEN1 can obtain good performance gains by training only on a single small corpus (low-resource scenarios). In the next step, we continue with the ZEN team to explore the optimization of PLM and improve the performance on downstream tasks.

下游效果 Performance

分类任务 Classification

model	dataset	准确率 Acc
IDEA-CCNL/Erlangshen-ZEN1-224M-Chinese	Tnews	56.82%

抽取任务 Extraction

model	dataset	F1值 F1
IDEA-CCNL/Erlangshen-ZEN1-224M-Chinese	OntoNote4.0	80.8%

使用 Usage

由于transformers库中未包含ZEN1的相关模型结构，因此您可以在我们的Fengshenbang-LM代码库中找到该模型结构并运行相关代码。

Since there is no structure of ZEN1 in transformers library, you can find the structure of ZEN1 and run the codes in Fengshenbang-LM.

 git clone https://github.com/IDEA-CCNL/Fengshenbang-LM.git

from fengshen.models.zen1.ngram_utils import ZenNgramDict
from fengshen.models.zen1.tokenization import BertTokenizer
from fengshen.models.zen1.modeling import ZenForSequenceClassification, ZenForTokenClassification

pretrain_path = 'IDEA-CCNL/Erlangshen-ZEN1-224M-Chinese'

tokenizer = BertTokenizer.from_pretrained(pretrain_path)
model_classification = ZenForSequenceClassification.from_pretrained(pretrain_path)
model_extraction = ZenForTokenClassification.from_pretrained(pretrain_path)
ngram_dict = ZenNgramDict.from_pretrained(pretrain_path, tokenizer=tokenizer)

你可以从下方的链接获得我们做分类和抽取的详细示例。

You can get classification and extraction examples below.

分类 classification example on fengshen

抽取 extraction example on fengshen

引用 Citation

如果您在您的工作中使用了我们的模型，可以引用我们的对该模型的论文：

If you are using the resource for your work, please cite the our paper for this model:

@inproceedings{diao-etal-2020-zen,
    title = "ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations",
    author = "Diao, Shizhe and Bai, Jiaxin and Song, Yan and Zhang, Tong and Wang, Yonggang",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
    month = nov,
    year = "2020",
    address = "Online",
    pages = "4729--4740",
}

如果您在您的工作中使用了我们的模型，也可以引用我们的总论文：

如果您将本资源用于您的工作，请引用我们的总论文：

@article{fengshenbang,
  author    = {Jiaxing Zhang and Ruyi Gan and Junjie Wang and Yuxiang Zhang and Lin Zhang and Ping Yang and Xinyu Gao and Ziwei Wu and Xiaoqun Dong and Junqing He and Jianheng Zhuo and Qi Yang and Yongfeng Huang and Xiayu Li and Yanghan Wu and Junyu Lu and Xinyu Zhu and Weifeng Chen and Ting Han and Kunhao Pan and Rui Wang and Hao Wang and Xiaojun Wu and Zhongshen Zeng and Chongpei Chen},
  title     = {Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence},
  journal   = {CoRR},
  volume    = {abs/2209.02970},
  year      = {2022}
}

也可以引用我们的网站：

您也可以引用我们的网站：

@misc{Fengshenbang-LM,
  title={Fengshenbang-LM},
  author={IDEA-CCNL},
  year={2021},
  howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
}