MiniLM-evidence-types

本模型是在证据类型数据集上对 microsoft/MiniLM-L12-H384-uncased 进行微调得到的版本。它在评估集上取得了以下结果：

损失：1.8672
宏 F1 值：0.3726
加权 F1 值：0.7030
准确率：0.7161
平衡准确率：0.3616

训练和评估数据

该数据集以及用于微调此模型的代码可在 GitHub 仓库 BA-Thesis-Information-Science-Persuasion-Strategies 中找到。

使用 HuggingFace Transformers

from openmind import AutoModelForCausalLM,AutoTokenizer, AutoModel, pipeline,is_torch_npu_available
from openmind_hub import snapshot_download
import torch
import argparse
import torch.nn.functional as F


def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_name_or_path",
        type=str,
        help="Path to model",
        default="zhouhui/MiniLM-evidence-types",
    )
    args = parser.parse_args()
    return args

def main():
    args = parse_args()
    model_path = args.model_name_or_path

    if is_torch_npu_available():
        device = "npu:0"
    else:
        device = "cpu"
        
    
    pipe = pipeline("sentiment-analysis", model=model_path, framework="pt",device=device)

    sentence_vecs = pipe("Rhonda has been volunteering for several years for a variety of charitable community programs.")
    print(sentence_vecs)


if __name__ == "__main__":
    main()

训练超参数

训练过程中使用了以下超参数：

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam，betas=(0.9,0.999)，epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 20
mixed_precision_training: Native AMP

训练结果

训练损失	轮次	步数	验证损失	宏F1值	加权F1值	准确率	平衡准确率
1.4106	1.0	250	1.2698	0.1966	0.6084	0.6735	0.2195
1.1437	2.0	500	1.0985	0.3484	0.6914	0.7116	0.3536
0.9714	3.0	750	1.0901	0.2606	0.6413	0.6446	0.2932
0.8382	4.0	1000	1.0197	0.2764	0.7024	0.7237	0.2783
0.7192	5.0	1250	1.0895	0.2847	0.6824	0.6963	0.2915
0.6249	6.0	1500	1.1296	0.3487	0.6888	0.6948	0.3377
0.5336	7.0	1750	1.1515	0.3591	0.6982	0.7024	0.3496
0.4694	8.0	2000	1.1962	0.3626	0.7185	0.7314	0.3415
0.4058	9.0	2250	1.3313	0.3121	0.6920	0.7085	0.3033
0.3746	10.0	2500	1.3993	0.3628	0.6976	0.7047	0.3495
0.3267	11.0	2750	1.5078	0.3560	0.6958	0.7055	0.3464
0.2939	12.0	3000	1.5875	0.3685	0.6968	0.7062	0.3514
0.2677	13.0	3250	1.6470	0.3606	0.6976	0.7070	0.3490
0.2425	14.0	3500	1.7164	0.3714	0.7069	0.7207	0.3551
0.2301	15.0	3750	1.8151	0.3597	0.6975	0.7123	0.3466
0.2268	16.0	4000	1.7838	0.3940	0.7034	0.7123	0.3869
0.201	17.0	4250	1.8328	0.3725	0.6964	0.7062	0.3704
0.1923	18.0	4500	1.8788	0.3708	0.7019	0.7154	0.3591
0.1795	19.0	4750	1.8574	0.3752	0.7031	0.7161	0.3619
0.1713	20.0	5000	1.8672	0.3726	0.7030	0.7161	0.3616

框架版本

Transformers 4.19.2
Pytorch 1.11.0+cu113
Datasets 2.2.2
Tokenizers 0.12.1

训练和评估数据

该数据集以及用于微调此模型的代码可在 GitHub 仓库 BA-Thesis-Information-Science-Persuasion-Strategies 中找到。

使用 HuggingFace Transformers

from openmind import AutoModelForCausalLM,AutoTokenizer, AutoModel, pipeline,is_torch_npu_available
from openmind_hub import snapshot_download
import torch
import argparse
import torch.nn.functional as F


def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_name_or_path",
        type=str,
        help="Path to model",
        default="zhouhui/MiniLM-evidence-types",
    )
    args = parser.parse_args()
    return args

def main():
    args = parse_args()
    model_path = args.model_name_or_path

    if is_torch_npu_available():
        device = "npu:0"
    else:
        device = "cpu"
        
    
    pipe = pipeline("sentiment-analysis", model=model_path, framework="pt",device=device)

    sentence_vecs = pipe("Rhonda has been volunteering for several years for a variety of charitable community programs.")
    print(sentence_vecs)


if __name__ == "__main__":
    main()

训练超参数

训练过程中使用了以下超参数：

learning_rate: 2e-05

train_batch_size: 16

eval_batch_size: 16

seed: 42

optimizer: Adam，betas=(0.9,0.999)，epsilon=1e-08

lr_scheduler_type: linear

num_epochs: 20

mixed_precision_training: Native AMP

训练结果

训练损失	轮次	步数	验证损失	宏F1值	加权F1值	准确率	平衡准确率
1.4106	1.0	250	1.2698	0.1966	0.6084	0.6735	0.2195
1.1437	2.0	500	1.0985	0.3484	0.6914	0.7116	0.3536
0.9714	3.0	750	1.0901	0.2606	0.6413	0.6446	0.2932
0.8382	4.0	1000	1.0197	0.2764	0.7024	0.7237	0.2783
0.7192	5.0	1250	1.0895	0.2847	0.6824	0.6963	0.2915
0.6249	6.0	1500	1.1296	0.3487	0.6888	0.6948	0.3377
0.5336	7.0	1750	1.1515	0.3591	0.6982	0.7024	0.3496
0.4694	8.0	2000	1.1962	0.3626	0.7185	0.7314	0.3415
0.4058	9.0	2250	1.3313	0.3121	0.6920	0.7085	0.3033
0.3746	10.0	2500	1.3993	0.3628	0.6976	0.7047	0.3495
0.3267	11.0	2750	1.5078	0.3560	0.6958	0.7055	0.3464
0.2939	12.0	3000	1.5875	0.3685	0.6968	0.7062	0.3514
0.2677	13.0	3250	1.6470	0.3606	0.6976	0.7070	0.3490
0.2425	14.0	3500	1.7164	0.3714	0.7069	0.7207	0.3551
0.2301	15.0	3750	1.8151	0.3597	0.6975	0.7123	0.3466
0.2268	16.0	4000	1.7838	0.3940	0.7034	0.7123	0.3869
0.201	17.0	4250	1.8328	0.3725	0.6964	0.7062	0.3704
0.1923	18.0	4500	1.8788	0.3708	0.7019	0.7154	0.3591
0.1795	19.0	4750	1.8574	0.3752	0.7031	0.7161	0.3619
0.1713	20.0	5000	1.8672	0.3726	0.7030	0.7161	0.3616

框架版本

Transformers 4.19.2

Pytorch 1.11.0+cu113

Datasets 2.2.2

Tokenizers 0.12.1