bert-kachakacha:可用于英文文本情感极性分类任务，该项目是 bert-base-uncased 在 amazon_polarity 数据集上的微调模型，评估集准确率达 0.9465，支持 NPU 和 CPU 硬件，基于 PyTorch 框架。【此简介由AI生成】

HuggingFace镜像/bert-kachakacha

训练过程

训练超参数

训练过程中使用了以下超参数：

learning_rate: 5e-05

train_batch_size: 1

eval_batch_size: 8

seed: 42

optimizer: Adam，betas=(0.9,0.999)，epsilon=1e-08

lr_scheduler_type: linear

lr_scheduler_warmup_steps: 1782000

training_steps: 17820000

训练结果

训练损失	轮次	步数	验证损失	准确率
0.7155	0.0	2000	0.7060	0.4622
0.7054	0.0	4000	0.6925	0.5165
0.6842	0.0	6000	0.6653	0.6116
0.6375	0.0	8000	0.5721	0.7909
0.4671	0.0	10000	0.3238	0.8770
0.3403	0.0	12000	0.3692	0.8861
0.4162	0.0	14000	0.4560	0.8908
0.4728	0.0	16000	0.5071	0.8980
0.5111	0.01	18000	0.5204	0.9015
0.4792	0.01	20000	0.5193	0.9076
0.544	0.01	22000	0.4835	0.9133
0.4745	0.01	24000	0.4689	0.9170
0.4403	0.01	26000	0.4778	0.9177
0.4405	0.01	28000	0.4754	0.9163
0.4375	0.01	30000	0.4808	0.9175
0.4628	0.01	32000	0.4340	0.9244
0.4488	0.01	34000	0.4162	0.9265
0.4608	0.01	36000	0.4031	0.9271
0.4478	0.01	38000	0.4502	0.9253
0.4237	0.01	40000	0.4087	0.9279
0.4601	0.01	42000	0.4133	0.9269
0.4153	0.01	44000	0.4230	0.9306
0.4096	0.01	46000	0.4108	0.9301
0.4348	0.01	48000	0.4138	0.9309
0.3787	0.01	50000	0.4066	0.9324
0.4172	0.01	52000	0.4812	0.9206
0.3897	0.02	54000	0.4013	0.9325
0.3787	0.02	56000	0.3837	0.9344
0.4253	0.02	58000	0.3925	0.9347
0.3959	0.02	60000	0.3907	0.9353
0.4402	0.02	62000	0.3708	0.9341
0.4115	0.02	64000	0.3477	0.9361
0.3876	0.02	66000	0.3634	0.9373
0.4286	0.02	68000	0.3778	0.9378
0.422	0.02	70000	0.3540	0.9361
0.3732	0.02	72000	0.3853	0.9378
0.3641	0.02	74000	0.3951	0.9386
0.3701	0.02	76000	0.3582	0.9388
0.4498	0.02	78000	0.3268	0.9375
0.3587	0.02	80000	0.3825	0.9401
0.4474	0.02	82000	0.3155	0.9391
0.3598	0.02	84000	0.3666	0.9388
0.389	0.02	86000	0.3745	0.9377
0.3625	0.02	88000	0.3776	0.9387
0.3511	0.03	90000	0.4275	0.9336
0.3428	0.03	92000	0.4301	0.9336
0.4042	0.03	94000	0.3547	0.9359
0.3583	0.03	96000	0.3763	0.9396
0.3887	0.03	98000	0.3213	0.9412
0.3915	0.03	100000	0.3557	0.9409
0.3378	0.03	102000	0.3627	0.9418
0.349	0.03	104000	0.3614	0.9402
0.3596	0.03	106000	0.3834	0.9381
0.3519	0.03	108000	0.3560	0.9421
0.3598	0.03	110000	0.3485	0.9419
0.3642	0.03	112000	0.3754	0.9395
0.3477	0.03	114000	0.3634	0.9426
0.4202	0.03	116000	0.3071	0.9427
0.3656	0.03	118000	0.3155	0.9441
0.3709	0.03	120000	0.2923	0.9433
0.374	0.03	122000	0.3272	0.9441
0.3142	0.03	124000	0.3348	0.9444
0.3452	0.04	126000	0.3603	0.9436
0.3365	0.04	128000	0.3339	0.9434
0.3353	0.04	130000	0.3471	0.9450
0.343	0.04	132000	0.3508	0.9418
0.3174	0.04	134000	0.3753	0.9436
0.3009	0.04	136000	0.3687	0.9422
0.3785	0.04	138000	0.3818	0.9396
0.3199	0.04	140000	0.3291	0.9438
0.4049	0.04	142000	0.3372	0.9454
0.3435	0.04	144000	0.3315	0.9459
0.3814	0.04	146000	0.3462	0.9401
0.359	0.04	148000	0.3981	0.9361
0.3552	0.04	150000	0.3226	0.9469
0.345	0.04	152000	0.3731	0.9384
0.3228	0.04	154000	0.2956	0.9471
0.3637	0.04	156000	0.2869	0.9477
0.349	0.04	158000	0.3331	0.9430
0.3374	0.04	160000	0.4159	0.9340
0.3718	0.05	162000	0.3241	0.9459
0.315	0.05	164000	0.3544	0.9391
0.3215	0.05	166000	0.3311	0.9451
0.3464	0.05	168000	0.3682	0.9453
0.3495	0.05	170000	0.3193	0.9469
0.305	0.05	172000	0.4132	0.9389
0.3479	0.05	174000	0.3465	0.947
0.3537	0.05	176000	0.3277	0.9449

框架版本

Transformers 4.10.2

Pytorch 1.7.1

Datasets 1.12.1

Tokenizers 0.10.3

训练过程

训练超参数

训练过程中使用了以下超参数：

learning_rate: 5e-05

train_batch_size: 1

eval_batch_size: 8

seed: 42

optimizer: Adam，betas=(0.9,0.999)，epsilon=1e-08

lr_scheduler_type: linear

lr_scheduler_warmup_steps: 1782000

training_steps: 17820000

训练结果

训练损失	轮次	步数	验证损失	准确率
0.7155	0.0	2000	0.7060	0.4622
0.7054	0.0	4000	0.6925	0.5165
0.6842	0.0	6000	0.6653	0.6116
0.6375	0.0	8000	0.5721	0.7909
0.4671	0.0	10000	0.3238	0.8770
0.3403	0.0	12000	0.3692	0.8861
0.4162	0.0	14000	0.4560	0.8908
0.4728	0.0	16000	0.5071	0.8980
0.5111	0.01	18000	0.5204	0.9015
0.4792	0.01	20000	0.5193	0.9076
0.544	0.01	22000	0.4835	0.9133
0.4745	0.01	24000	0.4689	0.9170
0.4403	0.01	26000	0.4778	0.9177
0.4405	0.01	28000	0.4754	0.9163
0.4375	0.01	30000	0.4808	0.9175
0.4628	0.01	32000	0.4340	0.9244
0.4488	0.01	34000	0.4162	0.9265
0.4608	0.01	36000	0.4031	0.9271
0.4478	0.01	38000	0.4502	0.9253
0.4237	0.01	40000	0.4087	0.9279
0.4601	0.01	42000	0.4133	0.9269
0.4153	0.01	44000	0.4230	0.9306
0.4096	0.01	46000	0.4108	0.9301
0.4348	0.01	48000	0.4138	0.9309
0.3787	0.01	50000	0.4066	0.9324
0.4172	0.01	52000	0.4812	0.9206
0.3897	0.02	54000	0.4013	0.9325
0.3787	0.02	56000	0.3837	0.9344
0.4253	0.02	58000	0.3925	0.9347
0.3959	0.02	60000	0.3907	0.9353
0.4402	0.02	62000	0.3708	0.9341
0.4115	0.02	64000	0.3477	0.9361
0.3876	0.02	66000	0.3634	0.9373
0.4286	0.02	68000	0.3778	0.9378
0.422	0.02	70000	0.3540	0.9361
0.3732	0.02	72000	0.3853	0.9378
0.3641	0.02	74000	0.3951	0.9386
0.3701	0.02	76000	0.3582	0.9388
0.4498	0.02	78000	0.3268	0.9375
0.3587	0.02	80000	0.3825	0.9401
0.4474	0.02	82000	0.3155	0.9391
0.3598	0.02	84000	0.3666	0.9388
0.389	0.02	86000	0.3745	0.9377
0.3625	0.02	88000	0.3776	0.9387
0.3511	0.03	90000	0.4275	0.9336
0.3428	0.03	92000	0.4301	0.9336
0.4042	0.03	94000	0.3547	0.9359
0.3583	0.03	96000	0.3763	0.9396
0.3887	0.03	98000	0.3213	0.9412
0.3915	0.03	100000	0.3557	0.9409
0.3378	0.03	102000	0.3627	0.9418
0.349	0.03	104000	0.3614	0.9402
0.3596	0.03	106000	0.3834	0.9381
0.3519	0.03	108000	0.3560	0.9421
0.3598	0.03	110000	0.3485	0.9419
0.3642	0.03	112000	0.3754	0.9395
0.3477	0.03	114000	0.3634	0.9426
0.4202	0.03	116000	0.3071	0.9427
0.3656	0.03	118000	0.3155	0.9441
0.3709	0.03	120000	0.2923	0.9433
0.374	0.03	122000	0.3272	0.9441
0.3142	0.03	124000	0.3348	0.9444
0.3452	0.04	126000	0.3603	0.9436
0.3365	0.04	128000	0.3339	0.9434
0.3353	0.04	130000	0.3471	0.9450
0.343	0.04	132000	0.3508	0.9418
0.3174	0.04	134000	0.3753	0.9436
0.3009	0.04	136000	0.3687	0.9422
0.3785	0.04	138000	0.3818	0.9396
0.3199	0.04	140000	0.3291	0.9438
0.4049	0.04	142000	0.3372	0.9454
0.3435	0.04	144000	0.3315	0.9459
0.3814	0.04	146000	0.3462	0.9401
0.359	0.04	148000	0.3981	0.9361
0.3552	0.04	150000	0.3226	0.9469
0.345	0.04	152000	0.3731	0.9384
0.3228	0.04	154000	0.2956	0.9471
0.3637	0.04	156000	0.2869	0.9477
0.349	0.04	158000	0.3331	0.9430
0.3374	0.04	160000	0.4159	0.9340
0.3718	0.05	162000	0.3241	0.9459
0.315	0.05	164000	0.3544	0.9391
0.3215	0.05	166000	0.3311	0.9451
0.3464	0.05	168000	0.3682	0.9453
0.3495	0.05	170000	0.3193	0.9469
0.305	0.05	172000	0.4132	0.9389
0.3479	0.05	174000	0.3465	0.947
0.3537	0.05	176000	0.3277	0.9449

框架版本

Transformers 4.10.2

Pytorch 1.7.1

Datasets 1.12.1

Tokenizers 0.10.3

bert-base-uncased-amazon_polarity

模型描述

预期用途与限制

训练与评估数据

训练过程

训练超参数

训练结果

框架版本

bert-base-uncased-amazon_polarity

模型描述

预期用途与限制

训练与评估数据

训练过程

训练超参数

训练结果

框架版本