hf_mirrors/MiniLM-evidence-types
Model Introduction
Files & versions
Analysis

MiniLM-evidence-types

This model is a fine-tuned version of microsoft/MiniLM-L12-H384-uncased on the evidence types dataset. It achieved the following results on the evaluation set:

  • Loss: 1.8672
  • Macro f1: 0.3726
  • Weighted f1: 0.7030
  • Accuracy: 0.7161
  • Balanced accuracy: 0.3616

Training and evaluation data

The data set, as well as the code that was used to fine tune this model can be found in the GitHub repository BA-Thesis-Information-Science-Persuasion-Strategies

Using HuggingFace Transformers

from openmind import AutoModelForCausalLM,AutoTokenizer, AutoModel, pipeline,is_torch_npu_available
from openmind_hub import snapshot_download
import torch
import argparse
import torch.nn.functional as F


def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_name_or_path",
        type=str,
        help="Path to model",
        default="zhouhui/MiniLM-evidence-types",
    )
    args = parser.parse_args()
    return args

def main():
    args = parse_args()
    model_path = args.model_name_or_path

    if is_torch_npu_available():
        device = "npu:0"
    else:
        device = "cpu"
        
    
    pipe = pipeline("sentiment-analysis", model=model_path, framework="pt",device=device)

    sentence_vecs = pipe("Rhonda has been volunteering for several years for a variety of charitable community programs.")
    print(sentence_vecs)


if __name__ == "__main__":
    main()

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20
  • mixed_precision_training: Native AMP

Training results

Training LossEpochStepValidation LossMacro f1Weighted f1AccuracyBalanced accuracy
1.41061.02501.26980.19660.60840.67350.2195
1.14372.05001.09850.34840.69140.71160.3536
0.97143.07501.09010.26060.64130.64460.2932
0.83824.010001.01970.27640.70240.72370.2783
0.71925.012501.08950.28470.68240.69630.2915
0.62496.015001.12960.34870.68880.69480.3377
0.53367.017501.15150.35910.69820.70240.3496
0.46948.020001.19620.36260.71850.73140.3415
0.40589.022501.33130.31210.69200.70850.3033
0.374610.025001.39930.36280.69760.70470.3495
0.326711.027501.50780.35600.69580.70550.3464
0.293912.030001.58750.36850.69680.70620.3514
0.267713.032501.64700.36060.69760.70700.3490
0.242514.035001.71640.37140.70690.72070.3551
0.230115.037501.81510.35970.69750.71230.3466
0.226816.040001.78380.39400.70340.71230.3869
0.20117.042501.83280.37250.69640.70620.3704
0.192318.045001.87880.37080.70190.71540.3591
0.179519.047501.85740.37520.70310.71610.3619
0.171320.050001.86720.37260.70300.71610.3616

Framework versions

  • Transformers 4.19.2
  • Pytorch 1.11.0+cu113
  • Datasets 2.2.2
  • Tokenizers 0.12.1