hf_mirrors/gbert-large-openmind
Model Introduction
Files & versions
Analysis

German BERT large

Released, Oct 2020, this is a German BERT language model trained collaboratively by the makers of the original German BERT (aka "bert-base-german-cased") and the dbmdz BERT (aka bert-base-german-dbmdz-cased). In our paper, we outline the steps taken to train our model and show that it outperforms its predecessors.

Overview

Paper: here
Architecture: BERT large
Language: German

Using in Openmind

from openmind import pipeline, AutoTokenizer, is_torch_npu_available
from openmind_hub import snapshot_download
import torch.nn.functional as F
from torch import Tensor
import openmind
import torch
import argparse
import time

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_name_or_path",
        type=str,
        help="Path to model",
        default="jeffding/gbert-large-openmind",
    )
    args = parser.parse_args()
    return args

def main():
    args = parse_args()
    model_path = args.model_name_or_path

    if is_torch_npu_available():
        device = "npu:0"
    else:
        device = "cpu"
    
    start_time = time.time()
    
    tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=True)
    pipe = pipeline('fill-mask', model=model_path, torch_dtype=torch.bfloat16, device_map=device)
    MASK_TOKEN = tokenizer.mask_token
    result = pipe("Hallo, ich bin ein {} model.".format(MASK_TOKEN))
    print(result)
    
    end_time = time.time()
    print(f"硬件环境:{device},推理执行时间:{end_time - start_time}秒")
    
if __name__ == "__main__":
    main()

Performance

GermEval18 Coarse: 80.08
GermEval18 Fine:   52.48
GermEval14:        88.16

See also:
deepset/gbert-base
deepset/gbert-large
deepset/gelectra-base
deepset/gelectra-large
deepset/gelectra-base-generator
deepset/gelectra-large-generator

Authors

Branden Chan: branden.chan@deepset.ai
Stefan Schweter: stefan@schweter.eu
Timo Möller: timo.moeller@deepset.ai

About us

deepset is the company behind the production-ready open-source AI framework Haystack.

Some of our other work:

  • Distilled roberta-base-squad2 (aka "tinyroberta-squad2")
  • German BERT, GermanQuAD and GermanDPR, German embedding model
  • deepset Cloud, deepset Studio

Get in touch and join the Haystack community

For more info on Haystack, visit our GitHub repo and Documentation.

We also have a Discord community open to everyone!

Twitter | LinkedIn | Discord | GitHub Discussions | Website | YouTube

By the way: we're hiring!