h2o-danube2-1.8b-sft is a chat fine-tuned model by H2O.ai with 1.8 billion parameters. We release three versions of this model:
| Model Name | Description |
|---|---|
| h2oai/h2o-danube2-1.8b-base | Base model |
| h2oai/h2o-danube2-1.8b-sft | SFT tuned |
| h2oai/h2o-danube2-1.8b-chat | SFT + DPO tuned |
This model was trained using H2O LLM Studio.
We adjust the Llama 2 architecture for a total of around 1.8b parameters. For details, please refer to our Technical Report. We use the Mistral tokenizer with a vocabulary size of 32,000 and train our model up to a context length of 8,192.
The details of the model architecture are:
| Hyperparameter | Value |
|---|---|
| n_layers | 24 |
| n_heads | 32 |
| n_query_groups | 8 |
| n_embd | 2560 |
| vocab size | 32000 |
| sequence length | 8192 |
from openmind import pipeline, is_torch_npu_available
from openmind_hub import snapshot_download
if is_torch_npu_available():
device = "npu:0"
else:
device = "cpu"
generate_text = pipeline(
model="SY_AICC/h2o-danube2-1.8b-sft",
torch_dtype=torch.bfloat16,
trust_remote_code=True,
device=device,)
output = generate_text(
"Why is drinking water so healthy?",
max_new_tokens=100)
print(res[0]["generated_text"])You can load the models using quantization by specifying load_in_8bit=True or load_in_4bit=True. Also, sharding on multiple GPUs is possible by setting device_map=auto.
MistralForCausalLM(
(model): MistralModel(
(embed_tokens): Embedding(32000, 2560, padding_idx=0)
(layers): ModuleList(
(0-23): 24 x MistralDecoderLayer(
(self_attn): MistralAttention(
(q_proj): Linear(in_features=2560, out_features=2560, bias=False)
(k_proj): Linear(in_features=2560, out_features=640, bias=False)
(v_proj): Linear(in_features=2560, out_features=640, bias=False)
(o_proj): Linear(in_features=2560, out_features=2560, bias=False)
(rotary_emb): MistralRotaryEmbedding()
)
(mlp): MistralMLP(
(gate_proj): Linear(in_features=2560, out_features=6912, bias=False)
(up_proj): Linear(in_features=2560, out_features=6912, bias=False)
(down_proj): Linear(in_features=6912, out_features=2560, bias=False)
(act_fn): SiLU()
)
(input_layernorm): MistralRMSNorm()
(post_attention_layernorm): MistralRMSNorm()
)
)
(norm): MistralRMSNorm()
)
(lm_head): Linear(in_features=2560, out_features=32000, bias=False)
)Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.
By using the large language model provided in this repository, you agree to accept and comply with the terms and conditions outlined in this disclaimer. If you do not agree with any part of this disclaimer, you should refrain from using the model and any content generated by it.