ai-medical-model-32bit：针对技术性医学问题微调的 Llama3

!python -m pip install --upgrade pip
!pip3 install torch==2.2.1  torchvision torchaudio xformers --index-url https://download.pytorch.org/whl/cu121
!pip install  bitsandbytes  accelerate

from openmind import AutoModelForCausalLM, AutoTokenizer 
import torch

model_name = "LF_AICC/ai-medical-model-32bit"
device_map = 'npu:0'

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float16,
    device_map=device_map
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token

def askme(question):
  prompt = f"<|start_header_id|>system<|end_header_id|> You are a Medical AI chatbot assistant. <|eot_id|><|start_header_id|>User: <|end_header_id|>This is the question: {question}<|eot_id|>"
  # Tokenizing the input and generating the output
  #prompt = f"{question}"
  # Tokenizing the input and generating the output
  inputs = tokenizer([prompt], return_tensors="pt").to("npu")
  outputs = model.generate(**inputs, max_new_tokens=256, use_cache=True)
  answer = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
  # Try Remove the prompt
  try:
      # Split the answer at the first line break, assuming system intro and question are on separate lines
      answer_parts = answer.split("\n", 1)
      # If there are multiple parts, consider the second part as the answer
      if len(answer_parts) > 1:
        answers = answer_parts[1].strip()  # Remove leading/trailing whitespaces
      else:
        answers = ""  # If no split possible, set answer to empty string
      print(f"Answer: {answers}")   
  except:
      print(answer)  

# Example usage
# - Question:  Make the question.
question="What was the main cause of the inflammatory CD4+ T cells?"
askme(question)

Answer: I'm happy to help!

The main cause of inflammatory CD4+ T cells is a complex process that involves multiple factors. However, some of the key triggers include:

1. Activation of CD4+ T cells: CD4+ T cells are activated by antigens, cytokines, and other signals, leading to their proliferation and differentiation into effector cells.
2. Cytokine production: Activated CD4+ T cells produce cytokines such as interleukin-2 (IL-2), interferon-gamma (IFN-γ), and tumor necrosis factor-alpha (TNF-α), which promote inflammation and immune responses.
3. Chemokine production: CD4+ T cells also produce chemokines, such as CCL3, CCL4, and CCL5, which attract other immune cells to the site of inflammation.
4. Toll-like receptor (TLR) activation: TLRs are pattern recognition receptors that recognize pathogen-associated molecular patterns (PAMPs) and activate CD4+ T cells.
5. Bacterial or viral infections: Infections caused by bacteria, viruses, or fungi can trigger the activation of CD4+ T cells and the production of cytokines and chemokines