Mem-Agent

The capital of France is Paris.

4

I don't have access to our previous conversation history in this session. 
Could you please provide the client's name or any context you'd like me to 
work with? If you're using an external memory system, please ensure the 
relevant context is loaded into the prompt.

Here's a Python function to check if a string is a palindrome:

```python
def is_palindrome(s: str) -> bool:
    """Check if a string is a palindrome (case-insensitive, ignores non-alphanumeric)."""
    # Normalize: lowercase and keep only alphanumeric characters
    cleaned = ''.join(c.lower() for c in s if c.isalnum())
    # Compare with reverse
    return cleaned == cleaned[::-1]

# Examples
print(is_palindrome("A man, a plan, a canal: Panama"))  # True
print(is_palindrome("race a car"))                       # False
print(is_palindrome("Was it a car or a cat I saw?"))    # True

pip install vllm vllm-ascend

from modelscope import snapshot_download
model_dir = snapshot_download('driaforall/mem-agent')
print(f'Model downloaded to: {model_dir}')

import os
os.environ['VLLM_ASCEND_RT_DEVICES'] = '0'

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

model_path = '/path/to/mem-agent'

# 加载 tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

# 加载模型
llm = LLM(
    model=model_path,
    max_model_len=4096,
    gpu_memory_utilization=0.9,
    trust_remote_code=True,
    dtype='bfloat16',
    enforce_eager=True,
)

# 构建对话
messages = [{'role': 'user', 'content': 'What is the capital of France?'}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

# 生成
outputs = llm.generate([prompt], SamplingParams(temperature=0.7, max_tokens=256))
print(outputs[0].outputs[0].text)

export VLLM_ASCEND_RT_DEVICES=0
export TASK_QUEUE_ENABLE=1
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True

python3 -m vllm.entrypoints.openai.api_server \
    --model /path/to/mem-agent \
    --max-model-len 65536 \
    --gpu-memory-utilization 0.95 \
    --trust-remote-code \
    --dtype bfloat16 \
    --enforce-eager

export VLLM_ASCEND_RT_DEVICES=0,1
export TASK_QUEUE_ENABLE=1
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True

python3 -m vllm.entrypoints.openai.api_server \
    --model /path/to/mem-agent \
    --max-model-len 65536 \
    --gpu-memory-utilization 0.95 \
    --trust-remote-code \
    --tensor-parallel-size 2 \
    --dtype bfloat16 \
    --enforce-eager

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "/path/to/mem-agent",
    "messages": [{"role": "user", "content": "Hello, how are you?"}],
    "max_tokens": 128,
    "temperature": 0.7
  }'

# 安装 lm-eval
pip install lm-eval

# 运行 GSM8K 5-shot 评估 (NPU)
lm_eval --model vllm \
  --model_args "pretrained=/path/to/mem-agent,tensor_parallel_size=1,dtype=bfloat16,gpu_memory_utilization=0.95,max_model_len=65536,trust_remote_code=True" \
  --tasks gsm8k \
  --num_fewshot 5 \
  --batch_size auto \
  --output_path ./results/mem-agent-gsm8k-npu

# 对比 NPU vs GPU 基线差异，门禁标准: ≤ ±2pp 即为通过