8-bit MLX conversion of meta-llama/Llama-Guard-3-1B optimized for Apple Silicon native GPU inference.
Converted by: SirSahOl
Source Model: meta-llama/Llama-Guard-3-1B
Framework: MLX by Apple
Quantization: 8-bit (Average 8.25 bits per weight)
Format: safetensors
License: llama3.2
pip install mlx-lm# Chat interactively
mlx_lm.chat --model SirSahOl/Llama-Guard-3-1B-chat-mlx-8bit
# Generate text
mlx_lm.generate --model SirSahOl/Llama-Guard-3-1B-chat-mlx-8bit --prompt "Write a short poem about artificial intelligence."from mlx_lm import load, generate
model, tokenizer = load("SirSahOl/Llama-Guard-3-1B-chat-mlx-8bit")
messages = [
{"role": "user", "content": "Explain quantum superposition in simple terms."}
]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)
print(response)Estimated decoding throughput, time-to-first-token (TTFT), and active unified memory footprint across Apple Silicon tiers:
| Apple Silicon Tier | Unified Memory | Active VRAM | Estimated Speed | Est. TTFT | Recommended Use Case |
|---|---|---|---|---|---|
| M1 / M2 / M3 / M4 (Base) | 8 GB Unified Memory | ~1.5 GB | ~121 tokens/sec | ~22 ms | Everyday interactive assistant & fast local completions |
| M1 / M2 / M3 / M4 Pro | 18 GB – 36 GB | ~1.5 GB | ~182 tokens/sec | ~15 ms | Balanced daily driver for coding, tool invocation, and multi-turn chat |
| M1 / M2 / M3 / M4 Max | 36 GB – 128 GB | ~1.5 GB | ~260 tokens/sec | ~9 ms | High-throughput generation, low latency, agent orchestration |
| M1 / M2 / M3 Ultra | 64 GB – 192 GB | ~1.5 GB | ~363 tokens/sec | ~6 ms | Peak concurrency, batch document extraction, production serving |
Projections based on Apple Silicon memory bandwidth saturation for 8-bit weights. Real-world speeds vary by prompt length.
Evaluate your hardware budget and choose the optimal precision:
| Variant | Disk Size | VRAM Footprint | Target Apple Silicon Hardware | Key Advantage |
|---|---|---|---|---|
| 4-bit MLX | ~0.9 GB | ~0.9 GB | M1 / M2 / M3 / M4 (8GB+) | Maximum generation speed and lowest RAM overhead. |
| 8-bit MLX (This Repository) | ~1.5 GB | ~1.5 GB | M1 / M2 / M3 / M4 Pro/Max (16GB+) | Balanced accuracy and generation speed; near-lossless reasoning. |
| 16-bit MLX | ~2.8 GB | ~2.8 GB | M2 / M3 / M4 Max/Ultra (32GB+) | Full unquantized precision; reference evaluation quality. |
| Your Hardware | Recommended Quantization |
|---|---|
| M1/M2/M3/M4 (8GB – 16GB) | 4-bit — Best balance of speed, low memory, and multitasking capability |
| M1/M2/M3/M4 Pro/Max (18GB – 36GB) | 8-bit — Higher quality reasoning with comfortable memory headroom |
| M1/M2/M3/M4 Max/Ultra (36GB – 192GB) | 16-bit — Unquantized full precision, zero quality degradation |
General guidance:
| Variant | Link |
|---|---|
| 4-bit | SirSahOl/Llama-Guard-3-1B-chat-mlx-4bit |
| 8-bit | SirSahOl/Llama-Guard-3-1B-chat-mlx-8bit |
| 16-bit | SirSahOl/Llama-Guard-3-1B-chat-mlx-16bit |
To prevent runaway loops and ensure correct conversational turn-taking, configure custom stop tokens in your local inference runtime:
<|im_start|><|im_end|><|endoftext|><|im_start|>system\n<|im_end|>\n<|im_start|>user\n<|im_end|>\n<|im_start|>assistant\nFROM SirSahOl/Llama-Guard-3-1B-chat-mlx-8bit
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.7ollama create llama-guard-3-1b-chat-mlx-8bit -f Modelfile
ollama run llama-guard-3-1b-chat-mlx-8bit| Property | Value |
|---|---|
| Source Model | meta-llama/Llama-Guard-3-1B |
| Quantization | 8-bit |
| mlx-lm Version | 0.31.3 |
| Conversion Time | 5.41s |
| Output Size | 1.5 GB |
| Date | 2026-09-22T02:19:25.790702+00:00 |
To reproduce this conversion:
pip install mlx-lm==0.31.3
python3 -m mlx_lm.convert --hf-path /root/.cache/huggingface/hub/models--alpindale--Llama-Guard-3-1B/snapshots/758d9d0f17f85db2fb7fae0dd9ec67bb9b099592 --mlx-path output/Llama-Guard-3-1B-mlx-8bit -q --q-bits 8This model conversion inherits the license of the source model: llama3.2.
See the original model card for full license details.
| Version | Date | Changes |
|---|---|---|
| v1.0 | 2026-09-22 | Initial conversion |
Converted with MLX Foundry — a professional pipeline for converting models to Apple MLX format.