This repository packages the model as a TurboQuant TQ3_4S GGUF for local deployment.
This quant requires a TurboQuant-capable runtime. For llama.cpp, use the turbo-tan/llama.cpp-tq3 fork rather than stock upstream llama.cpp if you want native TQ3_4S support.
| File | Quant | Size |
|---|---|---|
Qwen3.6-27B-TQ3_4S.gguf | TQ3_4S | ~13.0 GB |
chat_template.jinja | chat template | text |
thumbnail.png | model card image | png |
Hardware:
Prompt processing:
llama-perplexity --chunks 10 -c 2048PPL = 6.2452 +/- 0.16138prompt eval = 712.02 tok/s16 GB VRAM fit checks on RTX 5060 Ti with the recommended KV settings:
32k context fits64k context fits128k context does not fitturbo-tan/llama.cpp-tq3 fork.mmproj artifact.--jinja enabled so the bundled chat template is honored.128K context when possible for reasoning-heavy workloads. On smaller local GPUs, reduce context as needed to fit memory.Default prompt-processing settings on 16 GB:
llama-bench \
-m Qwen3.6-27B-TQ3_4S.gguf \
-ngl 99 \
-ctk q4_0 \
-ctv tq3_0 \
-fa 1 \
-p 2048 -n 0 -r 3Default chat/server settings:
llama-server \
-m Qwen3.6-27B-TQ3_4S.gguf \
--host 127.0.0.1 --port 8080 \
-ngl 99 -c 4096 -np 1 \
-ctk q4_0 -ctv tq3_0 -fa on \
--jinjallama-cli \
-m Qwen3.6-27B-TQ3_4S.gguf \
--jinja \
-ngl 99 \
-c 4096Build/runtime:
git clone https://github.com/turbo-tan/llama.cpp-tq3[!Note] The upstream Qwen repository contains model weights and configuration files for the post-trained model in the Hugging Face Transformers format.
Those upstream artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers, and related runtimes.
Following the February release of the Qwen3.5 series, Qwen describes Qwen3.6 as the first open-weight Qwen3.6 variant, built for stronger stability and real-world utility.

qwen3527B6451201740816 × (3 × (Gated DeltaNet -> FFN) -> 1 × (Gated Attention -> FFN))48 for V, 16 for QK, head dim 12824 for Q, 4 for KV, head dim 25664262,144SWE-bench Verified: 77.2Terminal-Bench 2.0: 59.3SkillsBench Avg5: 48.2GPQA Diamond: 87.8AIME26: 94.1MMMU: 82.9AndroidWorld: 70.3