The thinker of Qwen/Qwen2.5-Omni-3B: the Qwen2 language model (36 × 2048) and the Whisper-shaped audio tower. It is converted to MLX as the semantic encoder of AuK in xocialize/mlx-auk-swift. The vision tower, LM head, talker and token2wav are not included. This is an encoder for its hidden states, not a chat model.
The 36 LLM layers and 32 audio-tower layers have their Linears quantised to affine 8-bit, group size 64 (4.26 GB). Embeddings, norms and projections are bf16.
Qwen RESEARCH LICENSE AGREEMENT: research or evaluation use only, non-commercial. The full text is in LICENSE, and the attribution notice in NOTICE. For commercial use, request a licence from Alibaba Cloud.
These are modified files: converted (and, for -8bit / -4bit, quantised) from the original.
Built with Qwen.
encoder.safetensors with thinker.model.* and thinker.audio_tower.* keys; audio conv weights in MLX (O, K, I)
layout.config.json (text, audio and token ids, plus quantization when quantised).At fp32 against the transformers 4.57.1 reference:
As AuK's encoder, the bf16, 8-bit and 4-bit tiers all pass AuK's pre-registered quality gate (error rate, CAM++ SIM, DNSMOS against fp32) in English. On Mandarin content the 4-bit tier fails it: AuK's de-accent CER goes from 3.6 % to 17.9 %, and content words are lost. Use the 8-bit tier for Mandarin edits.