HEART — Hybrid Efficient Attention with Rank-factorized bias Transformer, by Philip Hofmann (Phips)
(Phips/HEART, Apache-2.0): a 16.7 M-parameter HAT-iLN window-attention
super-resolution network, trained only on the CC0 Phips/lucid-cc0-v2-hc-512
corpus — converted to MLX in fp32 for the Swift/MLX port
xocialize/mlx-heart-swift (MLXEngine imageUpscale, the fidelity
stills tier). The four released checkpoints, per-tensor exact conversions of the upstream files at revision
868878ce4c253a8061300f923b620fdc6edf090a (oracle/convert_weights.py: upstream keys unchanged, conv weights
(O,I,kH,kW) → (O,kH,kW,I), nothing else touched):
| file | upstream checkpoint | variant | scale | role |
|---|---|---|---|---|
heart_4x_otf_v2_fp32.safetensors | models/heart_4x_otf_v2.safetensors | .fidelity (default) | 4× | OTF fidelity — the best damaged-input arm on the Forge bench |
heart_4x_otf_gan_fp32.safetensors | models/heart_4x_otf_gan.safetensors | .sharp | 4× | OTF GAN — sharpest real-world output |
heart_4x_pretrain_fp32.safetensors | models/heart_4x_pretrain.safetensors | .clean | 4× | official pretrain — clean sources |
heart_2x_fp32.safetensors | models/heart_2x.safetensors | .clean2x | 2× | official 2× pretrain |
config.json | — | — | — | architecture, variants, source sha256s, the fp16 dtype rule |
Lanes: mlx-community/HEART-fp16 is the shipping lane (conv / linear tensors fp16; the i-LN and affine weight/bias and
the RIB implicit-net parameters stay fp32 — 220 of 748 tensors — and the port runs every reduction in fp32: the i-LN
statistics, the RIB position tables, the softmax, the global average pool and its squeeze/excite head);
mlx-community/HEART-fp32 is the parity / reference lane. Each variant is a
separate file so a package pulls only the checkpoint it uses.
Against the upstream PyTorch implementation (heart_arch.py + traiNNer-redux helpers, CPU fp32, same input), fp32 lane:
.fidelity 127.5 dB PSNR at 128² (.sharp 125.3, .clean 122.5, .clean2x 124.0); every sub-op within 1e-5
relative. Against the author's ONNX exports under ONNX Runtime the port agrees exactly as far as torch itself does
(90.2 / 84.9 / 95.8 dB at 128²; the heart_4x_pretrain ONNX file does not reproduce its checkpoint, 7.8 dB, and ORT drifts
with image size). Details, the fp16 study and the tiling study: PORTING-SPEC.md in the port repo.
import MLXServeCore, MLXHEART
let engine = MLXServeEngine()
let id = try await engine.register(HEARTUpscalePackage.registration,
configuration: HEARTConfiguration(variant: .fidelity, quant: .fp16))
let out = try await engine.run(ImageUpscaleRequest(image: image), package: id) // 4× (2× via .clean2x)The engine materializes the selected variant's file from this repo into its model store on first use (WeightSourcing).
Apache-2.0 throughout: the HEART code and pretrained weights (Philip Hofmann / Phips), the HAT and HAT-iLN
architecture family (XPixelGroup, Apache-2.0; arXiv 2205.04437, 2504.06629), traiNNer-redux's hat_iln_arch.py
(Apache-2.0) and SST's RIB (arXiv 2603.06738). Training corpus: Phips/lucid-cc0-v2-hc-512 — platform-declared CC0
(LUCID ← nyuuzyou/pxhere ← pxhere.com), which the re-host takes as governing. Credit the author when you use these
weights.