hf_mirrors/unimate-uniml3d-f60-v3-cross-attn-bf16
Model Introduction
Files & versions
Analysis

UniMate UniML3D f60 v3 cross-attention (MLX, bfloat16)

UniMate (SIGGRAPH Asia 2026) generates motion from text for skeletons of any topology: animals, humanoids and rigged objects. A flow-matching transformer, conditioned on a flan-t5-base encoding of the prompt and on the skeleton's T-pose, hierarchy and joint names, turns noise into 60 frames (2 s at 30 fps) of motion.

Paper · Project page · Code · Original weights

This repository holds the unimate_uniml3d_f60_v3_cross_attn checkpoint converted to MLX for Apple silicon, for use with mlx-vlm.

Checkpoint

SourceLinzhan/UniMate, folder unimate_uniml3d_f60_v3_cross_attn, checkpoints/checkpoint_step_150000.pt (step 150000), revision 174fc755ec248d0c98f27759b7a95648e7f05905
WeightsThe EMA weights (decay 0.9999), which upstream inference loads
Variantfull attention x cross-attention text conditioning, the alternative the paper compares with the recommended graph x adaLN model. One attention runs over the flattened joint x time tokens; the caption tokens enter every block as cross-attention keys and values.
Training dataUniML3D, skeletons with 5 to 70 joints
Text encodergoogle/flan-t5-base encoder, revision 7bcac572ce56db69c1ea7c8af255c5d7c9672fc2; the decoder is not included

mlx-community/unimate-uniml3d-f60-v3-bf16 holds the recommended graph attention x adaLN model (unimate_uniml3d_f60_v3), converted the same way.

Files

FileSizeContent
model.safetensors352 MB415 bfloat16 tensors: the denoiser (66.2M parameters) and the flan-t5-base encoder (109.6M) under text_encoder.
model.safetensors.index.json31.3 kBWeight index
config.json2.03 kBModel configuration and the normalization statistics of upstream dataset_stats.npy
spiece.model, tokenizer.json, tokenizer_config.json, special_tokens_map.json792 kB, 2.42 MB, 2.54 kB, 2.2 kBflan-t5-base tokenizer
LICENSE20 kBUpstream license (CC BY-NC 4.0)

Changes from the original

This is an unofficial conversion, not made or endorsed by the UniMate authors. Changes made:

  • Extracted the EMA weights (ema_state_dict.shadow_params) and named them after the model's state_dict keys. The raw training weights, optimizer and scheduler states are not included.
  • Dropped the 80 per-block aliases of the shared spectral RoPE (transformer_blocks.N.attn.rope.*) that the upstream state_dict repeats; its weights are stored once, under rope.*.
  • Cast all weights from float32 to bfloat16.
  • Bundled the flan-t5-base encoder weights and tokenizer, which upstream downloads on first use, and stored the normalization statistics in config.json.

Converted with python -m mlx_vlm.models.unimate.convert --dtype bfloat16.

Usage

Requires an mlx-vlm version with UniMate support.

import numpy as np
from mlx_vlm import load
from mlx_vlm.models.unimate.generate import expand_motion, generate_motion, joint_positions
from mlx_vlm.models.unimate.processing_unimate import load_skeleton

model, processor = load("mlx-community/unimate-uniml3d-f60-v3-cross-attn-bf16")
skeleton = load_skeleton("UniMate/assets/examples/eagle")  # a rig_preprocess asset directory

motion = generate_motion(model, processor, skeleton, "An object flaps its wings.", seed=0)
positions = joint_positions(motion)  # (60, J, 3) global joint positions
np.save("eagle-flap.npy", np.array(motion))  # (60, J, 12) float32

# One 160-frame motion from three prompts; consecutive segments share 10 frames.
long = expand_motion(
    model,
    processor,
    skeleton,
    ["An object flaps its wings.", "An object glides forward.", "An object strikes forward."],
    seed=0,
)

These functions return lazy mx.arrays: nothing is computed until the result is evaluated, for example with np.array(...) or mx.eval(...). generate_motion also takes a list of prompts and num_samples, giving (N, 60, J, 12), and cfg_scale (default 3; higher follows the prompt more strongly). inbetween_motion and edit_motion in the same module fill in the frames between kept keyframes and regenerate all but the named joints.

From the command line:

python -m mlx_vlm.models.unimate.generate \
  --model mlx-community/unimate-uniml3d-f60-v3-cross-attn-bf16 \
  --asset UniMate/assets/examples/eagle \
  --prompt "An object flaps its wings." "An object strikes forward." \
  --num-repetitions 3 --seed 0 --output-dir outputs/samples/custom

This writes motions/<asset>-<prompt>-rep_<r>-<i>.npy, captions.json and manifest.json into --output-dir, the layout of upstream sample.py. --motion-expand chains the prompts into one motion instead (--expand-overlap, default 10 frames).

Inputs and outputs

  • Prompt: English, in the style of the training captions. Start with "An object" and describe one motion ("An object flaps its wings.", not "A blue parrot flaps its wings in the sky.").
  • Skeleton: load_skeleton reads an asset directory that upstream rig_preprocess builds from a rigged GLB, glTF or FBX file (the repository's assets/examples has an eagle, a shark and a Unitree Go2), or one entry of a UniML3D cond file ("dataset/features/truebones/cond.npy:Horse"). Skeleton.from_arrays(parents, rest_positions, joint_names) builds one from arrays: parents root first, the T-pose in UniMate's canonical frame (root at the XZ origin, Y up, facing +Z), and cleaned joint names such as "Left Wing". The model was trained on rigs with 5 to 70 joints.
  • Output: (60, J, 12) float32, 60 frames at 30 fps. Per frame and joint: a position (3), a rotation relative to the T-pose in 6D form (6) and a velocity (3), denormalized, as upstream sample.py saves them. joint_positions turns them into global joint positions. Upstream scripts/run_animate_motion.sh, run from the UniMate repository in its own environment on the CLI's output directory, drives each asset's mesh with the .npy motions and exports GLB and FBX files.

Precision

The weights are stored in bfloat16. At load, the attention and MLP weights of the transformer blocks are cast to float16 (block_dtype in config.json) and everything else to float32, so the residual stream, the RMSNorm statistics, adaLN, the time embedding, the input and output layers and the dopri5 solver run in float32. Computing in bfloat16 is avoided: it makes the adaptive solver take about three times as many steps. The solver also runs on the skeleton's own joints by default (joint trimming); upstream pads every skeleton to 71 joints, which changes only the solver's step sizes.

To reproduce the PyTorch reference closely, set "block_dtype": "float32" in a local copy of config.json, pass pad_joints=True (CLI --pad-joints) and set MLX_ENABLE_TF32=0 in the environment (on M5 and later GPUs, MLX otherwise runs float32 matmuls at TF32 precision).

During development the MLX port matched the PyTorch float32 reference to about 3e-5 (relative) per forward pass, in float32 with TF32 off. With these bfloat16 weights and the default settings (float16 blocks), samples differ from the float32 reference by about 2e-3 to 2e-2 relative to the largest feature.

Performance

M5 Max, MLX 0.32.3, default settings, cfg_scale 3, seeds 0 to 2, warm. NFE counts the guided model calls of the adaptive dopri5 solve.

TaskTimeNFE
2 s sample, eagle (51 joints)2.5 s (2.3 to 2.8)116 to 134
2 s sample, shark (16 joints)0.68 s (0.63 to 0.75)110 to 116
5.3 s motion (3 prompts, 160 frames), eagle, CLI4.6 sfirst segment as above, then 50 Euler steps per added segment

The CLI time is the first solve in a new process, model loading excluded. Peak MLX memory for an eagle sample is 1.78 GiB, weights included. The reference-precision settings above take 14.9 s per eagle sample.

License

The weights are released by the UniMate authors under CC BY-NC 4.0 for non-commercial use only; this conversion keeps that license (see LICENSE). The license covers only the authors' rights: the training data (Truebones ZOO, Mixamo, Objaverse-XL) keep their own terms. See the License section of the upstream model card, including its note on the provenance of the Truebones data. The bundled flan-t5-base encoder and tokenizer come from google/flan-t5-base under the Apache 2.0 license. The UniMate code is MIT licensed.

Citation

@article{mou2026unimate,
  title   = {UniMate: One Unified Model to Animate Diverse Skeletons},
  author  = {Mou, Linzhan and Lei, Jiahui and Dou, Zhiyang and Cai, Chenyue and Song, Chaoyue and Finkelstein, Adam and Rusinkiewicz, Szymon},
  journal = {arXiv preprint arXiv:2609.05415},
  year    = {2026}
}