UniMate (SIGGRAPH Asia 2026) generates motion from text for skeletons of any topology: animals, humanoids and rigged objects. A flow-matching transformer, conditioned on a flan-t5-base encoding of the prompt and on the skeleton's T-pose, hierarchy and joint names, turns noise into 60 frames (2 s at 30 fps) of motion.
Paper · Project page · Code · Original weights
This repository holds the unimate_uniml3d_f60_v3_cross_attn checkpoint converted to MLX for
Apple silicon, for use with mlx-vlm.
| Source | Linzhan/UniMate, folder unimate_uniml3d_f60_v3_cross_attn, checkpoints/checkpoint_step_150000.pt (step 150000), revision 174fc755ec248d0c98f27759b7a95648e7f05905 |
| Weights | The EMA weights (decay 0.9999), which upstream inference loads |
| Variant | full attention x cross-attention text conditioning, the alternative the paper compares with the recommended graph x adaLN model. One attention runs over the flattened joint x time tokens; the caption tokens enter every block as cross-attention keys and values. |
| Training data | UniML3D, skeletons with 5 to 70 joints |
| Text encoder | google/flan-t5-base encoder, revision 7bcac572ce56db69c1ea7c8af255c5d7c9672fc2; the decoder is not included |
mlx-community/unimate-uniml3d-f60-v3-bf16 holds the recommended graph attention x adaLN model (unimate_uniml3d_f60_v3), converted the same way.
| File | Size | Content |
|---|---|---|
model.safetensors | 352 MB | 415 bfloat16 tensors: the denoiser (66.2M parameters) and the flan-t5-base encoder (109.6M) under text_encoder. |
model.safetensors.index.json | 31.3 kB | Weight index |
config.json | 2.03 kB | Model configuration and the normalization statistics of upstream dataset_stats.npy |
spiece.model, tokenizer.json, tokenizer_config.json, special_tokens_map.json | 792 kB, 2.42 MB, 2.54 kB, 2.2 kB | flan-t5-base tokenizer |
LICENSE | 20 kB | Upstream license (CC BY-NC 4.0) |
This is an unofficial conversion, not made or endorsed by the UniMate authors. Changes made:
ema_state_dict.shadow_params) and named them
after the model's state_dict keys. The raw training weights, optimizer
and scheduler states are not included.transformer_blocks.N.attn.rope.*) that the upstream
state_dict repeats; its weights are stored once, under rope.*.config.json.Converted with python -m mlx_vlm.models.unimate.convert --dtype bfloat16.
Requires an mlx-vlm version with UniMate support.
import numpy as np
from mlx_vlm import load
from mlx_vlm.models.unimate.generate import expand_motion, generate_motion, joint_positions
from mlx_vlm.models.unimate.processing_unimate import load_skeleton
model, processor = load("mlx-community/unimate-uniml3d-f60-v3-cross-attn-bf16")
skeleton = load_skeleton("UniMate/assets/examples/eagle") # a rig_preprocess asset directory
motion = generate_motion(model, processor, skeleton, "An object flaps its wings.", seed=0)
positions = joint_positions(motion) # (60, J, 3) global joint positions
np.save("eagle-flap.npy", np.array(motion)) # (60, J, 12) float32
# One 160-frame motion from three prompts; consecutive segments share 10 frames.
long = expand_motion(
model,
processor,
skeleton,
["An object flaps its wings.", "An object glides forward.", "An object strikes forward."],
seed=0,
)These functions return lazy mx.arrays: nothing is computed until the
result is evaluated, for example with np.array(...) or mx.eval(...).
generate_motion also takes a list of prompts and num_samples, giving
(N, 60, J, 12), and cfg_scale (default 3; higher follows the prompt more
strongly). inbetween_motion and edit_motion in the same module fill in
the frames between kept keyframes and regenerate all but the named joints.
From the command line:
python -m mlx_vlm.models.unimate.generate \
--model mlx-community/unimate-uniml3d-f60-v3-cross-attn-bf16 \
--asset UniMate/assets/examples/eagle \
--prompt "An object flaps its wings." "An object strikes forward." \
--num-repetitions 3 --seed 0 --output-dir outputs/samples/customThis writes motions/<asset>-<prompt>-rep_<r>-<i>.npy, captions.json and
manifest.json into --output-dir, the layout of upstream sample.py.
--motion-expand chains the prompts into one motion instead
(--expand-overlap, default 10 frames).
load_skeleton reads an asset directory that upstream
rig_preprocess builds from
a rigged GLB, glTF or FBX file (the repository's
assets/examples has an eagle, a shark
and a Unitree Go2), or one entry of a UniML3D cond file
("dataset/features/truebones/cond.npy:Horse"). Skeleton.from_arrays(parents, rest_positions, joint_names) builds one from arrays: parents root first,
the T-pose in UniMate's canonical frame (root at the XZ origin, Y up,
facing +Z), and cleaned joint names such as "Left Wing". The model was
trained on rigs with 5 to 70 joints.(60, J, 12) float32, 60 frames at 30 fps. Per frame and
joint: a position (3), a rotation relative to the T-pose in 6D form (6)
and a velocity (3), denormalized, as upstream sample.py saves them.
joint_positions turns them into global joint positions. Upstream
scripts/run_animate_motion.sh,
run from the UniMate repository in its own environment on the CLI's output
directory, drives each asset's mesh with the .npy motions and exports GLB
and FBX files.The weights are stored in bfloat16. At load, the attention and MLP weights
of the transformer blocks are cast to float16 (block_dtype in
config.json) and everything else to float32, so the residual stream, the
RMSNorm statistics, adaLN, the time embedding, the input and output layers
and the dopri5 solver run in float32. Computing in bfloat16 is avoided: it
makes the adaptive solver take about three times as many steps. The solver
also runs on the skeleton's own joints by default (joint trimming); upstream
pads every skeleton to 71 joints, which changes only the solver's step
sizes.
To reproduce the PyTorch reference closely, set "block_dtype": "float32"
in a local copy of config.json, pass pad_joints=True (CLI
--pad-joints) and set MLX_ENABLE_TF32=0 in the environment (on M5 and
later GPUs, MLX otherwise runs float32 matmuls at TF32 precision).
During development the MLX port matched the PyTorch float32 reference to about 3e-5 (relative) per forward pass, in float32 with TF32 off. With these bfloat16 weights and the default settings (float16 blocks), samples differ from the float32 reference by about 2e-3 to 2e-2 relative to the largest feature.
M5 Max, MLX 0.32.3, default settings, cfg_scale 3, seeds 0 to 2, warm.
NFE counts the guided model calls of the adaptive dopri5 solve.
| Task | Time | NFE |
|---|---|---|
| 2 s sample, eagle (51 joints) | 2.5 s (2.3 to 2.8) | 116 to 134 |
| 2 s sample, shark (16 joints) | 0.68 s (0.63 to 0.75) | 110 to 116 |
| 5.3 s motion (3 prompts, 160 frames), eagle, CLI | 4.6 s | first segment as above, then 50 Euler steps per added segment |
The CLI time is the first solve in a new process, model loading excluded. Peak MLX memory for an eagle sample is 1.78 GiB, weights included. The reference-precision settings above take 14.9 s per eagle sample.
The weights are released by the UniMate authors under
CC BY-NC 4.0 for
non-commercial use only; this conversion keeps that license (see LICENSE).
The license covers only the authors' rights: the training data (Truebones
ZOO, Mixamo, Objaverse-XL) keep their own terms. See the License section of
the upstream model card, including
its note on the provenance of the Truebones data. The bundled flan-t5-base
encoder and tokenizer come from
google/flan-t5-base under the
Apache 2.0 license. The UniMate
code is MIT licensed.
@article{mou2026unimate,
title = {UniMate: One Unified Model to Animate Diverse Skeletons},
author = {Mou, Linzhan and Lei, Jiahui and Dou, Zhiyang and Cai, Chenyue and Song, Chaoyue and Finkelstein, Adam and Rusinkiewicz, Szymon},
journal = {arXiv preprint arXiv:2609.05415},
year = {2026}
}