MoSa: Motion Generation with Scale-wise Autoregressive Modeling

Text-driven 3D human motion generation has gained considerable attention, and the dominant Vector Quantization-guided Generative Transformers (VQ-GT) paradigm shows great promise. Recent methods adopt residual VQ to preserve fine-grained motion details with multi-layer tokens. However, the multi-fold increase in token count substantially complicates the GT phase. Existing methods typically employ two independent transformers to separately model base and residual tokens, leading to inherent cross-layer misalignment that compromises structural coherence. To address this, we introduce MoSa, a cohesive framework that reconciles quantization and generation through a coarse-to-fine hierarchical process. We propose Multi-scale Token Preservation Strategy (MTPS) that applies progressively scaled interpolations at each residual quantization layer to effectively retain multi-scale tokens. This empowers a single transformer to jointly model all hierarchical tokens via Scale-wise AutoRegressive (SAR) modeling. With this, SAR facilitates coherent cross-scale dependency modeling within a unified architecture while logically aligning inference steps with quantization layers. Furthermore, as frequent interpolation in MTPS induces reconstruction degradation, we propose CAQ-VAE, a convolution-attention hybrid VQ-VAE that enhances residual blocks and captures global dependencies for high-fidelity reconstruction. Extensive experiments demonstrate that MoSa achieves state-of-the-art performance in both generation quality and efficiency. On Motion-X, MoSa attains an FID of 0.06 (versus MoMask's 0.20) while reducing inference time by 27%. Beyond generation, we demonstrate that MoSa supports motion editing without additional fine-tuning.