SAT-HMR - Real-Time Multi-Person 3D Mesh Estimation - using Scale-Adaptive Tokens

Sep 14, 2026

Uploaded by William

SAT-HMR presents a one-stage approach for real-time multi-person 3D human mesh estimation from a single RGB image. While DETR-style one-stage methods achieve strong results with high-resolution inputs, they often waste computation, benefiting small-scale individuals while incurring heavy overhead. SAT-HMR introduces scale-adaptive tokens within the DETR framework, dynamically adjusting resolution per individual based on relative image scale and distilling background regions. Experiments show preserved accuracy benefits and substantially reduced computational cost, enabling real-time inference with performance comparable to state-of-the-art systems.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

Abstract

1. Introduction

Multi-stage vs one-stage approaches

This CVPR paper is the Open Access version, provided by the Computer Vision Foundation. Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore. SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokens Chi Su 1 Xiaoxuan Ma 1,B Jiajun Su5 Yizhou Wang 1,2,3,4,B 1 Center on Frontiers of Computing Studies, School of Computer Science, Peking University 2 Inst. for Artificial Intelligence, Peking University 3 Nat’l Eng. Research Center of Visual Technology 4 State Key Laboratory of General Artificial Intelligence, Peking University 5 International Digital Economy Academy (IDEA) [suchi@stu.pku.edu.cn](suchi@stu.pku.edu.cn) , {maxiaoxuan, [yizhou.wang}@pku.edu.cn](yizhou.wang}@pku.edu.cn) , [sujiajun@idea.edu.cn](sujiajun@idea.edu.cn) B Corresponding authors Image Scale-adaptive Tokens Predictions Elevated View (a) | | | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ROMP (512)
BEV (512)

Multi-HMR (896) | | --- | | Ours (644*)

| Multi-HMR (1288) AiOS (1333) 50 100 150 200 250 300 350 400 450 Inference time (ms) (b) Figure 1 . (a) We propose scale-adaptive tokens in our one-stage framework for real-time multi-person 3D mesh estimation. Our method introduces scale-adaptive tokens, dynamically adjusted based on the relative size of individuals in the image, to more efficiently encode features, enabling real-time and accurate multi-person mesh estimation. We present a conceptual visualization of the scale-adaptive tokens. The right column visualizes the predicted meshes projected onto an image from 3DPW [49] dataset and from an elevated view. (b) Comparison of estimation error and inference time across different methods, with input resolutions in parentheses. Our method, using a mixed resolution with a base resolution of 644, achieves comparable performance to state-of-the-art methods on AGORA [33] test set while maintaining real-time inference efficiency. Code and models are available at [https://ChiSu001](https://ChiSu001) .github .io/SAT-HMR/ . Abstract We propose SAT-HMR, a one-stage framework for real-time multi-person 3D human mesh estimation from a single RGB image. While current one-stage methods, which follow a DETR-style pipeline, achieve state-of-the-art (SOTA) performance with high-resolution inputs, we observe that this particularly benefits the estimation of individuals in smaller scales of the image (e.g., those of young age or far from the camera), but at the cost of significantly increased computation overhead. To address this, we introduce scale-adaptive tokens that are dynamically adjusted based on the relative scale of each individual in the image within the DETR framework. Specifically, individuals in smaller scales are processed at higher resolutions, larger ones at lower resolutions, and background regions are further distilled. These scale-adaptive tokens more efficiently encode the image features, facilitating subsequent decoding to regress the human mesh, while allowing the model to allocate computational resources more effectively and focus on more challenging cases. Experiments show that our method preserves the accuracy benefits of high-resolution processing while substantially reducing computational cost, achieving real-time inference with performance comparable to SOTA methods. Figure 2 . Estimation errors and FPS of baselines with different resolutions and our method across individuals at various scales. The scale of an individual refers to the person’s size relative to the overall image and please refer to Sec. 3.3 for mathematical definition. The colored lines show the Mean Vertex Error (MVE) errors (left y-axis) of the baselines with different resolutions (Res.) on the A

FAQs

What problem does SAT-HMR address?

SAT-HMR targets real-time multi-person 3D human mesh estimation from a single RGB image, estimating meshes for all individuals in the scene.

Why do existing DETR-style one-stage methods require high-resolution inputs?

The paper states that high-resolution processing is crucial for accuracy, but it also greatly increases computational cost, limiting real-time capability.

How do scale-adaptive tokens improve efficiency and accuracy?

Scale-adaptive tokens are dynamically adjusted in resolution within the DETR framework according to each person’s relative scale, using higher resolutions for small individuals and lower resolutions for larger ones while distilling background regions.

Sep 14, 2026

Uploaded by William