[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85711-en":3,"doc-seo-85711-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85711,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Research & Report","Towards Real World Wearable Motion Reconstruction","The work addresses wearable motion capture as the task of reconstructing full-body movement from whatever sensing hardware is worn at a given moment, rather than assuming a fixed sensor setup. It proposes an unobtrusive, consumer-device centered framework using smartphones, smartwatches, smart glasses, and smart insoles. Three contributions follow: a large synchronized multimodal dataset with 3D ground truth across 50 activities, a baseline generative model (WHIP) for arbitrary sensor subsets, and a study quantifying sensor complementarity.","Towards Real-World Wearable Motion Reconstruction  \nAndrea Boscolo Camiletto 1 ,2 , Rishabh Dabral 1 ,2 , Eduardo Alvarado 1 , Thabo Beeler3 , Marc Habermann 1 ,2 , and Christian Theobalt 1 ,2  \n1 Max Planck Institute for Informatics, Saarland Informatics Campus, Germany  \n2 Saarbrücken Research Center for Visual Computing, Interaction and AI, Germany  \n3 Google, Switzerland  \narXiv :2607 .09780v 1 [ cs .CV] 8 Jul 2026  \nFig. 1: We present a multimodal wearable sensor suite for motion capture (left) and show full-body motion reconstructed by our model from sparse sensor readings (right) .  \nAbstract. The modern-day surge in popularity of wearable devices poses a fundamentally unique motion capture problem: reconstructing fullbody movement from any set of sensing hardware worn at a given moment. Yet, most research efforts assume fixed sensor configurations (e.g. , IMU suits or HMD-centric rigs) and cannot generalize across them.  \nIn contrast, we argue that motion capture should prioritize unobtrusive and lightweight devices such as smartphones, smartwatches, smart glasses, and smart insoles, and study the interplay between them. To this end, we make three contributions. First, we present a large-scale multimodal dataset synchronizing these consumer-grade sensors with groundtruth 3D motion, spanning 50 diverse activities including everyday tasks, sports, and social interactions. Second, we propose WHIP, a baseline generative model that reconstructs motion from arbitrary subsets of available sensors, robustly handling missing modalities and producing physically plausible motions. Third, we conduct a systematic study of sensor complementarity, quantifying how different modalities complement one another. Code and dataset are available at this URL.  \nKeywords: Motion capture · Wearable sensors · Multimodal learning · Human pose estimation  \n2 A. Boscolo Camiletto et al.  \n1 Introduction  \nHuman Motion Capture (MoCap) is essential across numerous fields, including gaming, sports, medicine, VR/AR, robotics, and filmmaking, enabling the accurate tracking and reproduction of human movements. Traditional MoCap solutions, however, present challenges for real-world deployment: optical markerbased systems [34] achieve excellent precision but require cumbersome suits and controlled environments, whereas markerless vision-based systems require extensive camera infrastructure [5] for high-quality results. This has motivated a shift toward lightweight, accessible sensing: smartphones and smartwatches are now ubiquitous, and smart insoles and head-mounted devices increasingly common. Together they offer complementary cues that no single modality provides, making them a promising basis for untethered motion capture.  \nHowever, converting such sparse, commodity signals into reliable full-body tracking remains a challenging and underconstrained problem. Prior work has addressed individual modalities in isolation. IMU-based methods use as few as six sensors [13, 22, 44, 45], yet even these multi-sensor setups require careful calibration and remain impractical for daily use. A parallel line of work leverages VR/AR hardware: headsets and controllers provide native head and hand tracking, often achieving strong reconstruction accuracy [14, 38] . However, inferring leg motion from only upper-body cues is fundamentally ambiguous. Headsets with body-facing cameras partially mitigate this ambiguity by directly observing the body [2,4,48], yet occlusions and limited hardware availability still limit their practicality. A complementary direction of increasing interest is plantar pressure sensing: by measuring foot–ground interactions, it provides informative cues on gait, balance, and contact events [1] . This trend is fueled by the growing availability of commercial smart insoles [25, 27–29], enabling practical out-of-lab use. These signals improve motion plausibility when fused with other inputs [39,46] and can even support standalone capture [40,41], tho","cbCaisblDqnDpwaH","https://ap.wps.com/l/cbCaisblDqnDpwaH","pdf",2392830,1,19,"English","en",105,"# Introduction\n## Motivation and limitations of existing MoCap\n## Toward adaptive real-world wearable capture\n# Related Work\n## Datasets for wearable motion capture","[{\"question\":\"What problem does the work target in wearable motion capture?\",\"answer\":\"It reconstructs full-body motion from any subset of sensors worn at a given moment, avoiding assumptions about fixed hardware configurations.\"},{\"question\":\"What dataset is introduced, and what does it include?\",\"answer\":\"A large-scale multimodal dataset synchronizes consumer wearables (e.g., smartphones, smartwatches, pressure-sensing insoles, and a VR headset) with high-quality ground-truth 3D motion from a calibrated markerless 120-camera system across 50 activities.\"},{\"question\":\"How does WHIP reconstruct motion when sensors are missing?\",\"answer\":\"WHIP is a flow-matching generative model that conditions on whichever modalities are available, using a DiT-style backbone with per-modality cross-attention and producing temporally consistent, physically plausible motions under missing modalities.\"}]",1784205739,48,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"towards-real-world-wearable-motion-reconstruction","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/towards-real-world-wearable-motion-reconstruction/85711/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What problem does the work target in wearable motion capture?","Question",{"text":74,"@type":75},"It reconstructs full-body motion from any subset of sensors worn at a given moment, avoiding assumptions about fixed hardware configurations.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"What dataset is introduced, and what does it include?",{"text":79,"@type":75},"A large-scale multimodal dataset synchronizes consumer wearables (e.g., smartphones, smartwatches, pressure-sensing insoles, and a VR headset) with high-quality ground-truth 3D motion from a calibrated markerless 120-camera system across 50 activities.",{"name":81,"@type":72,"acceptedAnswer":82},"How does WHIP reconstruct motion when sensors are missing?",{"text":83,"@type":75},"WHIP is a flow-matching generative model that conditions on whichever modalities are available, using a DiT-style backbone with per-modality cross-attention and producing temporally consistent, physically plausible motions under missing modalities.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},"General","general"]