[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82067-en":3,"doc-seo-82067-105":29,"detail-sidebar-cat-0-en-105":89},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82067,13056703019404,"Miles","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","AgenticFocus Object Preserving Mixed Reality Synthesis from Human FPV Video for Dexterous Humanoid Learning","Human egocentric video provides scalable supervision for humanoid policy learning, yet existing conversion pipelines struggle with hand–object occlusion, over-simplified motion, and reliance on specialized capture hardware. AgenticFocus proposes an object-preserving Mixed Reality synthesis pipeline that restores occluded object geometry, reconstructs full-hand motion, and retargets it to a humanoid via camera-relative alignment and layered compositing. The produced dataset aligns focused visual observations with synchronized robot actions and states. It yields lower trajectory error and smoother wrist motion than cross-embodiment baselines, including SPARC score improvements (−5.18 vs −5.56 and −6.05).","AgenticFocus: Object-Preserving Mixed Reality Synthesis from Human FPV Video for Dexterous Humanoid Learning  \nIaroslav Kolomiets 1,2 * Miguel Altamirano Cabrera 1,2† Artem Lykov 1,2‡ Jeffrin Sam 1§ Dmitrii Iarchuk 1,2¶ Yara Mahmoud 1|| Daniia Zinniatullina 1,2 ** Mikhail Konenkov 1,2††  \nDzmitry Tsetserukou 1‡‡  \narXiv :2607 .08857v 1 [ cs .RO] 9 Jul 2026  \n1 Intelligent Space Robotics Lab, Skolkovo Institute of Science and Technology, Moscow, Russian Federation  \n2 R&D Center, MWS, Moscow, Russian Federation  \nFigure 1: AgenticFocus converts human FPV videos into robot-trainable mixed-reality demonstrations by restoring occluded objects, retargeting full-hand motion, and producing synchronized visual-action data.  \nABSTRACT  \nHuman egocentric video is a scalable supervision source for humanoid policy learning, but current pipelines struggle with handobject occlusion, oversimplified motion, or specialized capture hardware. We introduce AgenticFocus, a Mixed Reality synthesis pipeline that converts ordinary first-person-view human videos into robot-trainable demonstrations by restoring occluded object geometry, reconstructing full-hand motion, and retargeting it to a humanoid embodiment through camera-relative alignment and layered compositing. The resulting dataset pairs focused visual observations with synchronized robot actions and states. AgenticFocus achieves lower trajectory error and smoother wrist motion than cross-embodiment baselines, with SPARC scores of −5 . 18 versus −5 .56 and −6 .05.  \nIndex Terms: Mixed Reality, Diminished Reality, Human-toRobot Demonstration, Dexterous Manipulation, Humanoid Robots, Cross-Embodiment Retargeting.  \n1 INTRODUCTION  \nRecent progress in Physical AI has made one limitation increasingly clear: the next frontier is not only better models but also bet-  \n* e-mail: [iaroslav.kolomiets@skoltech.ru](iaroslav.kolomiets@skoltech.ru)[ ](iaroslav.kolomiets@skoltech.ru)†[e-mail: m.altamirano@skoltech.ru](e-mail: m.altamirano@skoltech.ru)[ ](e-mail: m.altamirano@skoltech.ru)‡[e-mail: Artem.Lykov@skoltech.ru](e-mail: Artem.Lykov@skoltech.ru)  \n§ e-mail: [jeffrin.sam@skoltech.ru](jeffrin.sam@skoltech.ru)[ ](jeffrin.sam@skoltech.ru)[e-mail: Dmitrii.Iarchuk@skoltech.ru](e-mail: Dmitrii.Iarchuk@skoltech.ru)  \n[e-mail: Yara.Mahmoud@skoltech.ru](e-mail: Yara.Mahmoud@skoltech.ru)[ ](e-mail: Yara.Mahmoud@skoltech.ru)** e-mail: [daniia.zinniatulina@skoltech.ru](daniia.zinniatulina@skoltech.ru)[ ](daniia.zinniatulina@skoltech.ru)††[e-mail: mikhail.konenkov@skoltech.ru](e-mail: mikhail.konenkov@skoltech.ru)[ ](e-mail: mikhail.konenkov@skoltech.ru)‡‡e-mail: d.tsetserukou@skoltech.ru  \nter supervision. Across Vision-Language-Action (VLA) policies, world-action models, and related visuomotor architectures, performance ultimately depends on access to large amounts of robotusable visual and action data [3, 8, 23] . Yet for dexterous humanoids, such data remains exceptionally difficult to obtain. Teleoperation is costly, wearable capture systems are cumbersome, and embodiment-specific data collection does not scale across platforms [16, 22, 6] . By contrast, human egocentric video is already abundant: it captures diverse object interactions in natural environments, often at internet scale, and therefore represents one of the most promising untapped supervision sources for humanoid manipulation [4, 6, 9] .  \nThe challenge is that human video is not robot data. Converting a first-person human demonstration into a training signal for a humanoid policy requires bridging three coupled gaps. The first is a viewpoint gap: human demonstrations are captured from head-or chest-mounted cameras, whereas humanoid observations are defined in robot-centered frames [9, 12, 6] . The second is an interaction-region gap: during dexterous manipulation, the hand frequently occludes the manipulated object, precisely at the contact regions where geometry, boundaries, and local depth relationships are most important for control. Conve","cbCaitn0QA781qZQ","https://ap.wps.com/l/cbCaitn0QA781qZQ","pdf",3374708,1,4,"English","en",105,"# Introduction\n## Motivation and dataset challenge\n## Coupled gaps: viewpoint, interaction-region, and action grounding\n# Related Work","[{\"question\":\"What problem does AgenticFocus address in humanoid learning from human FPV video?\",\"answer\":\"It addresses the difficulty of converting first-person human demonstrations into robot-trainable data when hand–object occlusion, mismatched motion, and robot-grounding alignment cause failures in existing pipelines.\"},{\"question\":\"How does AgenticFocus handle hand–object occlusion in the synthesized demonstrations?\",\"answer\":\"It restores occluded object geometry and uses layered compositing to recover the visual evidence at contact regions needed for control.\"},{\"question\":\"What improvements does AgenticFocus report compared with cross-embodiment baselines?\",\"answer\":\"It reports lower trajectory error and smoother wrist motion, with SPARC score results of −5.18 compared to −5.56 and −6.05.\"}]",1784178007,10,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":84,"head_meta":86,"extra_data":88,"updated_unix":27},"agenticfocus-object-preserving-mixed-reality-synthesis-from-human-fpv-video-for-dexterous-humanoid-learning","",{"@graph":35,"@context":83},[36,52,66],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":21},"https://docshare.wps.com/document/agenticfocus-object-preserving-mixed-reality-synthesis-from-human-fpv-video-for-dexterous-humanoid-learning/82067/",{"url":51,"name":13,"@type":53,"author":54,"headline":13,"publisher":56,"fileFormat":59,"inLanguage":23,"description":14,"dateModified":60,"datePublished":60,"encodingFormat":59,"isAccessibleForFree":61,"interactionStatistic":62},"DigitalDocument",{"name":9,"@type":55},"Person",{"url":40,"name":57,"@type":58},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":63,"interactionType":64,"userInteractionCount":4},"InteractionCounter",{"@type":65},"ViewAction",{"@type":67,"mainEntity":68},"FAQPage",[69,75,79],{"name":70,"@type":71,"acceptedAnswer":72},"What problem does AgenticFocus address in humanoid learning from human FPV video?","Question",{"text":73,"@type":74},"It addresses the difficulty of converting first-person human demonstrations into robot-trainable data when hand–object occlusion, mismatched motion, and robot-grounding alignment cause failures in existing pipelines.","Answer",{"name":76,"@type":71,"acceptedAnswer":77},"How does AgenticFocus handle hand–object occlusion in the synthesized demonstrations?",{"text":78,"@type":74},"It restores occluded object geometry and uses layered compositing to recover the visual evidence at contact regions needed for control.",{"name":80,"@type":71,"acceptedAnswer":81},"What improvements does AgenticFocus report compared with cross-embodiment baselines?",{"text":82,"@type":74},"It reports lower trajectory error and smoother wrist motion, with SPARC score results of −5.18 compared to −5.56 and −6.05.","https://schema.org",{"og:url":51,"og:type":85,"og:title":13,"og:site_name":57,"og:description":14},"article",{"robots":87,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":90},[91,95,99,103,108,113,118,121,126,129,132],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":92,"show_sort_weight":93,"slug":94},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":96,"show_sort_weight":97,"slug":98},"Literature",80,"literature",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":100,"show_sort_weight":101,"slug":102},"Exam",70,"exam",{"id":104,"doc_module":4,"doc_module_name":45,"category_name":105,"show_sort_weight":106,"slug":107},5,"Comic",60,"comic",{"id":109,"doc_module":4,"doc_module_name":45,"category_name":110,"show_sort_weight":111,"slug":112},6,"Technology",50,"technology",{"id":114,"doc_module":4,"doc_module_name":45,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":119,"slug":120},30,"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":28,"doc_module":4,"doc_module_name":45,"category_name":130,"show_sort_weight":28,"slug":131},"Lifestyle","lifestyle",{"id":133,"doc_module":4,"doc_module_name":45,"category_name":134,"show_sort_weight":104,"slug":135},19,"General","general"]