[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84153-en":3,"doc-seo-84153-105":29,"detail-sidebar-cat-0-en-105":82},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84153,687197207057,"Sage","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation","ProxyPose addresses six-degree-of-freedom (6-DoF) pose tracking from monocular video by reframing the task as video-to-video translation. With only an input video and a single marked pixel in the first frame, a fine-tuned video diffusion model generates a synthetic proxy video showing a known colored polyhedron executing the same local rigid-body motion. Recovering the complete 6-DoF trajectory becomes classical pose estimation using standard solvers. The method avoids 3D models, depth, masks, identity assumptions, or global rigidity, absorbs challenging occlusions and deformations in the translation step, and extends to face tracking and camera pose in unconstrained scenes.","arXiv :2607 .06555v 1 [ cs .CV] 7 Jul 2026  \nProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation  \nRuihang Zhang∗1, Felix Taubner∗1 ,2 , Pooja Ravi 1 , Kiriakos N. Kutulakos 1 ,2 , David B. Lindell 1 ,2  \n1University of Toronto 2Vector Institute  \n* Equal contribution  \ninitial frame & query pixel original video frames, overlaid 6-DoF poses & generated proxy frames 6-DoF pose track  \nFigure 1: Our approach enables tracking relative 6-DoF pose in diverse, highly dynamic scenes without employing foundation models or 3D inference pipelines—including the top of a swinging baseball bat (top); individual regions on a porcelain vase as it fractures (middle); and even Captain Pete’s hand in this cartoon clip from Steamboat Willie (bottom) . Highlighted points are the proxy cube’s center. We use this core capability to track multiple surface regions as they move, deform or occlude each other in unconstrained internet videos (see the videos on the Project Webpage for several such examples) .  \nAbstract  \nTracking the six-degree-of-freedom (6-DoF) pose of objects and surfaces from monocular video is a long-standing problem in computer vision. To tackle this problem, existing methods require inputs beyond the video itself—such as 3D models, depth maps, object masks, or task-specific learned features—and they struggle with textureless, transparent, reflective, or deformable surfaces. Here, we introduce ProxyPose, which recasts 6-DoF pose tracking as video-to-video translation. Given only a video and a single marked pixel in the first frame, a finetuned video diffusion model translates the input into a proxy video—a synthetic  \nvideo depicting a colored polyhedron undergoing the same local rigid-body motion as the surface region at the marked pixel. Because the proxy’s geometry and appearance are known by construction, recovering its full 6-DoF trajectory reduces to classical pose estimation with off-the-shelf solvers. This formulation leverages large-scale video pre-training to absorb the hardest aspects of pose tracking—handling challenging materials, occlusions, and deformations—into the translation step, while operating at the pixel level with no assumptions about object identity, boundaries, or global rigidity. ProxyPose achieves state-of-the-art 6-DoF pose tracking accuracy without the additional inputs required by competing methods and after fine-tuning the video model only on synthetic data. We further demonstrate that ProxyPose extends to face tracking, camera pose estimation, and challenging in-the-wild scenes that are beyond the reach of existing approaches. Video results are available on the Project Webpage.  \n1 Introduction  \nTracking the six-degree-of-freedom (6-DoF) pose of objects and surfaces from video is a fundamental problem in computer vision that traces its roots back more than thirty years. Classical techniques spanned a range of model-based approaches, from methods that aligned rigid CAD models to images (Lowe, 1991; Harris, 1993; Drummond and Cipolla, 2002; Comport et al., 2006) to deformabletemplate and statistical-shape approaches developed primarily for non-rigid face and human-body tracking (Cootes et al., 1998; Bregler and Malik, 1998; Blanz and Vetter, 1999; DeCarlo and Metaxas, 2000), to tracking-by-detection methods which leveraged discriminative classifiers or convolutional neural networks (Shotton et al., 2011; Hinterstoisser et al., 2012; Kehl et al., 2017; Xiang et al., 2018) . A complementary line of research combined 2D feature detection (Lucas and Kanade, 1981; Shi and Tomasi, 1994; Lowe, 2004) with structure from motion (SfM) to simultaneously recover 3D models and pose of rigid (Fitzgibbon and Zisserman, 1998) or non-rigid (Torresani et al., 2001) scenes by global bundle adjustment (Triggs et al., 1999; Agarwal et al., 2009) or sequential state estimation (Davison et al., 2007; Engel et al., 2014; Mur-Artal and Tardós, 2017) . This general approach has been extended to include neural-netwo","cbCaitGZLJMh923k","https://ap.wps.com/l/cbCaitGZLJMh923k","pdf",45944219,1,23,"English","en",105,"# Abstract\n# Introduction","[{\"question\":\"What limitations of existing 6-DoF tracking approaches does ProxyPose aim to overcome?\",\"answer\":\"ProxyPose avoids additional inputs such as 3D models, depth maps, object masks, or task-specific features, and improves robustness on challenging materials, occlusions, and deformations while working at the pixel level without assumptions about object identity or boundaries.\"}]",1784193478,58,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":77,"head_meta":79,"extra_data":81,"updated_unix":27},"proxypose-6-dof-pose-tracking-via-video-to-video-translation","",{"@graph":35,"@context":76},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/proxypose-6-dof-pose-tracking-via-video-to-video-translation/84153/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70],{"name":71,"@type":72,"acceptedAnswer":73},"What limitations of existing 6-DoF tracking approaches does ProxyPose aim to overcome?","Question",{"text":74,"@type":75},"ProxyPose avoids additional inputs such as 3D models, depth maps, object masks, or task-specific features, and improves robustness on challenging materials, occlusions, and deformations while working at the pixel level without assumptions about object identity or boundaries.","Answer","https://schema.org",{"og:url":51,"og:type":78,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":80,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":83},[84,88,92,96,101,106,111,114,119,122,126],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":45,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":45,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general"]