[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-327149-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-327149-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","cross4d-jepa-dense-cross-modal-correspondence-distillation-for-4d-point-cloud-representation-learning","Cross4D-JEPA - Dense Cross-modal Correspondence Distillation for 4D Point Cloud Representation Learning","","Automatic understanding of dynamic 4D point clouds, capturing 3D point sequences over time with depth sensors and LiDAR, is vital for robotics and embodied perception, yet dense annotation is costly. Self-supervised pretraining offers transfer-ready representations, but most existing 4D pretext tasks remain intra-modal or compress 2D knowledge into single global embeddings. Cross4D-JEPA distills frozen 2D/2D-video teacher features into a 4D point encoder using dense correspondence and a per-point latent matching objective. Evaluations across four benchmarks show consistent gains and strong cross-domain transfer.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/cross4d-jepa-dense-cross-modal-correspondence-distillation-for-4d-point-cloud-representation-learning/327149/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/cross4d-jepa-dense-cross-modal-correspondence-distillation-for-4d-point-cloud-representation-learning/327149.png","ImageObject",300,407,{"name":42,"@type":43},"Patrick","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-29","2026-09-21",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",20,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does Cross4D-JEPA address in 4D point cloud representation learning?","Question",{"text":63,"@type":64},"Cross4D-JEPA addresses the lack of dense supervision from appearance and object semantics in existing 4D self-supervised pretraining, and the coarse nature of prior 2D-to-point transfer that collapses patch features into a single global embedding.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does Cross4D-JEPA perform cross-modal distillation at dense granularity?",{"text":68,"@type":64},"It renders each frame, records per-pixel nearest-point indices, maps teacher patch features onto the corresponding 3D points, and trains the student with a per-point latent objective that matches teacher embeddings without masking, negatives, or a decoder.",{"name":70,"@type":61,"acceptedAnswer":71},"What results does the paper report on benchmark tasks?",{"text":72,"@type":64},"Under a matched protocol, Cross4D-JEPA consistently outperforms intra-modal and global cross-modal baselines across MSR-Action3D, DeformingThings4D, NTU-RGB+D 60, and HOI4D, and shows improved label efficiency and full-label fine-tuning while using a much smaller encoder.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},327149,1790180207,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":55,"slug":118},9,"Religion & Spirituality","religion-spirituality",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":55,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":109,"preview_page_count":4},549758146520,"https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470","Cross4D-JEPA: Dense Cross-modal Correspondence Distillation for 4D Point Cloud Representation Learning  \nTrung Thanh Nguyen 1 , Hai Nguyen-Truong2 , Tu Vo3 , Hoang M. Truong4 , and Tuan-Anh Vu5 *  \n1Nagoya University, Japan 2Northeastern University, USA 3 KC Machine Learning Lab, Korea  \n4University of Science, Vietnam National University Ho Chi Minh City, Vietnam  \n5University of California, Los Angeles, USA  \narXiv :2607 .005 14v 1 [ cs .CV] 1 Jul 2026  \nAbstract  \nAutomatic understanding of dynamic 4D point clouds, the 3D-point sequences captured over time by depth sensors and LiDAR, is central to robotics and embodied perception. Yet annotating them densely is expensive, making selfsupervised pretraining the natural route to transferable representations. Existing pretext tasks, however, are almost entirely intra-modal, and the few methods that transfer knowledge from 2D foundation models rely on a single global embedding per clip, discarding the rich per-patch semantics that these models compute. To address this gap, we propose Cross4D-JEPA, a teacher–student method that distills a frozen 2D foundation model, an image model DINOv2, or a video model V-JEPA 2, into a 4Dpoint encoder. The proposed method combines (1) a dense cross-modal correspondence that maps every 3D point to the teacher patch feature it projects to, and (2) a per-point objective that trains the student to match these features in latent space with no masking, negatives, or decoder. We evaluate Cross4D-JEPA on four benchmarks, MSR-Action3D, DeformingThings4D, NTU-RGB+D 60, and HOI4D, against intra-modal and global cross-modal baselines. Experimental results show that, under a matched protocol, the proposed method consistently outperforms intra-modal and global cross-modal baselines across the four benchmarksand is competitive with heavier published 4D methods; further analysis attributes this gain primarily to the granularity of the correspondence rather than the teacher modality. Beyond recognition accuracy, the dense representation learned by Cross4D-JEPA transfers across domains, improves label efficiency, and improves full-label fine-tuning under the same training budget, while a 13 × smaller encoder matches a heavyweight pooling backbone.  \n* Corresponding author: [tuananh.vu@ucla.edu](tuananh.vu@ucla.edu).  \n1. Introduction  \nAutomatic understanding of dynamic 4D point clouds, the sequences of 3D points captured over time by depth sensors, LiDAR, and multi-view rigs, is central to robotics and embodied perception [8, 9, 40], where recognizing actions, interactions, and deformations [14–16, 31] underlies downstream decisions. However, dense temporal labels are expensive to obtain, making self-supervised pretraining the natural route to transferable 4D representations [10, 32, 48] .  \nYet existing pretext tasks for 4D point clouds are almost entirely intra-modal. Spatio-temporal contrastive learning [6, 10, 33], masked structure prediction [32, 48], and complete-to-partial geometry distillation [45] all supervise the encoder from the point clouds alone, and never inject the appearance and object semantics that large 2D models already capture. The one method that transfers from 2D video into a point encoder, CrossVideo [18], contrastively matches a single global embedding per clip. This signal is coarse, collapsing a structured 2D feature map into a single vector and discarding where each feature came from.  \nThis is a missed opportunity, as the richest 2D foundation models are precisely those with spatially dense, perpatch features that a single global embedding collapses. DINOv2 [22] produces a patch-level feature map whose local descriptors support dense correspondence and segmentation without any fine-tuning. Static cross-modal distillation already exploits this property, as CrossJEPA [25] and Concerto [44] distill an image model into a static point encoder by predicting target embeddings in latent space [2], the “beyond-masking” recipe that also und","cbCaibOrFpPCVznR","https://ap.wps.com/l/cbCaibOrFpPCVznR","pdf",1773670,16,"English","# Abstract\n# 1. Introduction\n## Motivation: dynamic 4D point clouds and costly dense labels\n## Limitations of existing intra-modal and global cross-modal methods\n## Proposed method: Cross4D-JEPA and its dense correspondence objective\n## Main contributions and benchmark evaluations","[{\"question\":\"What problem does Cross4D-JEPA address in 4D point cloud representation learning?\",\"answer\":\"Cross4D-JEPA addresses the lack of dense supervision from appearance and object semantics in existing 4D self-supervised pretraining, and the coarse nature of prior 2D-to-point transfer that collapses patch features into a single global embedding.\"},{\"question\":\"How does Cross4D-JEPA perform cross-modal distillation at dense granularity?\",\"answer\":\"It renders each frame, records per-pixel nearest-point indices, maps teacher patch features onto the corresponding 3D points, and trains the student with a per-point latent objective that matches teacher embeddings without masking, negatives, or a decoder.\"},{\"question\":\"What results does the paper report on benchmark tasks?\",\"answer\":\"Under a matched protocol, Cross4D-JEPA consistently outperforms intra-modal and global cross-modal baselines across MSR-Action3D, DeformingThings4D, NTU-RGB+D 60, and HOI4D, and shows improved label efficiency and full-label fine-tuning while using a much smaller encoder.\"}]","Cross4D-JEPA - Dense Cross-modal Correspondence Distillation for 4D Point Cloud Representation Learning | PDF",1789976551]