[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85874-en":3,"doc-seo-85874-105":29,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85874,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","InterPet4D Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation","InterPet4D addresses the lack of high-quality large-scale data for human–pet interaction estimation and motion generation by introducing a multimodal 4D dataset focused on natural human–dog obedience behaviors. A synchronized multi-view capture system records 6.8 million frames from 13 dogs across 11 breeds with 23 human participants, providing multiview and egocentric videos, segmentations, 2D/3D keypoints, meshes, and audio tracks. An InterPetMoGen MotionGPT-style framework further generates realistic 3D dog motion conditioned on gestures and audio, achieving an FID of 11.21.","InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation  \nYichen Peng* 1 , Jyun-Ting Song* 1 ,2 , Chen-Chieh Liao* 1 , Kris Kitani2 , Hideki Koike 1 , and Erwin Wu 1  \n1 Institute of Science Tokyo  \n2 Carnegie Mellon University  \narXiv :2607 . 10287v1 [ cs .CV] 11 Jul 2026  \n“Ruby, sit down!”  \nMultiview Video Egoview Video Audio Command  \nHuman Body & Hand & Dog’s Pose  \nFig. 1: The InterPet4D multimodal Human-Pet Interaction Dataset.  \nAbstract. Human-pet interaction estimation and generation remain underexplored due to the absence of high-quality large-scale dataset. We present InterPet4D, the first multimodal dataset capturing natural interactions between humans and dogs. Using a synchronized multi-view capture system, we record human–dog obedience tasks and provide annotations for both humans and dogs, including multiview and egocentric videos, segmentations, 2D/3D keypoints, meshes, and audio tracks. Interpet4D consists of 6.8 million frames collected from 13 dogs of 11 breeds interacting with 23 human participants. We further introduce the InterPetMoGen framework for human-pet interaction motion generation. Our proposed model achieves an FID score of 11.21, substantially outperforms the Seq2Seq or DiT baselines, demonstrating the effectiveness of Interpet4D for modeling realistic human–pet interactions.  \n* Equal contribution.  \n2 Y. Peng et al.  \n1 Introduction  \nHuman–animal interactions exhibit temporally coordinated and mutually responsive movement patterns, where the motion of one agent directly influencesand adapts to the other. Modeling such structured relationships is fundamental for understanding cross-species behavior and has important applications in socially aware robotics, virtual agents, animation, and behavioral analysis.  \nHowever, most existing research on interactive behavior has focused on human–human or human–object interactions, largely due to the absence of largescale and realistic human-animal datasets. Capturing such interaction data presents unique challenges, as it requires structured and repeatable coordination between human and animal participants, which is difficult to achieve without controlled training, making large-scale data collection particularly challenging. Furthermore, close-range human–animal interactions can result in severe cross-occlusion, which degrades the reconstruction of fine-grained details on the participants.  \nTo address these challenges, we introduce InterPet4D, the first large-scale multimodal 4D dataset of naturalistic human-dog interactions. Our dataset captures synchronized multiview, egocentric, audio, and 3D motion data from 23 participants and 13 dogs. We design a systematic interaction protocol covering 4 categories of common interactions: petting, commanding, calling, and free-form, enabling structured analysis of dog behavior. Figure 1 provides an overview of the InterPet4D dataset, including multiview video, egocentric video, audio commands, and reconstructed human and dog motion. To facilitate future research, we release InterPet4D on Hugging Face: [https://huggingface.co/datasets/](https://huggingface.co/datasets/)[ ](https://huggingface.co/datasets/)[ohicarip/interpet4d](ohicarip/interpet4d.)[.](ohicarip/interpet4d.)  \nBeyond the dataset, we propose InterPetMoGen (IPMG), a Motion GPTbased framework for gesture-to-pet motion generation. Given a sequence of human hand/body gestures and accompanying audio, IPMG generates plausible 3D dog motion responses conditioned on human gestures and audio. The model adopts a MotionGPT-style autoregressive transformer that predicts discrete pet motion tokens learned by a PetVAE tokenizer. We further introduce modalityaware attention (MAA) masks that enable coarse-to-fine motion generation by combining bidirectional conditioning with causal autoregressive decoding.  \nOur contributions are summarized as follows:  \n– We introduce InterPet4D, the first large-scale multimodal 4D dataset of human–pe","cbCailkErcfPyFfw","https://ap.wps.com/l/cbCailkErcfPyFfw","pdf",3650687,1,24,"English","en",105,"# Introduction\n# Related Work\n## Human-centered Interaction Datasets\n## Animal Pose and Shape Estimation","[{\"question\":\"How does InterPetMoGen generate pet motion from human inputs?\",\"answer\":\"InterPetMoGen is a MotionGPT-style autoregressive transformer that predicts discrete pet motion tokens learned by a PetVAE tokenizer, using modality-aware attention to combine gesture/body and audio conditioning for realistic 3D dog motion responses.\"}]",1784206864,60,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":27},"interpet4d-multimodal-4d-human-pet-interaction-dataset-for-pet-motion-generation","",{"@graph":35,"@context":77},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/interpet4d-multimodal-4d-human-pet-interaction-dataset-for-pet-motion-generation/85874/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"How does InterPetMoGen generate pet motion from human inputs?","Question",{"text":75,"@type":76},"InterPetMoGen is a MotionGPT-style autoregressive transformer that predicts discrete pet motion tokens learned by a PetVAE tokenizer, using modality-aware attention to combine gesture/body and audio conditioning for realistic 3D dog motion responses.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,101,106,111,114,119,122,126],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":45,"category_name":99,"show_sort_weight":28,"slug":100},5,"Comic","comic",{"id":102,"doc_module":4,"doc_module_name":45,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":45,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":45,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":98,"slug":129},19,"General","general"]