[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84422-en":3,"doc-seo-84422-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84422,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","On Occlusions in Video Action Detection Benchmark Datasets And Training Recipes","Research examines occlusions in video action detection and their effect on model performance. The work introduces five benchmark datasets: O-UCF and O-JHMDB with synthetically controlled static/dynamic occlusions, OVIS-UCF and OVIS-JHMDB with realistic occluder motions, and Real-OUCF targeting realistic-world scenarios. Results confirm that performance degrades with increasing occlusion severity and differs between static and moving occluders. Neural-network behaviors are analyzed, enabling inductively robust training recipes that improve occlusion handling and yield strong vMAP gains over prior detectors.","arXiv :2410 . 19553v2 [ cs .CV] 10 Jul 2026  \nOn Occlusions in Video Action Detection: Benchmark Datasets And Training Recipes  \nRajat Modi 1 ∗, Vibhav Vineet2 , Yogesh Singh Rawat 1 CRCV, University of Central Florida 1 , and Microsoft Research2  \nAbstract  \nThis paper explores the impact of occlusions in video action detection. We facilitate this study by introducing five new benchmark datasets namely O-UCFand O-JHMDB consisting of synthetically controlled static/dynamic occlusions, OVIS-UCF and OVIS-JHMDB consisting of occlusions with realistic motions and Real-OUCF for occlusions in realistic-world scenarios. We formally confirm an intuitive expectation: existing models suffer a lot as occlusion severity is increased and exhibit different behaviours when occluders are static vs when they are moving.  \nWe discover several intriguing phenomenon emerging in neural nets: 1) transformers can naturally outperform CNN models which might have even used occlusion asa form of data augmentation during training 2) incorporating symbolic-components like capsules to such backbones allows them to bind to occluders never even seen during training and 3) Islands of agreement can emerge in realistic images/videos without instance-level supervision, distillation or contrastive-based objectives2 (eg.  \nvideo-textual training) . Such emergent properties allow us to derive simple yet effective training recipes which lead to robust occlusion models inductively satisfying the first two stages of the binding mechanism (grouping/segregation) . Models leveraging these recipes outperform existing video action-detectors under occlusion by 32.3% on O-UCF, 32.7% on O-JHMDB & 2.6% on Real-OUCF in terms of the vMAP metric. The code for this work has been released at [https:](https:)//[github.com/rajatmodi62/OccludedActionBenchmark](github.com/rajatmodi62/OccludedActionBenchmark).  \n1 Introduction  \nDeep learning[41] has led to significant advances in object detection/segmentation for both image[16, 19] and video domain[84] . Such deep neural networks are in turn widely used in self-driving cars and safety critical scenarios. A key concern for such applications is whether they are able to perform well when encountering realistic occlusions: e.g. are they able to reliably localize a pedestrian even when an occluder (say a dog) comes in front of him. However, one major limitations being existing dataset test split doesn’t contain such occlusions. This raises a concern, whether these models will be robust to real-world occlusions or not.  \nOne would suspect that the inherent inductive biases of these architectures would be enough to induce natural occlusion robustness. To verify the hypothesis, we run two preliminary setups: (Fig. 1)(A) Firstly, superimposing a single occluder (eg bus) over the actor, and,(B) Then, we analyze the performance if the occlusion is shifted to background (i.e. no occlusion over actor at all) . We observe a relative drop of 20-50% across multiple existing state-of-the-art approaches [46, 13] . This verifies our hypothesis that existing approaches are not robust to occlusion. It opens up the area that there is a need to study these models behaviour under different types and severity of occlusion.  \n∗ Corresponding Author, email: [rajatmodi@ucf.edu](rajatmodi@ucf.edu)  \n2 Grouping pixels is a perceptual phenomenon of the visual-cortex[59] . Assigning a group to a class is a property of language. If a group is identified by multiple-names, then a neural-net can learn this one-many  \n37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.  \nFigure 1: A Toy Experiment. (i-iii) superimposing a single occluder (bus) on an actor and varying its size results in drops as large as 50% . (iv) even a simple occluder (cat) in background results in 30% drop. (v) highest drops are observed if background is entirely masked. (vi-ix) Clean refers to all methods evaluated on unoccluded test sets. Best ","cbCaimHpiYAxPXnn","https://ap.wps.com/l/cbCaimHpiYAxPXnn","pdf",23212144,1,30,"English","en",105,"# Abstract\n# Introduction\n## Motivation and key observations\n# Related Work","[{\"question\":\"What is the main focus regarding occlusions in video action detection?\",\"answer\":\"The paper studies how occlusions affect spatio-temporal video action detection, including the impact of occlusion severity and whether occluders are static or moving.\"},{\"question\":\"What benchmark datasets are introduced in this work?\",\"answer\":\"It introduces O-UCF and O-JHMDB for synthetically controlled static/dynamic occlusions, OVIS-UCF and OVIS-JHMDB for realistic occluder motions, and Real-OUCF for occlusions in realistic-world scenarios.\"},{\"question\":\"How do the proposed training approaches improve robustness under occlusion?\",\"answer\":\"The work finds augmentations help across models and develops a transformer token-masking strategy, producing robust occlusion models that outperform existing detectors under occlusion.\"}]",1784195541,76,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"on-occlusions-in-video-action-detection-benchmark-datasets-and-training-recipes","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/on-occlusions-in-video-action-detection-benchmark-datasets-and-training-recipes/84422/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main focus regarding occlusions in video action detection?","Question",{"text":75,"@type":76},"The paper studies how occlusions affect spatio-temporal video action detection, including the impact of occlusion severity and whether occluders are static or moving.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What benchmark datasets are introduced in this work?",{"text":80,"@type":76},"It introduces O-UCF and O-JHMDB for synthetically controlled static/dynamic occlusions, OVIS-UCF and OVIS-JHMDB for realistic occluder motions, and Real-OUCF for occlusions in realistic-world scenarios.",{"name":82,"@type":73,"acceptedAnswer":83},"How do the proposed training approaches improve robustness under occlusion?",{"text":84,"@type":76},"The work finds augmentations help across models and develops a transformer token-masking strategy, producing robust occlusion models that outperform existing detectors under occlusion.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":21,"slug":121},"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]