[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84529-en":3,"doc-seo-84529-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84529,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","Segmenting Fast and Slow Real-Time Open-Vocabulary Video Instance Segmentation with Dual-Path Processing","Object-centric open-vocabulary video instance segmentation inspired by DETR faces a key barrier: real-time on-device inference at high frame rates remains difficult due to the computational cost of dense mask prediction and multimodal fusion. This paper presents SegFS, a dual-stream fast–slow framework that improves efficiency by using a sparse keyframe open-vocabulary object model, then projecting instance representations into backbone feature space to condition a lightweight fast network. Experiments show up to 14× lower latency than MOBIUS while maintaining competitive OV-VIS accuracy.","arXiv :2607 .00124v1 [ cs .CV] 30 Jun 2026  \nSegmenting, Fast and Slow: Real-Time Open-Vocabulary Video Instance Segmentation with Dual-Path Processing  \nLuca Barsellotti3⋆ Martin Sundermeyer 1 Mattia Segu 1 Nikita Araslanov2 Muhammad Ferjad Naeem 1 Marcella Cornia3 Yongqin Xian 1 Maxim Berman 1  \n1 Google  \n2 TU Munich; Munich Center for Machine Learning  \n3 University of Modena and Reggio Emilia  \nAbstract. Object-centric models inspired by DETR have become the dominant paradigm for open-vocabulary video instance segmentation (OV-VIS) . While recent efforts have reduced the computational cost of pixel decoding, textual modality fusion, and object decoding to make these architectures more suitable for mobile devices, real-time on-device inference at high frame rates remains an open challenge. In this paper, we introduce SegFS, a dual-stream fast-slow framework that significantly improves efficiency without sacrificing accuracy. On sparse keyframes, an open-vocabulary object-based model predicts instance-level representations. These representations are then projected back into the backbone feature space to condition a lightweight fast network, which efficiently relocalizes and segments the instances in subsequent frames. By shifting instance propagation from object decoding to feature-space conditioning, our approach decouples multimodal semantic understanding from dense mask prediction and enables efficient temporal propagation. The proposed fast branch achieves up to 14 × lower latency than the mobileoriented MOBIUS model, while maintaining competitive segmentation performance on standard OV-VIS benchmarks.  \nKeywords: Real-Time · Open-Vocabulary · Mobile Device · Video Instance Segmentation  \n1 Introduction  \nWhile modern deep learning models exhibit remarkable accuracy and scalability, their substantial computational overhead heavily restricts real-time deployment on edge devices. This bottleneck becomes particularly severe for dense spatiotemporal tasks. In this work, we focus on Open-Vocabulary Video Instance Segmentation (OV-VIS), which aims at identifying and tracking object masks of open-set categories specified by text prompts. Established architectures, such as GLEE [39], comprise three main vision components: (i) a visual backbone that  \n⋆ Work done during an internship at Google.  \n2 L. Barsellotti et al.  \nLatency (ms) FLOPS (G)  \n15  \n10  \n5  \n0  \n60  \n40  \n20  \n0  \nMOBIUS-Mini-M  \n40  \n20  \n0  \n400  \n200  \n0  \nGLEE-Lite  \n30  \n20  \n10  \n0  \n60  \n40  \n20  \n0  \nTROY-VIS  \nBackbone  \n Feature Enhancer  Object Decoder  SegFS (Ours)  \nFig. 1: Efficiency analysis of OV-VIS architectures. We compare the FLOPs (top) and on-device latency on a Samsung Galaxy S25 Ultra (bottom) for the core components of three baseline models, MOBIUS-Mini-M [31], GLEE-Lite [39], and TROY-VIS [36], when assuming an input image with resolution 480 ×480 and 40 textual categories, and the Fast Feature Aggregation from our proposed SegFS.  \nproduces multi-scale features; (ii) a feature enhancer that fuses the multi-scale features in a single high-resolution feature map and injects the text embeddingsto guide the segmentation process toward identifying corresponding instances; and (iii) an object decoder that extracts representative feature vectors describing the identified instances. Among these, the feature enhancer represents the most severe computational bottleneck.  \nRecent works, such as MOBIUS [31] and TROY-VIS [36], have introduced lighter and more mobile-friendly feature enhancer modifications. However, such components still overwhelmingly dominate the inference cost: Fig. 1 visualizes the FLOPS and the on-device latency on a Samsung Galaxy S25 Ultra of the components of MOBIUS-Mini-M, GLEE-Lite, and TROY-VIS. This dominance is even more pronounced in on-device latency than in theoretical FLOPs, and scales prohibitively with higher image resolutions and vocabulary size.  \nTo overcome this bottleneck, we draw inspiration from keyframe-","cbCaicxBMOzUQHYm","https://ap.wps.com/l/cbCaicxBMOzUQHYm","pdf",13985508,1,25,"English","en",105,"# Abstract\n# Introduction\n## Problem: real-time OV-VIS on edge devices\n## Existing OV-VIS architectures and bottlenecks\n## Keyframe amortization idea\n# SegFS Framework Overview\n## Slow path on sparse keyframes\n## Fast path on every frame via feature-space conditioning","[{\"question\":\"What challenge does OV-VIS face when deploying on edge or mobile devices?\",\"answer\":\"The heavy computational overhead of dense spatiotemporal processing limits real-time deployment at high frame rates. The feature enhancer is identified as a major inference bottleneck, especially as resolution and vocabulary size increase.\"},{\"question\":\"How does SegFS improve efficiency without losing accuracy?\",\"answer\":\"SegFS uses a dual-path design: the slow path runs on sparse keyframes to compute instance representations, then projects them back into backbone feature space to condition a lightweight fast network. The fast path performs efficient instance relocalization and segmentation on subsequent frames.\"},{\"question\":\"What performance gains does SegFS report compared with mobile-oriented baselines?\",\"answer\":\"SegFS achieves up to 14× lower latency than the MOBIUS-Mini-M model and delivers about 3× amortized FPS, crossing the 30 FPS real-time threshold while preserving competitive segmentation performance (≤1 AP relative to the reference upper bound on standard benchmarks).\"}]",1784196428,63,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"segmenting-fast-and-slow-real-time-open-vocabulary-video-instance-segmentation-with-dual-path-processing","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/segmenting-fast-and-slow-real-time-open-vocabulary-video-instance-segmentation-with-dual-path-processing/84529/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What challenge does OV-VIS face when deploying on edge or mobile devices?","Question",{"text":75,"@type":76},"The heavy computational overhead of dense spatiotemporal processing limits real-time deployment at high frame rates. The feature enhancer is identified as a major inference bottleneck, especially as resolution and vocabulary size increase.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does SegFS improve efficiency without losing accuracy?",{"text":80,"@type":76},"SegFS uses a dual-path design: the slow path runs on sparse keyframes to compute instance representations, then projects them back into backbone feature space to condition a lightweight fast network. The fast path performs efficient instance relocalization and segmentation on subsequent frames.",{"name":82,"@type":73,"acceptedAnswer":83},"What performance gains does SegFS report compared with mobile-oriented baselines?",{"text":84,"@type":76},"SegFS achieves up to 14× lower latency than the MOBIUS-Mini-M model and delivers about 3× amortized FPS, crossing the 30 FPS real-time threshold while preserving competitive segmentation performance (≤1 AP relative to the reference upper bound on standard benchmarks).","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]