[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82346-en":3,"doc-seo-82346-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82346,1374391974585,"Genevieve","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Multimodal Scenario Similarity Search for Autonomous Driving","Large-scale autonomous-driving datasets contain vast numbers of recorded scenarios, creating a need for efficient retrieval methods that can identify situations similar to a given query. Existing solutions often depend on either visual representations or motion-based descriptions, limiting a clear comparison of their relative strengths. A multimodal framework is proposed that unifies visual and trajectory-based representations for scenario retrieval. Two trajectory approaches, Exo-Trajectory and ScenarioFormer, are evaluated alongside vision baselines. Results show trajectory embeddings excel on motion-centric events, appearance helps when cues are informative, and fusion consistently improves retrieval quality.","Multimodal Scenario Similarity Search for Autonomous Driving  \nTams Matuszka aiMotive  \n[tamas.matuszka@aimotive.com](tamas.matuszka@aimotive.com)  \nAndrs TamsyaiMotive  \n[andras.tamasy@aimotive.com](andras.tamasy@aimotive.com)  \nBalzs SzolraiMotive  \n[balazs.szolar@aimotive.com](balazs.szolar@aimotive.com)  \narXiv :2607 .09428v1 [ cs .CV] 10 Jul 2026  \nAbstract  \nLarge-scale autonomous-driving datasets contain vast numbers of recorded scenarios, creating a need for efficient retrieval methods that can identify situations similar to a given query. Existing approaches typically rely on either visual representations or motion-based descriptions, making it difficult to understand their relative strengths and limitations for scenario retrieval. In this work, we present amultimodal framework for autonomous-driving scenario retrieval that combines visual and trajectory-based representations within a unified retrieval pipeline. We investigate two trajectory-based approaches: Exo-Trajectory, an explicit matching method based on surrounding-agent motion, and ScenarioFormer, a transformer-based representation learned from object trajectories using contrastive learning. We compare these approaches against strong visionbased baselines and analyze their behavior across a diverse set of driving scenarios. Experimental results show that trajectory representations provide strong retrieval performance for motion-centric events such as cut-ins, turning maneuvers, and traffic queueing, while visual embeddings excel when appearance cues are informative. Most importantly, combining visual and trajectory information consistently improves retrieval quality, yielding the best overall performance. These findings demonstrate that appearance and motion capture are complementary notions of scenario similarity and motivate multimodal retrieval systems for autonomous-driving data mining, dataset curation, and scenario-based validation.  \n1. Introduction  \nModern autonomous-driving systems generate and process enormous amounts of sensor data, resulting in datasets containing millions of recorded driving scenarios. Efficient retrieval of scenarios similar to a given query is essential for dataset curation, corner-case discovery, validation, and data-driven development workflows. However, defining similarity between driving scenarios remains a challenging problem. Two scenarios may appear visually different while  \nexhibiting similar traffic interactions, or conversely share a similar visual context while containing fundamentally different vehicle and agent behaviors.  \nExisting scenario retrieval approaches typically focus on a single notion of similarity. Vision-based methods leverage video representations to capture scene appearance and spatio-temporal context, while trajectory-based approaches compare vehicle motions and traffic participant behavior. Although both perspectives are relevant for autonomousdriving applications, their relative strengths and limitations have not been systematically studied within a unified retrieval framework.  \nIn this work, we investigate multimodal scenario retrieval by combining visual and trajectory-based representations. We evaluate two vision-based retrieval methods and introduce two trajectory-based approaches. The first, Exo-Trajectory, explicitly compares the motion patterns of surrounding traffic participants using trajectory matching. The second, ScenarioFormer, is a transformer-based representation learned from object trajectories using contrastive learning. Together, these methods enable a direct comparison between appearance-driven and motion-driven notions of scenario similarity.  \nExperimental results on a manually annotated drivingscene similarity benchmark show that visual and trajectory representations capture complementary information. Vision models perform best when appearance cues are informative, whereas trajectory-based methods excel on motioncentric scenarios such as cut-ins, turning maneuvers, and tr","cbCaisQ3XkJcEov2","https://ap.wps.com/l/cbCaisQ3XkJcEov2","pdf",5653806,1,9,"English","en",105,"# Introduction\n## Related Work\n### Visual Scenario Retrieval\n### Trajectory-Based Scenario Retrieval","[{\"question\":\"Why is scenario similarity search important in autonomous driving datasets?\",\"answer\":\"Modern autonomous-driving pipelines generate massive sensor data, leading to millions of recorded scenarios. Efficient retrieval supports dataset curation, corner-case discovery, validation, and data-driven development workflows.\"},{\"question\":\"What multimodal framework does the work propose for scenario retrieval?\",\"answer\":\"The framework combines visual representations with trajectory-based representations in a unified retrieval pipeline. It enables joint study of appearance-driven and motion-driven notions of scenario similarity.\"},{\"question\":\"How do visual and trajectory representations differ in what they capture?\",\"answer\":\"Trajectory representations deliver strong performance for motion-centric events such as cut-ins, turning maneuvers, and traffic queueing. Visual embeddings perform best when appearance cues are informative, and combining both improves overall retrieval quality.\"}]",1784179787,23,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"multimodal-scenario-similarity-search-for-autonomous-driving","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/multimodal-scenario-similarity-search-for-autonomous-driving/82346/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why is scenario similarity search important in autonomous driving datasets?","Question",{"text":75,"@type":76},"Modern autonomous-driving pipelines generate massive sensor data, leading to millions of recorded scenarios. Efficient retrieval supports dataset curation, corner-case discovery, validation, and data-driven development workflows.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What multimodal framework does the work propose for scenario retrieval?",{"text":80,"@type":76},"The framework combines visual representations with trajectory-based representations in a unified retrieval pipeline. It enables joint study of appearance-driven and motion-driven notions of scenario similarity.",{"name":82,"@type":73,"acceptedAnswer":83},"How do visual and trajectory representations differ in what they capture?",{"text":84,"@type":76},"Trajectory representations deliver strong performance for motion-centric events such as cut-ins, turning maneuvers, and traffic queueing. Visual embeddings perform best when appearance cues are informative, and combining both improves overall retrieval quality.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]