[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83450-en":3,"doc-seo-83450-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":11,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83450,1099513958607,"Jiven","https://ap-avatar.wpscdn.com/avatar/100002390cf8733938c?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778829742770036399",8,"Research & Report","A Synthetic-Driven Vision System for Assembly Step Recognition","Industrial assembly quality control requires reliable real-time monitoring to prevent costly defects. Vision-based inspection is effective, yet training typically depends on task-specific real-world data that is expensive and time-consuming to collect and annotate. The proposed system automatically generates realistic assembly sequences from CAD models and step descriptions, then trains real-time inspection models using the synthetic data. It combines physics-based motion generation, domain-randomized photorealistic rendering, and YOLO-style object detection with temporal filtering, reaching 92.4% accuracy on a real-world case while improving key metrics.","A Synthetic-Driven Vision System for Assembly Step Recognition  \nHui Zhang 1,2,*,†, Xuanang Lei 1,* , Rui Wang 1 , Julian Ferchow 1,2 , and Mirko Meboldt 1  \n1ETH Zurich, Switzerland; 2inspire AG, Switzerland  \narXiv :2607 .00129v1 [ cs .CV] 30 Jun 2026  \nAbstract—Quality control in industrial assembly is essential, and real-time monitoring of the assembly process is crucial for preventing costly defects and ensuring production reliability. Vision-based automated inspection offers a powerful solution for such real-time monitoring. However, due to the specialized industrial components and processes, training these models typically relies on task-specific real-world data, which is costly and labor-intensive to collect and annotate. In this paper, we propose a system that automatically generates realistic assembly sequences and further trains real-time inspection models using the synthetic data. It can be efficiently applied toa given task within an hour, requiring only CAD models and simple step descriptions. Focusing on practical challenges, our system integrates a physics-based motion generation module to capture the variance of different human assembly, designs domain-randomized rendering to deal with the environmental complexity and variation, and employs an object-detectionbased step recognition module for robust sim-to-real transfer, leading to 92.4% accuracy on a real-world assembly case with 46.7%, 15.8% and 61.2% performance improvement, respectively. Overall, our system provides a practical solution for industrial assembly inspection without requiring expensive real-world data collection and annotation, with the effectiveness validated on real industrial assembly tasks.  \nI. INTRODUCTION  \nHigh-mix low-volume production has become increasingly important in modern manufacturing, where manual assembly remains indispensable with its flexibility. However, manual assembly is inherently error-prone, as workers might miss steps, follow incorrect sequences, or misplace components, which can lead to costly defects if errors are not detected in real time. Computer vision systems show promise for realtime assembly inspection [1], [2] . Nevertheless, due to the specialized components and processes, training models for such systems requires task specific data, which is costly to collect and annotate in the real world, especially for high-mix low-volume production which requires fast adaptation.  \nTo address this limitation, leveraging synthetic data to train inspection models with minimal manual effort is highly desirable. However, several practical challenges remain in industrial assembly scenarios: (1) industrial components and assembly processes are highly task-specific, leading to large domain gaps from daily objects and across tasks, so general-purpose models often fail and task-specific data are required; (2) manual assembly involves continuous hand–tool–component interactions and severe occlusions, which are difficult to model and synthesize and have  \n*Equal Contribution.  \n†Correspondence email: [huizhang@ethz.ch](huizhang@ethz.ch).  \nFig. 1: Real assembly data vs. Our synthetic data  \nconstrained prior works that only render static component combinations without hands, making them applicable only to simple tasks with fully visible parts and hand-free inspection [3], [4], [5]; (3) different workers adopt diverse strategies and factories exhibit frequent changes in lighting and backgrounds, demanding broad data coverage and model robustness to large motion and appearance variations; and (4) synthetic data inevitably suffers from sim-to-real gapsin texture, illumination, and motion realism, which become more pronounced in dynamic assembly processes within complex industrial environments and must be mitigated for practical deployment.  \nTo directly tackle these practical challenges, we propose a synthetic-data-driven system for industrial assembly step recognition, which is composed of three key modules: physics-based motion ","cbCaitBc4TL0gG98","https://ap.wps.com/l/cbCaitBc4TL0gG98","pdf",17340536,4,1,"English","en",105,"# Introduction\n## Challenges in industrial assembly inspection\n## Proposed synthetic-driven system modules\n## Experimental validation","[{\"question\":\"Why is real-time vision-based monitoring important in industrial assembly?\",\"answer\":\"Real-time monitoring helps detect missing steps, wrong sequences, or misplaced components early, preventing costly defects and improving production reliability.\"},{\"question\":\"How does the system generate training data without extensive real-world annotation?\",\"answer\":\"It constructs task-specific assembly sequences from provided CAD models and step descriptions, then renders photorealistic RGB sequences and physics-based motions to create synthetic data for training.\"},{\"question\":\"How does the approach address sim-to-real gaps for robust deployment?\",\"answer\":\"It uses domain-randomized rendering to vary backgrounds and lighting, generates diverse physics-based motions, and trains a YOLO-based detector on synthetic data combined with a rule-based temporal filter for reliable video step recognition.\"}]",1784188006,20,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"a-synthetic-driven-vision-system-for-assembly-step-recognition","",{"@graph":35,"@context":84},[36,52,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":20},"https://docshare.wps.com/document/a-synthetic-driven-vision-system-for-assembly-step-recognition/83450/",{"url":51,"name":13,"@type":53,"author":54,"headline":13,"publisher":56,"fileFormat":59,"inLanguage":23,"description":14,"dateModified":60,"datePublished":61,"encodingFormat":59,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":55},"Person",{"url":40,"name":57,"@type":58},"DocShare","Organization","application/pdf","2026-07-24","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":20},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"Why is real-time vision-based monitoring important in industrial assembly?","Question",{"text":74,"@type":75},"Real-time monitoring helps detect missing steps, wrong sequences, or misplaced components early, preventing costly defects and improving production reliability.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How does the system generate training data without extensive real-world annotation?",{"text":79,"@type":75},"It constructs task-specific assembly sequences from provided CAD models and step descriptions, then renders photorealistic RGB sequences and physics-based motions to create synthetic data for training.",{"name":81,"@type":72,"acceptedAnswer":82},"How does the approach address sim-to-real gaps for robust deployment?",{"text":83,"@type":75},"It uses domain-randomized rendering to vary backgrounds and lighting, generates diverse physics-based motions, and trains a YOLO-based detector on synthetic data combined with a rule-based temporal filter for reliable video step recognition.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":57,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,119,122,126,129,133],{"id":21,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":28,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":28,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":28,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]