[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-663081-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-663081-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","a-multi-stage-hybrid-framework-for-automated-interpretation-of-multi-view-engineering-drawings-using-vision-language-model","A Multi-Stage Hybrid Framework for Automated Interpretation of Multi-View Engineering Drawings Using Vision Language Model","","Engineering drawings are central to manufacturing communication, conveying design intent, tolerances, and production details, yet automatic interpretation of dense multi-view drawings remains difficult with manual workflows, generic OCR, or single-stage deep learning. This paper introduces a three-stage hybrid pipeline for OCR-free 2D multi-view drawing interpretation using vision language models. Layout segmentation localizes views, title blocks, and notes, orientation-aware detection extracts symbolic/textual annotations, and task-specialized Donut-based VLMs parse alphabetical and numerical content into structured JSON.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/a-multi-stage-hybrid-framework-for-automated-interpretation-of-multi-view-engineering-drawings-using-vision-language-model/663081/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/a-multi-stage-hybrid-framework-for-automated-interpretation-of-multi-view-engineering-drawings-using-vision-language-model/663081.png","ImageObject",300,407,{"name":42,"@type":43},"pop out","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-30","2026-10-28",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",5,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What is the main goal of the proposed framework?","Question",{"text":63,"@type":64},"The framework aims to automatically interpret 2D multi-view engineering drawings by extracting design, annotation, and quantitative information into structured, machine-readable output.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How does the three-stage pipeline work?",{"text":68,"@type":64},"Stage one performs layout segmentation (e.g., views, title blocks, notes), stage two uses orientation-aware detection for annotations, and stage three uses specialized OCR-free Donut-based vision language models to parse textual and numerical content.",{"name":70,"@type":61,"acceptedAnswer":71},"Why are vision language models used instead of OCR?",{"text":72,"@type":64},"The approach targets OCR-free understanding to better handle rotated text, dense layouts, and symbolic structures such as GD&T constructs and their frames, which are difficult for generic OCR tools.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},663081,1793225002,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,100,105,110,114,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":97,"show_sort_weight":98,"slug":99},"Comic",60,"comic",{"id":101,"doc_module":4,"doc_module_name":25,"category_name":102,"show_sort_weight":103,"slug":104},6,"Technology",50,"technology",{"id":106,"doc_module":4,"doc_module_name":25,"category_name":107,"show_sort_weight":108,"slug":109},7,"Healthcare",40,"healthcare",{"id":111,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":112,"slug":113},8,30,"research-report",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":55,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":111,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":123,"language":139,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":140,"faqs":141,"seo_title":142,"seo_description":12,"update_tm":80,"read_time":143},962090760408,"https://ap-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45","A Multi-Stage Hybrid Framework for Automated Interpretation of Multi-View Engineering Drawings Using Vision Language Model  \nMuhammad Tayyab Khan 1, 3*\\#, Zane Yong2\\#, Lequn Chen2, Wenhe Feng 1, Nicholas  \nYew Jin Tan 1, Seung Ki Moon3  \n1 Singapore Institute of Manufacturing Technology (SIMTech), Agency for Science, Technology and Research (A*STAR), 5 CleanTech Loop, \\#01-01 CleanTech Two Block B, Singapore  \n636732, Republic of Singapore  \n2 Advanced Remanufacturing and Technology Centre (ARTC), Agency for Science, Technology and Research (A*STAR), 3 CleanTech Loop, \\#01-01 CleanTech Two, Singapore 637143,  \nRepublic of Singapore  \n3 School of Mechanical and Aerospace Engineering, Nanyang Technological University, 639798, Singapore  \n*[KHAN0022@e.ntu.edu.sg](KHAN0022@e.ntu.edu.sg)  \n[\\#](# Equal Contribution)[ Equal Contribution](# Equal Contribution)  \nAbstract. Engineering drawings are fundamental to manufacturing communication, serving as the primary medium for conveying design intent, tolerances, and production details. However, interpreting complex multi-view drawings with dense annotations remains challenging using manual methods, generic optical character recognition (OCR) systems, or traditional deep learning approaches, due to varied layouts, orientations, and mixed symbolic–textual content. To address these challenges, this paper proposes a three-stage hybrid framework for the automated interpretation of 2D multi-view engineering drawings using modern detection and vision language models (VLMs) . In the first stage, YOLOv11-det performs layout segmentation to localize key regions such as views, title blocks, and notes. The second stage uses YOLOv11-obb for orientation-aware, fine-grained detection of annotations, including measures, GD&T symbols, and surface roughness indicators. The third stage employs two Donut-based, OCRfree VLMs for semantic content parsing: the Alphabetical VLM extracts textual and categorical information from title blocks and notes, while the Numerical VLM interprets quantitative data such as measures, GD&T frames, and surface roughness. Two specialized datasets were developed to ensure robustness and generalization: 1,000 drawings for layout detection and 1,406 for annotationlevel training. The Alphabetical VLM achieved an overall F1 score of 0.672, while the Numerical VLM reached 0.963, demonstrating strong performance in textual and quantitative interpretation, respectively. The unified JSON output enables seamless integration with CAD and manufacturing databases, providing a scalable solution for intelligent engineering drawing analysis.  \nKeywords: 2D Engineering Drawings, Multi-View Interpretation, Vision Language Models, Technical Document Understanding, Structured Data Extraction.  \n1 Introduction  \nEngineering drawings remain central to industrial communication, containing key information that influences manufacturability, product quality, and cost [1–3] . Converting this information into structured, machine-readable formats is critical for enabling automation in computer-aided design (CAD), process planning, and inspection. However, in industrial practice, this extraction process remains largely manual, creating a major bottleneck in digital manufacturing workflows. Manual transcription is time-consuming and non-scalable, particularly for complex drawings where hundreds of annotations are distributed across multiple drawing views. Generic optical character recognition (OCR) tools worsen these issues, as they struggle to interpret rotated text, dense layouts, and symbolic constructs such as geometric dimensioning and tolerancing (GD&T) feature frames [4] . These limitations highlight the need for an OCR-free, layout-aware approach to reliable information extraction.  \nDeep learning has recently been applied to automate selected aspects of this problem. For instance, Reddy et al. [5] fine-tuned detection models to recognize GD&T symbols in mechanical drawings, while Yazed et al. [6] ","cbCairf3kD79WL4L","https://ap.wps.com/l/cbCairf3kD79WL4L","pdf",822419,"English","# Abstract\n# 1 Introduction","[{\"question\":\"What is the main goal of the proposed framework?\",\"answer\":\"The framework aims to automatically interpret 2D multi-view engineering drawings by extracting design, annotation, and quantitative information into structured, machine-readable output.\"},{\"question\":\"How does the three-stage pipeline work?\",\"answer\":\"Stage one performs layout segmentation (e.g., views, title blocks, notes), stage two uses orientation-aware detection for annotations, and stage three uses specialized OCR-free Donut-based vision language models to parse textual and numerical content.\"},{\"question\":\"Why are vision language models used instead of OCR?\",\"answer\":\"The approach targets OCR-free understanding to better handle rotated text, dense layouts, and symbolic structures such as GD\\u0026T constructs and their frames, which are difficult for generic OCR tools.\"}]","A Multi-Stage Hybrid Framework for Automated Interpretation of Multi-View Engineering Drawings Using Vision Language Model | PDF",25]