[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-657700-105":3,"detail-sidebar-cat-0-en-105":84,"doc-detail-657700-en":134},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":77,"head_meta":79,"extra_data":81,"updated_unix":83},105,"en","legend-informed-symbol-recognition-in-engineering-diagrams-self-supervised-learning","Legend-Informed Symbol Recognition in Engineering Diagrams - Self-Supervised Learning","","Engineering diagrams are critical documents across industries, yet they are often stored as images and converted into modern formats for reuse and adaptation. This work introduces a self-supervised method for automated recognition of engineering diagram symbols, leveraging diagram legends that provide prototypical examples. Unique legend entries enable contrastive learning to embed symbols, extracting them via a detector trained on synthetic data. A matching step classifies unknown symbols by nearest legend embeddings and rejects non-legend cases, achieving over 80% accuracy despite depiction variations.",{"@graph":14,"@context":76},[15,34,55],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/legend-informed-symbol-recognition-in-engineering-diagrams-self-supervised-learning/657700/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/legend-informed-symbol-recognition-in-engineering-diagrams-self-supervised-learning/657700.png","ImageObject",300,407,{"name":42,"@type":43},"Quinn Holloway","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-30","2026-10-26",true,{"@type":52,"interactionType":53,"userInteractionCount":33},"InteractionCounter",{"@type":54},"ViewAction",{"@type":56,"mainEntity":57},"FAQPage",[58,64,68,72],{"name":59,"@type":60,"acceptedAnswer":61},"Why are engineering diagram symbols difficult to recognize in practice?","Question",{"text":62,"@type":63},"Many diagrams lack semantic, machine-readable information and vary strongly in drawing style, age, and source. Public labeled datasets for robust symbol recognition are also difficult to obtain in industry settings.","Answer",{"name":65,"@type":60,"acceptedAnswer":66},"How does the method use diagram legends for self-supervised symbol recognition?",{"text":67,"@type":63},"Legend entries provide prototypical symbol examples. The approach learns symbol embeddings via contrastive learning by contrasting symbols found inside the legends, enabling classification of unknown symbols by matching their embeddings to legend examples.",{"name":69,"@type":60,"acceptedAnswer":70},"What is the training pipeline before the embedding and matching stage?",{"text":71,"@type":63},"The pipeline generates synthetic data, trains a generalized symbol detector on synthetic diagrams to extract symbols from diagrams and their corresponding legends, then trains a symbol encoder using legend symbols with a self-supervised contrastive learning framework.",{"name":73,"@type":60,"acceptedAnswer":74},"How does the system handle symbols that do not appear in the legend?",{"text":75,"@type":63},"It uses an adaptive embedding distance threshold per diagram. Symbols whose embeddings are too far from legend embeddings are rejected as not represented in the legend.","https://schema.org",{"og:url":32,"og:type":78,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":80,"canonical":32},"index,follow",{"doc_id":82,"site_id":7},657700,1793114879,{"code":4,"msg":85,"data":86},"success",[87,91,95,99,104,109,114,118,123,126,130],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":88,"show_sort_weight":89,"slug":90},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":92,"show_sort_weight":93,"slug":94},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":96,"show_sort_weight":97,"slug":98},"Exam",70,"exam",{"id":100,"doc_module":4,"doc_module_name":25,"category_name":101,"show_sort_weight":102,"slug":103},5,"Comic",60,"comic",{"id":105,"doc_module":4,"doc_module_name":25,"category_name":106,"show_sort_weight":107,"slug":108},6,"Technology",50,"technology",{"id":110,"doc_module":4,"doc_module_name":25,"category_name":111,"show_sort_weight":112,"slug":113},7,"Healthcare",40,"healthcare",{"id":115,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":116,"slug":117},8,30,"research-report",{"id":119,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":121,"slug":122},9,"Religion & Spirituality",20,"religion-spirituality",{"id":121,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":121,"slug":125},"World Cup","world-cup",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":127,"slug":129},10,"Lifestyle","lifestyle",{"id":131,"doc_module":4,"doc_module_name":25,"category_name":132,"show_sort_weight":100,"slug":133},19,"General","general",{"code":4,"msg":85,"data":135},{"doc_id":82,"user_id":136,"nickname":42,"user_avatar":137,"doc_module":4,"category_id":115,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":138,"file_id":139,"file_url":140,"file_type":141,"file_size":142,"view_count":33,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":143,"language":144,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":145,"faqs":146,"seo_title":147,"seo_description":12,"update_tm":148,"read_time":149},2336474466712,"https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd","Legend-Informed Symbol Recognition in Engineering Diagrams with Self-Supervised Learning  \nAntonia Hain 1 (􀀀), Simon Gölzhäuser1 , Nicolas Réhault 1 , Thomas Brox2 , and  \nMatthias Demant 1  \n1 Fraunhofer Institute for Solar Energy Systems ISE, Germany  \n{antonia.hain, simon.goelzhaeuser, nicolas .rehault, matthias .demant}@ise .[fraunhofer.de](fraunhofer.de)  \n2 University of Freiburg, Germany  \nAbstract. Engineering diagrams are vital documents in many industries. Historically stored as image data, conversion of such diagrams into modern formats is required for further use and adaptation. Therefore, research towards automated digitization has gained traction. To recognize symbols in the diagrams, recent studies rely on supervised learning, but large labeled datasets are difficult to acquire in industry settings.  \nIn this paper, we present a self-supervised approach towards automated recognition of engineering diagram symbols. We validate the method on diagrams from the building sector, where they are used for technical plant planning, installation, and monitoring. The method makes use of diagram legends, which show prototypical examples of the symbols occurring in the diagram. As the legend entries are unique, they can be used to learn embeddings through contrastive learning for a self-supervised classification of diagram symbols. The method circumvents most of the labeling efforts: all symbols are extracted from the set of diagrams with a symbol region detector trained on a synthetic dataset. Then, we train a symbol encoder by contrasting the symbols found inside the legends with eachother. The encoder is subsequently used in a matching procedure that classifies unknown diagram symbols by comparing them to the legend examples. Furthermore, it can recognize when symbols do not appear in the legend at all. Generalizing beyond variations in diagram drawing style, this matching procedure achieves over 80% accuracy. The results demonstrate the potential of legends for engineering diagram digitization without the need to invest in labeled datasets.  \nKeywords: Engineering Diagram · Building Services · Heating, Ventilation and Air Conditioning · Contrastive Learning  \n1 Introduction  \nThe operation of buildings, specifically heating and cooling systems, contributes significantly to carbon emissions, causing an estimated 26% of global emissions  \nFig. 1. Exemplary engineering diagrams with different style, coloring and symbols.  \nrelated to energy [11] . Therefore, steps towards renovation or operational optimization of heating, ventilation, and air conditioning (HVAC) systems must be taken promptly. However, much effort goes into planning such steps: Before deriving retrofit or optimization measures, existing systems are thoroughly analyzed by reviewing building data and transferring the often decades-old data into a modern CAD format. This entails high workloads for engineers and technicians.  \nAn integral part of building data are engineering diagrams, which illustrate a system’s components, using symbols, and its topology. Similar diagrams are used in other industry sectors such as process engineering or electrical engineering. The diagrams may also contain text, tables, or a legend explaining the symbols. Despite existing norms, their style and qualities vary strongly depending on age and source, as shown in Fig. 1. Diagrams may be hand-drawn, scanned or photographed from a print, or provided as a PDF file. In most cases, they therefore do not contain any semantic machine-readable information.  \nThough considerable advances have been made towards automated recognition and digitization of engineering diagrams, creating a solution that is robust to variation in depiction styles is difficult, especially in regard to diagram symbols. A lack of public and labeled datasets in the domain makes the training of machine learning models for this purpose additionally challenging.  \nWe propose a novel pipeline for the recognition of symbo","cbCaibGEvgdgDkKJ","https://ap.wps.com/l/cbCaibGEvgdgDkKJ","pdf",2672688,18,"English","# Introduction\n## Motivation and challenges\n## Proposed legend-based pipeline\n# Related Work\n## Engineering Diagram Digitization","[{\"question\":\"Why are engineering diagram symbols difficult to recognize in practice?\",\"answer\":\"Many diagrams lack semantic, machine-readable information and vary strongly in drawing style, age, and source. Public labeled datasets for robust symbol recognition are also difficult to obtain in industry settings.\"},{\"question\":\"How does the method use diagram legends for self-supervised symbol recognition?\",\"answer\":\"Legend entries provide prototypical symbol examples. The approach learns symbol embeddings via contrastive learning by contrasting symbols found inside the legends, enabling classification of unknown symbols by matching their embeddings to legend examples.\"},{\"question\":\"What is the training pipeline before the embedding and matching stage?\",\"answer\":\"The pipeline generates synthetic data, trains a generalized symbol detector on synthetic diagrams to extract symbols from diagrams and their corresponding legends, then trains a symbol encoder using legend symbols with a self-supervised contrastive learning framework.\"},{\"question\":\"How does the system handle symbols that do not appear in the legend?\",\"answer\":\"It uses an adaptive embedding distance threshold per diagram. Symbols whose embeddings are too far from legend embeddings are rejected as not represented in the legend.\"}]","Legend-Informed Symbol Recognition in Engineering Diagrams - Self-Supervised Learning | PDF",1792986433,45]