Learning to Ground Before Reading - Unified PCB Engineering Drawing Parsing with Compact Vision-Language Models

Oct 29, 2026

Upload by Logic

PCB engineering drawings combine sparse graphics, dense tables, and spatially dependent text, making downstream recovery difficult when regions are missed. A compact vision-language model reads the entire page and outputs a sequence of region classes, normalized bounding boxes, and corresponding text or HTML. Bounding boxes are supervised via coordinate tokens. Inference avoids any detector or crop parser, while a Localization-First curriculum learns class-box targets before adding content targets using content-aware resampling. On the ED dataset, strict localization F1 improves by 0.0955 over joint training.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

AccessFree to download

Abstract

Introduction

Related Work

arXiv : 2608 .29268v 1 [cs .CV] 29 Aug 2026 Learning to Ground Before Reading: Unified PCB Engineering Drawing Parsing with Compact Vision-Language Models Jinghao Liu2⋆, Xingrun Liu2 ,∗ , Gengchen Sun 1 ,2 ,3 ,4 , Han Xiao4 , Xingyu Chen4 , and Yuhui Deng 1 ,2 1 Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science, Zhuhai 519087, China 2 Beijing Normal–Hong Kong Baptist University, Zhuhai 519087, China 3 Faculty of Science, Hong Kong Baptist University, Hong Kong SAR 999077, China 4 Hong Kong aiKnow Limited, Hong Kong SAR, China {sungengchen, [ivandeng}@bnbu.edu.cn](ivandeng}@bnbu.edu.cn) Abstract. PCB engineering drawings mix sparse graphics, dense tables, and text whose meaning depends on page position. Localizing the regions and sending crops to specialized recognizers are determined as the methods for most parsers, so missed regions cannot be recovered downstream. We train a compact VLM to read the full page and get a sequence of region classes, normalized boxes, and text or HTML content. Bounding boxes are converted to coordinate tokens for supervision. Inference uses no detector or crop parser. The joint target is difficult to optimize because class and box tokens are sparse relative to the much longer content sequences. Our localization-first curriculum learns the class–box format before adding content targets with content-aware resampling. On the fixed validation split of the Engineering Drawing Dataset (ED dataset), LocalizationFirst improves strict localization F1 by 0.0955 over joint training (paired image-bootstrap 95% interval: [0 .0350 , 0. 1572]) . G-Unified has the lowest NED, highest cell F1, and only nonzero exact-match score. It provides a detector-free baseline for full-page PCB drawing parsing. Keywords: Engineering drawings · document understanding · visual grounding · vision-language models · structured extraction 1 Introduction PCB engineering drawings are usually highly informative, including graphical views, fabrication notes, layer stackups, drill schedules, and heterogeneous tableson a single high-resolution page. Experienced people can easily locate the wanted information, however, it is hard for the vision model, as this is more than the recognition problem: content must be interpreted together with its spatial context and semantic role. Beyond identifying where content occurs, explicit localization ⋆ These authors contributed equally to this work. 2 Authors Suppressed Due to Excessive Length recovers the page layout required for downstream tracing, because relationships among drawing elements often have to be inferred from their relative positions and spatial arrangement rather than from content alone.A complete parser therefore needs to identify relevant regions, determine what each region represents, recover its textual or tabular content, and retain its location on the page. Most existing document and engineering-drawing systems address these requirements with a cascaded pipeline. A detector or layout analyzer first localizes regions, after which individual crops are routed to Optical Character Recognition(OCR) engines or Vision-Language Models for recognition. Systems such as eDOCr2 combine geometry-driven segmentation with specialized recognition modules [21], while Hybrid-VL uses a trained YOLO11m-OBB detector before parsing detected crops with a VLM [10, 11, 23] . This decomposition creates a hard dependency between localization and recognition: a missed or incorrectly cropped region cannot be recovered by the downstream parser. It also introduces detector-specific training, crop-routing rules, and multiple inference stages. We investigate whether this cascade can be replaced by a single compact VLM operating directly on the full page. Our Unified Grounding Training framework (G-Unified) formulates PCB parsing as structured sequence generation, where each predicted region is represented by its semantic class, b

FAQs

Why is the Localization-First curriculum needed?

Class and coordinate targets are short and sparse compared with much longer content tokens, causing optimization imbalance under joint training. Localization-First learns the class-box format before introducing content targets, with content-aware resampling.

Oct 29, 2026

Upload by Logic