DIAG2GRAPH - REPRESENTING DEEP LEARNING DIAGRAMS IN RESEARCH PAPERS AS KNOWLEDGE GRAPHS

Oct 26, 2026

Diag2Graph addresses retrieval of specific deep learning architecture details from research papers by going beyond text-only academic search. It presents an end-to-end framework that localizes deep learning diagram figures, classifies them, and parses the diagram content by extracting components and their structural relations. The extracted entities and relations are encoded as a deep knowledge graph. Evaluation on a real-world annotated dataset demonstrates the approach’s effectiveness for semantic diagram search and retrieval.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages5

Reading Time13 min

AccessFree to download

Abstract

1. Introduction

2. Diagram Parsing: Diag2Graph

DIAG2GRAPH: REPRESENTING DEEP LEARNING DIAGRAMS IN RESEARCH PAPERS AS KNOWLEDGE GRAPHS Aditi Roy, Ioannis Akrotirianakis, Amar V. Kannan, Dmitriy Fradkin Arquimedes Canedo, Kaushik Koneripalli, Tugba Kulahcioglu Siemens Corporate Technology, 755 College Road East, Princeton, NJ 08540 ABSTRACT ‘Which are the segmentation algorithms proposed during 2018-2019 in CVPR that have CNN architecture?’ Answering this question involves identifying and analyzing the deep learning architecture diagrams from several research papers. Retrieving such information poses signiﬁcant challenge as most of the existing academic search engines are primarily based on only the text content. In this paper, we introduce Diag2Graph, an end-to-end framework for parsing deep learning diagram-ﬁgures, that enables powerful search and retrieval of architectural details in research papers. Our proposed approach automatically localizes ﬁgures from research papers, classiﬁes them, and analyses the content of the diagram-ﬁgures. The key steps in analyzing the ﬁgure content is the extraction of the different components data and ﬁnding their structural relation. Finally, the extracted components and their relations are represented in the form of a deep knowledge graph. A thorough evaluation on a real-word annotated dataset has been done to demonstrate the efﬁcacy of our approach. Index Terms— Diagram parsing, knowledge graph, deep learning, curator, similarity analysis 1. INTRODUCTION In the last decade, a whopping 1,710,000 research papers have been published in this area of Deep Learning (DL)1. Managing this large growth of DL publications has been a challenge for the researchers during validation, approval and dissemination DL related scientiﬁc information. Existing academic search engines like Google Scholar, arxiv, NOA [1] are inherently limited by their data mining and indexing approach which is restricted to only the text content of the papers. Given that scientiﬁc advancements require conceptualization, explanations and reproducible implementations, it is important to have a uniform representation that holistically represents a scientiﬁc publication as a machine curatable This work was supported by DARPA grant HR00111990010 . 1 [https://scholar.google.co.in/scholar?hl=en&as_](https://scholar.google.co.in/scholar?hl=en&as_)sdt=1%2C5&as_ylo=2010&as_yhi=2019&as_vis=1&q=deep+ learning&btnG= Fig. 1: Architecture of Diag2Graph to extract and represent DL diagrams in a research paper using knowledge graph. model for easy dissemination of scientiﬁc facts. To address this unique requirement, we propose to create a Deep Knowledge Graph (DKG) repository for papers related to DL algorithms and methods to help improve search and retrieval of relevant information in the academic domain. A RDF knowledge graph (KG) [2] encodes semantic information as uniquely identiﬁable entities and relationships between them in the form of a Subject-Predicate-Object (SPO) triple. Common machine queryable representation like KG of each scientiﬁc paper will allow scientists and engineers to explore the vast knowledge that resides in these papers and identify atomic scientiﬁc facts. Taking inspiration from large-scale social KGs such as YAGO4 [3] and DBPedia5 [4] that supports various general knowledge-related queries, we develop DKG for the ﬁrst time to the best of our knowledge. To achieve our goal, we propose a novel framework that automatically parses a research paper to extract DL architecture related information. It has been observed that most of the research papers explain the DL model through a ﬁgure. So, in this paper we focus on representing the DL diagram ﬁgure through a DKG. Our novel end-to-end framework named Diag2Graph (see ﬁgure 1) automatically localizes all ﬁgures from a research paper, classiﬁes them, and extracts the content of the DL diagram ﬁgures. Our proposed approach can localize a variety of ﬁgures and sub-ﬁgures, classify them by lever

FAQs

What problem does Diag2Graph solve in deep learning literature search?

It targets the difficulty of answering architecture-specific questions from research papers when existing search engines rely mainly on text. Diag2Graph parses deep learning diagram figures so architectural details become searchable.

How does Diag2Graph analyze deep learning diagram figures?

It localizes figures in papers, classifies them, and extracts the diagram components and their structural relationships. These extracted elements are then represented as a deep knowledge graph.

What form does the final representation take and how is it used?

The framework represents extracted components and their relations in a deep knowledge graph. The paper also describes a graph embedding method to find papers with similar diagrams based on diagram type and area.

Oct 26, 2026

Upload by Guten Appetit_Joghurt