[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85851-en":3,"doc-seo-85851-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85851,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","KGCQual: An Interpretable Framework for Evaluating the Knowledge Graph Construction Quality from Text","Knowledge Graphs are increasingly generated from text via automated extraction pipelines, yet such systems often produce spurious, incomplete, or semantically distorted triples that harm downstream reasoning and querying. KGCQual introduces an intrinsic, interpretable metric that compares an automatically extracted graph to an ideal reference graph derived from key noun phrases, predicate relations, and linguistic phenomena like negation. The framework combines entity-level completeness and connectivity with relation-level predicate preservation and multiplicity using lexical similarity, dependency alignment, and lightweight negation handling, validated across multiple datasets and extraction systems.","arXiv :2607 . 102 12v 1 [ cs .AI] 11 Jul 2026  \nKGCQual: An Interpretable Framework for Evaluating the Knowledge Graph Construction Quality from Text  \nNipun Misra 1 , Vikranth Udandarao2 , Aanchal Gupta2 , Yogender Kumar2 , Manuj Mukherjee2 , and Raghava Mutharaju3  \n1VIT Vellore, Vellore, India  \n[nipun.misra2022@vitstudent.ac.in](nipun.misra2022@vitstudent.ac.in)  \n2 IIIT-Delhi, Delhi, India  \n{aanchal21224, yogender21505, vikranth22570, [manuj}@iiitd.ac.in](manuj}@iiitd.ac.in)  \n3 Indian Institute of Technology Palakkad, Kerala, India.  \n[raghava@iitpkd.ac.in](raghava@iitpkd.ac.in)  \nAbstract. Knowledge Graphs (KGs) are increasingly constructed through automated extraction pipelines; however, such systems often introduce spurious or incomplete triples, which degrade downstream performance.  \nExisting evaluation practices rely heavily on task-specific metrics or small-scale manual verification, offering limited insight into the structural and semantic fidelity of extracted graphs. We propose a novel, interpretable metric for intrinsic KG quality assessment that measureshow closely an automatically extracted graph approximates an “ideal”  \ngraph capturing the key noun phrases, predicate relations, and basic linguistic phenomena such as negation expressed in the source text. Our framework integrates two complementary components: (1) an entity-level assessment that evaluates completeness, resolution quality, and connectivity, and (2) a relation-level assessment that judges predicate preservation and multiplicity using lexical similarity, dependency-parse alignment, and light-weight negation handling to ensure semantic faithfulness. We evaluate our metric across multiple state-of-the-art triple extraction systems and datasets, including WebNLG, TinyButMighty, and BenchIE, demonstrating that it reliably identifies omissions, redundancy, and structural deviations that existing metrics overlook. Our work offers a scalable, model-agnostic, and interpretable framework for comparing automated KG construction methods and provides a foundation for standardised evaluation. We further validate the metric through an ablation study isolating noun and verb components, and a downstream evaluation showing that KGCQual scores correlate significantly with link prediction performance (ρ = −0 .900 , p = 0 .037) on the same extracted KGs. The code repository is available at [https://github.com/kracr/kg-quality](https://github.com/kracr/kg-quality)  \n-metric.  \nKeywords: Knowledge Graph · Knowledge Graph Construction · Knowledge Graph Construction Quality · Information Extraction · Evaluation Metrics  \n2 N. Misra et al.  \n1 Introduction  \nKnowledge Graphs (KGs) have become a foundational abstraction enabling structured representation of entities, relations, and facts in a form amenable to reasoning, querying, and integration across heterogeneous data sources [11,18] . A growing fraction of modern KGs spanning domains such as scientific literature, news, and enterprise documents are constructed through automated Information Extraction (IE) pipelines. These systems transform natural language text into subject–predicate–object triples that are later serialized into RDF, linked to ontologies, or validated through SHACL constraints.  \nHowever, despite their widespread adoption, the quality of triples produced by IE systems remains highly variable. Existing KG quality frameworks (e.g.,[18,6]) provide rich dimensions such as accuracy, completeness, and consistency, but presuppose that the underlying triples are already correct or human-curated. In contrast, for automatically extracted KGs, the fundamental bottleneck lies one level earlier: the lack of an intrinsic, sentence-level method for assessing whether extracted triples faithfully preserve the semantics expressed in the source text. Current evaluations rely predominantly on (i) downstream task performance, which is indirect and application-specific, or (ii) manual annotation, which is costly and non","cbCaijBLzovtOzBU","https://ap.wps.com/l/cbCaijBLzovtOzBU","pdf",1003057,5,1,20,"English","en",105,"# Abstract\n# Introduction\n## Intrinsic evaluation gap\n## Proposed KGCQual framework\n### Entity Quality (Noun-Level)\n### Relation Quality (Verb-Level)\n# Evaluation (described in abstract)\n## Ablation and correlation with link prediction","[{\"question\":\"What problem does KGCQual address in knowledge graph construction pipelines?\",\"answer\":\"KGCQual targets the lack of intrinsic, sentence-level evaluation for automatically extracted triples, which can be incomplete, redundant, or semantically misaligned with the source text and then degrade downstream KG tasks.\"},{\"question\":\"How does KGCQual define and approximate an “ideal” reference graph?\",\"answer\":\"It approximates the ideal reference graph using part-of-speech cues, dependency relations, and lightweight semantic similarity to reflect salient noun phrases, predicate relations, and linguistic phenomena such as negation.\"},{\"question\":\"What are the two main components of KGCQual’s evaluation metric?\",\"answer\":\"KGCQual evaluates two complementary dimensions: entity quality (completeness, granularity/resolution, and connectivity) and relation quality (predicate preservation, multiplicity, and semantic similarity).\"}]",1784206703,50,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"kgcqual-an-interpretable-framework-for-evaluating-the-knowledge-graph-construction-quality-from-text","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/kgcqual-an-interpretable-framework-for-evaluating-the-knowledge-graph-construction-quality-from-text/85851/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-25","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does KGCQual address in knowledge graph construction pipelines?","Question",{"text":76,"@type":77},"KGCQual targets the lack of intrinsic, sentence-level evaluation for automatically extracted triples, which can be incomplete, redundant, or semantically misaligned with the source text and then degrade downstream KG tasks.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does KGCQual define and approximate an “ideal” reference graph?",{"text":81,"@type":77},"It approximates the ideal reference graph using part-of-speech cues, dependency relations, and lightweight semantic similarity to reflect salient noun phrases, predicate relations, and linguistic phenomena such as negation.",{"name":83,"@type":74,"acceptedAnswer":84},"What are the two main components of KGCQual’s evaluation metric?",{"text":85,"@type":77},"KGCQual evaluates two complementary dimensions: entity quality (completeness, granularity/resolution, and connectivity) and relation quality (predicate preservation, multiplicity, and semantic similarity).","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,110,114,119,122,126,129,133],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":29,"slug":113},6,"Technology","technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":22,"slug":125},9,"Religion & Spirituality","religion-spirituality",{"id":22,"doc_module":4,"doc_module_name":46,"category_name":127,"show_sort_weight":22,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":46,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":46,"category_name":135,"show_sort_weight":20,"slug":136},19,"General","general"]