[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84871-en":3,"doc-seo-84871-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84871,8796095461564,"Liam","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","Faithful or Findable? Evaluating LLM-Generated Metadata for RDF Dataset Search","Dataset search relies on metadata, so LLM-generated synthetic metadata becomes consequential for retrieval systems. The study examines six RDF metadata-generation settings, from simple rewriting to profile-grounded and agentic graph-based generation, evaluating joint outcomes for retrieval effectiveness and faithfulness. Unconstrained rewriting yields the largest retrieval gains yet lowest faithfulness, indicating benefits can stem from unsupported semantic expansion. More grounded methods improve faithfulness; profile-grounded rewriting offers the best balance. Results frame synthetic metadata as a system-level IR problem requiring joint evaluation of effectiveness, provenance, and trust.","Faithful or Findable? Evaluating LLM-Generated Metadata for  \nRDF Dataset Search  \nRiccardo Terrenzi  \n[rite@mmmi.sdu.dk](rite@mmmi.sdu.dk)[ ](rite@mmmi.sdu.dk)University of Southern Denmark Sønderborg, Denmark  \nSerkan Ayvaz  \nUniversity of Southern Denmark Sønderborg, Denmark [seay@mmmi.sdu.dk](seay@mmmi.sdu.dk)  \narXiv :2607 .05970v 1 [ cs .IR] 7 Jul 2026  \nAbstract  \nDataset search depends heavily on metadata, making LLM-generated metadata a consequential form of synthetic content in retrieval systems. We study six metadata-generation settings for RDF datasets, ranging from simple rewriting to profile-grounded and agentic graph-based generation, and evaluate them jointly for retrieval effectiveness and faithfulness. Unconstrained metadata rewriting delivers the strongest retrieval gains over the original metadata, but it is also the least faithful, showing that search improvements can be driven by unsupported semantic expansion. More grounded settings substantially improve faithfulness, and profile-grounded rewriting provides the most balanced trade-off between retrieval effectiveness and grounding. These findings position synthetic metadata asa system-level IR problem in which effectiveness, provenance, and trust must be evaluated together.  \nCCS Concepts  \n• Information systems → Information retrieval; Evaluation of retrieval results; Relevance assessment; Retrieval effectiveness.  \nKeywords  \nDataset search, Synthetic Metadata, Faithfulness, Large Language Models, RDF Datasets, Information Retrieval  \n1 Introduction  \nDataset search depends heavily on metadata [5, 16]. Unlike many traditional document retrieval settings, users rarely interact directly with the underlying data at search time; instead, retrieval systems index and rank dataset surrogates such as titles, descriptions, and keywords [4] . This makes metadata a particularly consequential layer in the retrieval pipeline: changing metadata can change how datasets are represented, matched, and ultimately discovered. Recent advances in large language models (LLMs) and tool-using agents make it increasingly feasible to rewrite, enrich, or generate dataset metadata automatically, raising the prospect of synthetic metadata becoming part of real-world search infrastructures [18] .  \nFrom an information retrieval perspective, this shift is promising but non-trivial. On the one hand, automated metadata generation may improve sparse or incomplete descriptions and expose signals that help ranking models retrieve datasets more effectively [7, 18] . On the other hand, synthetic metadata is not a neutral transformation: it can introduce new terms, new levels of specificity, and new interpretations of the underlying dataset. In mixed human– synthetic retrieval environments, this creates a central question not only about effectiveness, but also about grounding. If synthetic metadata improves search performance while drifting away from the original metadata or the dataset itself, then the retrieval gains  \ncome with corresponding risks for faithfulness, provenance, and trust.  \nIn this paper, we investigate this problem on a subset of ACORDAR 2.0 [6] containing roughly 1,000 RDF datasets. We study several metadata production settings that reflect increasingly automated forms of synthesis: metadata rewrite from original metadata, profile rewrite aided by a dataset profile, profile gen from profile-based inputs, profile title gen that retains the original title, and agentic generation with direct access to the underlying dataset through dataset agent and dataset title agent. We evaluate the resulting metadata from two complementary perspectives. First, we benchmark their impact on dataset retrieval using standard IR metrics, including NDCG, MAP, and MRR. Second, we assess faithfulness through an LLM-based judge, with claim-level verification against the appropriate evidence source for each setting, including original metadata, profiles, and the underlying dataset.  \nOur aim is th","cbCaitcD34v19Gci","https://ap.wps.com/l/cbCaitcD34v19Gci","pdf",436396,1,5,"English","en",105,"# Introduction\n## Metadata as a retrieval representation\n## Research goals and evaluation setup\n# Related Work","[{\"question\":\"How does metadata affect RDF dataset search performance?\",\"answer\":\"Dataset search ranks dataset surrogates such as titles, descriptions, and keywords, so metadata quality directly changes how datasets are represented and matched, influencing discovery outcomes.\"},{\"question\":\"What trade-off is observed between retrieval effectiveness and faithfulness in LLM-generated metadata?\",\"answer\":\"Unconstrained metadata rewriting improves retrieval the most but is least faithful, suggesting that retrieval gains may come from semantic expansion not supported by evidence. More grounded settings improve faithfulness, with profile-grounded rewriting offering the most balanced trade-off.\"},{\"question\":\"How is faithfulness evaluated for different metadata-generation settings?\",\"answer\":\"An LLM-based judge performs claim-level verification by checking each claim against the appropriate evidence source for that setting, including original metadata, dataset profiles, and the underlying RDF dataset.\"}]",1784198933,13,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"faithful-or-findable-evaluating-llm-generated-metadata-for-rdf-dataset-search","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/faithful-or-findable-evaluating-llm-generated-metadata-for-rdf-dataset-search/84871/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"How does metadata affect RDF dataset search performance?","Question",{"text":75,"@type":76},"Dataset search ranks dataset surrogates such as titles, descriptions, and keywords, so metadata quality directly changes how datasets are represented and matched, influencing discovery outcomes.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What trade-off is observed between retrieval effectiveness and faithfulness in LLM-generated metadata?",{"text":80,"@type":76},"Unconstrained metadata rewriting improves retrieval the most but is least faithful, suggesting that retrieval gains may come from semantic expansion not supported by evidence. More grounded settings improve faithfulness, with profile-grounded rewriting offering the most balanced trade-off.",{"name":82,"@type":73,"acceptedAnswer":83},"How is faithfulness evaluated for different metadata-generation settings?",{"text":84,"@type":76},"An LLM-based judge performs claim-level verification by checking each claim against the appropriate evidence source for that setting, including original metadata, dataset profiles, and the underlying RDF dataset.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":21,"slug":137},19,"General","general"]