Curated Retrieval versus Open Web Search in Public AI Information Services: A Coverage-Trust Trade-off

Jul 16, 2026

Upload by Oliver

Public institutions increasingly use large language models to answer citizens’ questions, often combining a curated knowledge base with live web search, while the trustworthiness of the cited sources has received limited empirical study. This pre-launch expert evaluation of Evrópuvefur, an Icelandic government-funded service, compares a curated RAG corpus with open web search. Expert judgments show web search yields broader coverage but frequently cites flagged sources, while curated retrieval stays trustworthy yet limited, with the model refusing when coverage is insufficient.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages38

Reading Time1 h 36 min

Introduction

Retrieval approaches: RAG vs open web search

Risks in public LLM question answering

Source trustworthiness as information quality dimension

Motivation for an expert evaluation

arXiv :2607 .052 17v2 [ cs .CY] 7 Jul 2026 Curated retrieval versus open web search in public AI information services: a coverage–trust trade-off Hafsteinn Einarsson ‗1, Hafsteinn Birgir Einarsson 2, Jón Gunnar Ólafsson 2, and Jón Gunnar Þorsteinsson3 1 Faculty of Industrial Engineering, Mechanical Engineering and Computer Science, University of Iceland, Reykjavík, Iceland 2 Faculty of Political Science, University of Iceland, Reykjavík, Iceland 3 The Icelandic Web of Science, University of Iceland, Reykjavík, Iceland Preprint, July 2026 Abstract Public institutions increasingly use large language models (LLMs) to answer citizens’questions, often pairing a curated knowledge base with live web search, yet whether the sources behind these answers can be trusted has received little empirical scrutiny. Wereport a pre-launch expert evaluation of Evrópuvefur, an independent, government-funded service run by the University of Iceland that answers questions about the European Union, conducted as Iceland prepared for its referendum of 29 August 2026 on whether to resume EU accession talks. Five domain experts produced 551 evaluations of 449 AI-generated answers, scoring each against a seven-criterion quality rubric and, separately, flagging individual cited sources. We compared two retrieval paths: a curated local corpus (RAG) and open web search. In more than a third of the reviewed web-search answers (35%, 65 of 187), at least one cited source was flagged, almost always as untrustworthy or irrelevant; curated sources were flagged far less often and only for being out of date. Web search answered more questions, but at the cost of source quality; the curated corpus was trustworthy yet limited in coverage, and the model declined to respond when it fell short. The citation mix also passed over strong sources: across all 287 web-search answers, the system never cited RÚV, the public broadcaster and the country’s most widely used news source. A companion prompt ablation shows how weak prompt-level steering is: a trusted-domain list in the system prompt raised the share of citations to listed domains only from 12% to 21% . Fluency and topical fit did not predict source trustworthiness. We argue that source trustworthiness is a measurable yet largely invisible dimension of information quality in public AI services, and we discuss transparency-oriented responsesand their trade-offs. Keywords: artificial intelligence · large language models · retrieval-augmented generation · information quality · source trustworthiness · public information · expert evaluation ‗Corresponding author: [hafsteinne@hi.is](hafsteinne@hi.is) 1 Introduction Public bodies have begun using large language models (LLMs) to answer citizens’ questions directly, and generative AI is already in widespread, if uneven, use across government (Bright et al., 2025; OECD, 2024) . If the aim is to ground answers in real material rather than the model’s own parametric memory, at least two approaches are available: retrievalaugmented generation (RAG) over a curated knowledge base the institution controls (Lewis et al., 2020), and live web search over the open internet. We study these two paths separately. The appeal of either is plain. One system can answer far more questions thana hand-written FAQ, in the citizen’s own language, at any hour. Some risks of putting LLMs in front of citizens are by now familiar. A public-sector chatbot can give confidently wrong advice; New York City’s business-help bot was caught telling firms to break the law (Offenhartz, 2024), and early systems hallucinated and mishandled citations. Those failures have eased as models matured, and they are not our subject. A subtler risk persists even when the prose is fluent and the citations resolve: the sources the system leans on may not deserve the trust readers place in them. Readers were never reliable judges of online source credibility, long before LLMs entered the picture (

FAQs

What core trade-off does the study examine between curated retrieval and open web search?

The study compares coverage versus source trustworthiness: web search answers more questions but often cites sources flagged as untrustworthy or irrelevant, while curated retrieval is more trustworthy but limited in coverage.

How was the Evrópuvefur service evaluated before launch?

Domain experts produced 551 evaluations of 449 AI-generated answers, scoring responses using a seven-criterion quality rubric and separately flagging individual cited sources.

Which retrieval path performed better in relation to the trustworthiness of cited sources?

Curated sources were flagged far less often and mainly for being out of date, whereas more than a third of web-search answers included at least one flagged cited source, usually untrustworthy or irrelevant.

Jul 16, 2026

Upload by Oliver