Granuscore - A Reference-Free Measure of Granularity for Text Analysis and Question Answering

Sep 15, 2026

Uploaded by Patrick

Natural language expresses information across a spectrum of granularity, from fine-grained references to broad descriptions. Granularity is central to communication, yet existing metrics mainly measure surface detail or sentence specificity. This work introduces Granuscore, a reference-free granularity measure using structural properties of a hierarchical embedding space, validating hierarchical order recovery and contextual differences. It further explains non-linear specificity variation and supports principled analysis of QA dataset difficulty across benchmarks and model outputs.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages18

Reading Time45 min

AccessFree to download

Abstract

1 Introduction

2 Background and Related Work

Granuscore: A Reference-Free Measure of Granularity for Text Analysis and Question Answering Lukas Ellinger, Alexander Fichtl, Miriam Anschütz, and Georg Groh School for Computation, Information and Technology Technical University of Munich, Germany {lukas.ellinger, miriam.anschuetz, alexander.fichtl}@tum.de, [grohg@cit.tum.de](grohg@cit.tum.de) arXiv : 2605 .26620v2 [cs .CL] 2 1 Aug 2026 Abstract Natural language conveys information at varying levels of granularity, from fine-grained references to broad descriptions. While granularity is fundamental to human communication, existing measures mostly capture surface detail or sentence specificity. We introduce Granuscore, a reference-free measure of granularity that leverages structural properties of a hierarchical embedding space. Granuscore reliably recovers hierarchical orderings on the GRANOLA-EQ dataset and captures expected differences in granularity across discourse contexts. Across domains, we further show that Granuscore explains non-linear variation in sentence specificity beyond sentence length. Finally, we apply Granuscore to four questionanswering benchmarks and analyze how granularity differs for questions, gold answers, and model outputs across response outcomes. The analysis reveals consistent differences in model behavior and provides a principled lens for characterizing the difficulty of QA datasets. Together, the results position Granuscore as a scalable, broadly applicable tool for analyzing granularity in text. 1 Introduction Natural language varies not only in what information is conveyed, but also in how coarsely or finely that information is expressed. Consider the sentences in Figure 1 . A speaker may refer to a personas Tony Hawk, a skateboarder, or a sportsman, and may locate an event in San Diego, California, orthe United States. These alternatives preserve the underlying fact while referring to it at different levels. We refer to this dimension as granularity: the level of abstraction at which entities or events are represented in language (Mulkar-Mehta et al., 2011 ; Rosch et al., 1976 ; Hobbs, 1985) . Granularity is not incidental: speakers adapt the level of abstraction of their descriptions depending on conversational context and task requirements Example | 29.32 |
Tony Hawk was born in San Diego. | | --- | --- | | 40.34 |
Tony Hawk was born in California. | | 54.77 |
Tony Hawk was born in the United States. | | 74.53 |
A skateboarder was born in the United States. | | 81.60 |
A sportsman was born in the United States. | Figure 1: Sentences with referential units varying in granularity. Units that differ across sentences are underlined. Replacing fine-grained terms with coarser alternatives increases sentence granularity: lower Granuscores indicate finer expressions. (Mulkar-Mehta et al., 2011 ; Hobbs, 1985) . When uncertain, speakers often prefer coarser fact descriptions that remain informative without overcommitting. Conversely, when common ground is established, more fine-grained references become appropriate (Yona et al., 2024) . Granularity should therefore be understood as a deliberate strategy that balances reliability and audience expectations. Prior work suggests that linguistic granularity affects how information is perceived and used. In dialogue systems, too fine-grained or coarse responses can reduce user satisfaction (Adiwardana et al., 2020 ; Thoppilan et al., 2022) . Similarly, in simplified language settings, controlling granularity is important for accessibility and comprehension as it reduces the cognitive load (OECD, 2024 ; Anschütz et al., 2025) . However, studying such effects systematically is difficult because existing approaches do not provide a scalable, reference-free measure of granularity at the sentence level. Our contributions are as follows: • We introduce Granuscore, a reference-free measure of granularity that exploits structural properties of a hierarchical

FAQs

What problem does Granuscore address in measuring granularity?

It targets the lack of scalable, reference-free sentence-level measures, since prior approaches focus mainly on surface detail or sentence specificity.

How does Granuscore compute granularity?

It leverages structural properties of a hierarchical embedding space rather than relying on manually constructed hierarchies.

How is Granuscore used for question answering research?

It analyzes how granularity differs for questions, gold answers, and model outputs across response outcomes, providing insight into QA dataset difficulty and model behavior.

Sep 15, 2026

Uploaded by Patrick