A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification

Oct 13, 2026

Upload by lilac

Instruction-tuned large language models generate self-explanations for their outputs, offering a user-facing rationale without specialized interpretability pipelines. This paper evaluates whether such self-explanations are plausible to humans and faithful to model decisions. Experiments compare self-explanations as input rationales with human annotations across sentiment classification, forced labour detection, and claim verification. Four open-weight LLMs are analyzed, including cross-lingual sentiment translations, and post-hoc attribution rationales are contrasted via intervention-based faithfulness tests.

LanguageEnglish

CategoryGeneral

FormatPDF

Pages21

Reading Time7 min

AccessFree to download

Introduction

Contributions

Experimental Setup

Evaluation of Plausibility

Evaluation of Faithfulness

Post-hoc Attribution Comparison

Findings and Implications

A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification Stephanie Brandl* Center for Social Data Science University of Copenhagen [stephanie.brandl@sodas.ku.dk](stephanie.brandl@sodas.ku.dk) Oliver Eberle∗ Machine Learning Group Technische Universität Berlin [oliver.eberle@tu-berlin.de](oliver.eberle@tu-berlin.de) Abstract Instruction-tuned LLMs are able to provide an explanation about their output to users by generating self-explanations, without requiring the application of complex interpretability techniques. In this paper, we analyse whether this ability results in a good explanation. We evaluate self-explanations in the form of input rationales with respect to their plausibility to humans. We study three text classification tasks: sentiment classification, forced labour detection and claim verification. We include Danish and Italian translations of the sentiment classification task and compare self-explanations to human annotations. For this, we collected human rationale annotations for Climate-Fever, a claim verification dataset. We furthermore evaluate the faithfulness of human and selfexplanation rationales with respect to correct model predictions, and extend the study by incorporating post-hoc attribution-based explanations. We analyse four open-weight LLMs and find that alignment between self-explanationsand human rationales highly depends on text length and task complexity. Nevertheless, selfexplanations yield faithful subsets of tokenlevel rationales, whereas post-hoc attribution methods tend to emphasize structural and formatting tokens, reflecting fundamentally different explanation strategies. 1 Introduction Providing model explanations to increase trustworthiness and transparency is a key goal of interpretability research, with LLMs offering new ways to trace model decision-making. As AI-based analyses increasingly have the power to influence decisions and perceptions, systematically tracing and comparing explanations of these decisions is essential for fostering trust, transparency and regulatory compliance. Today, LLMs are being used for a wide variety of tasks, ranging from creative writing * Equal contribution. Figure 1: An example from the SST sentiment classification dataset. With rationale annotations by humans and generated by Llama3 . and homework assistance to offering advice, summarizing reports and translation, while providing self-generated explanations of their outputs, i.e., self-explanations, in the process.1 This makes it even more important to understand the quality of those self-explanations, how reliable they are and to evaluate their faithfulness to the model and their plausibility to humans. In this paper, we evaluate self-explanations from two text classification tasks for which human rationale annotations are available: sentiment classification and forced labour detection, i.e., identifying pre-defined risk indicators of forced labour. We also collect and release new rationale annotations by three personally instructed annotators for a subset of Climate-Fever, a claim verification dataset with claims related to climate change. We instruct 4 different LLMs (Gemma3, Llama3, Mistral and Qwen3) to solve the respective tasks and generate rationales based on the input text in a zero-shot experiment. For sentiment classification, we consider two different subsets from two different annotation studies, one also including Italian and Danish translations alongside the English original. Following established evaluation methods in the interpretability and Explainable AI literature (DeYoung et al., 2020 ; Jacovi and Goldberg, 2020), we assess the plausibility of model rationales by measuring their agreement with human annotations and evaluate faithfulness via interventions on input tokens to determine their importance for the [1](1www.washingtonpost.com/technology/2024/08/04/)[www.washingtonpost.com/technology/2024/08/04/](1w

FAQs

What does the paper evaluate about LLM self-explanations?

It assesses whether self-explanations are plausible to humans and faithful to the model’s decisions.

Which text classification tasks are used in the study?

The paper studies sentiment classification, forced labour detection, and claim verification.

How does the study test faithfulness of explanations?

It uses interventions on input tokens to measure their importance for the model’s decision, comparing human and self-explanation rationales and also post-hoc attribution methods.

Oct 13, 2026

Upload by lilac