When Writing Style Drifts - Benchmarking Authorship Verification under Distribution Shifts in Genre, Time and the AI-Era

Sep 15, 2026

Uploaded by Maeve

Authorship verification assumes that an individual’s writing style stays stable enough to distinguish it from other writers. In real settings, distribution shifts from changing genre, time, and AI-assisted writing undermine this assumption. Existing benchmarks often vary these factors separately and focus mainly on English. The work introduces AVShift, a German benchmark for evaluating verification across multiple shifts. AVShift covers 150,000+ text pairs across three genres and 21 years, supporting controlled cross-genre, temporal, and AI-era assessments. Results compare feature, embedding, and LLM approaches, showing LLMs generalize best across genres and temporal drift is a dominant degradation factor, while no measurable AI-era shift appears.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

Introduction

Authorship analysis and verification

Distribution shifts and benchmark gap

AVShift contributions

Related Work

Authorship analysis methods

When Writing Style Drifts: Benchmarking Authorship Verification under Distribution Shifts in Genre, Time and the AI-Era Lotta Kiefer1 , Brisca Balthes2 , Christoph Leiter3 , Yamen Ajjour4 , Elena Schmidt, Steffen Eger5 1,2,3,4,5University of Technology Nuremberg (UTN) [1](1lotta.kiefer@utn.de 5steffen.eger@utn.de)[lotta.kiefer@utn.de](1lotta.kiefer@utn.de 5steffen.eger@utn.de)[ 5](1lotta.kiefer@utn.de 5steffen.eger@utn.de)[steffen.eger@utn.de](1lotta.kiefer@utn.de 5steffen.eger@utn.de) arXiv : 2608 . 17979v 1 [cs .CL] 18 Aug 2026 Abstract Authorship verification (AV) assumes that an author’s writing style remains sufficiently stable to distinguish it from that of other writers. In practice, however, this assumption is challenged by distribution shifts caused by changes in genre, time, and AI-assisted writing. Existing AV benchmarks typically study these factors in isolation and focus predominantly on English, limiting our understanding of model robustness under realistic conditions. We introduce AVShift, the first German benchmark for systematically evaluating AV under multiple distribution shifts. AVShift comprises over 150,000 text pairs spanning three genres and 21 years, enabling controlled evaluation of crossgenre, temporal, and AI-era shifts within a unified framework. We benchmark representative feature-based, embedding-based, and LLMbased approaches. Our experiments show that fine-tuned LLMs generalize best across genres and benefit substantially from stylistically diverse training data. We further demonstrate that temporal drift is one of the strongest factors affecting AV, with performance degrading significantly as the time gap between documents increases. In contrast, we find no evidence of a measurable AI-era distribution shift within AVShift. Finally, our feature analysis reveals stylistic features that remain stable across genres, while their relative importance varies depending on the specific genre transition. We release AVShift and our code for future research. 1 Introduction Authorship analysis aims to identify the author of a text based on characteristic patterns of individual language use, commonly referred to as an idiolect (Coulthard, 2004 ; Nini, 2023) . Numerous benchmark datasets spanning books, blogs, news, emails, reviews, social media, and darknet forums have established authorship analysis as an important tool in literary studies, plagiarism detection, and forensic linguistics (Tyo et al., 2023 ; Boenninghoff et al., Figure 1: Overview of the stylistic distribution shifts covered by AVShift: cross-genre variation, temporal evolution, and AI-era changes before and after the widespread adoption of generative AI. 2024 ; Lewis et al., 2004 ; Manolache et al., 2022 ; Overdorf and Greenstadt, 2016 ; Stamatatos et al., 2023) . This work focuses on authorship verification (AV), which determines whether two texts were written by the same author. Unlike authorship attribution that relies on a predefined set of candidates, AV naturally generalizes to previously unseen authors without requiring retraining, making it particularly attractive for forensic investigations involving new suspects. The concept of linguistic individuality is sometimes described as a linguistic fingerprint (Kreuz, 2023 ; Eder, 2011) . However, this analogy can be misleading, giving the wrong impression that writing style is an immutable concept like a biological fingerprint (Coulthard, 2004 ; Nini, 2023) . Instead, linguistic style is influenced by numerous contextual factors, including the intended audience, topic, communicative situation, medium, and evolves over time. We therefore view AV as a learning problem that is inherently exposed to distribution shifts. In practice, both training and test data, as well as the paired texts themselves, may differ in genre, time, or broader language use. Existing benchmarks typ- ically investigate these challenges in isolation and focus predominantly on English.

FAQs

What is the key assumption behind authorship verification, and why does it fail in practice?

Authorship verification assumes writing style remains stable across documents. It fails because genre changes, temporal evolution, and AI-assisted writing can shift stylistic patterns over time.

What is AVShift and what evaluation scenarios does it cover?

AVShift is a German benchmark for evaluating authorship verification under three distribution shifts: cross-genre changes, temporal shifts spanning decades, and AI-era differences before and after widespread generative AI adoption.

Which approaches perform best across genres, and what are the main findings about temporal and AI-era shifts?

Fine-tuned LLMs generalize best across genres when trained with stylistically diverse data. Temporal drift is found to strongly degrade verification performance, while the benchmark shows no measurable AI-era distribution shift.

Sep 15, 2026

Uploaded by Maeve