LLM_annotate - A Python package for annotating and analyzing fiction characters

Sep 17, 2026

LLM_annotate is a Python package for analyzing the personality and behaviors of fiction characters using large language models. It standardizes workflows for annotating character behaviors in full texts, inferring character traits, and validating annotation and inference quality through a human-in-the-loop GUI. The package provides text chunking, LLM-based annotation, character name disambiguation, quality scoring, and computation of character-level statistics and embeddings. It supports any LLM and is demonstrated with The Simpsons Movie and Pride and Prejudice.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

Annotating character behaviors and traits

LLM_annotate package overview

Existing approaches and their limitations

Alternative: neural models and LLM-based annotation

Running head: LLM_ANNOTATE PACKAGE 1 LLM_annotate: A Python package for annotating and analyzing fiction characters Hannes Rosenbusch ([h.rosenbusch@uva.nl](h.rosenbusch@uva.nl); 0000-0002-4983-3615) Department of Psychological Methods, University of Amsterdam LLM_annotate is a Python package for analyzing the personality of fiction characters with large language models. It standardizes workflows for annotating character behaviors in full texts (e.g. , books/movie scripts) , inferring character traits, and validating annotation/inference quality via a human-in-the-loop GUI. The package includes functions for text chunking, LLM-based annotation, character name disambiguation, quality scoring, and computation of character-level statistics and embeddings. Researchers can use any LLM—commercial, open-source, or custom—for the package’s functionality. Through tutorial examples using The Simpsons Movie and the novel Pride and Prejudice, I demonstrate the usage of the package for efficient and reproducible character analyses. LLM_annotate: A Python package for annotating and analyzing fiction characters The impact of a story depends on its characters ( Bourrier & Thelwall, 2020 ; Cohen, 2013 ; Raney, 2004 ; Šauperl, 2012; Tchernev, 2022) . Readers might relate to their book’s protagonist, hate the antagonist, or fall in love with a quirky sidekick. Similarly, TV audiences might laugh, cry, emulate, or refuse to spend any more time with the characters on the screen. Accordingly, research from behavioral, aesthetic, and computational disciplines often analyzes character behaviors and attributes (e.g. , Forster, 1985; Gottschall et al. , 2003; Priyatno et al. , 2024; Yuan et al. , 2024; Yu et al. , 2023) . Here, I introduce a small Python package, LLM_annotate, that standardizes the use of LLMs for the purpose of annotating character behaviors and scoring character traits. The package consists of functions to deliver raw annotations, disambiguate character names, collect character-level statistics, and provide a GUI for human-in-the-loop validation. After a brief review of character annotation methods, I will demonstrate the use of the package, point out best practices, and discuss shortcomings and possible extensions. Annotating character behaviors and traits Researchers annotate behaviors and traits of fictional characters to analyze historical developments ( Underwood et al. , 2018) , predict authorship (e.g. , Wolyn & Simske, 2023) , review societal norms (e.g. , Abbasi et al. , 2023) , investigate the psychology of authors and readers (e.g. , Kokesh & Sternadori, 2015 ; Mishara, 2010) , and map reader preferences to book contents (e.g. , Johnson et al. , 1984 ; Krakowiak, 2015) . The practice of analyzing character attributes has become so central in literary studies that researchers have started building character databases for books and movie scripts (Sang et al. , 2022; Yu et al. , 2023) . LLM_ANNOTATE PACKAGE 2 Commonly, researchers are interested in a set of attributes or behaviors among characters in a book or other media piece, and thus have human annotators generate the desired annotations ( Douglas, 2008; Faber & Mayer, 2009; Hesse et al. , 2005) . Given the high cost of such annotations, computational researchers have developed a range of methods to (partially) automate this process. For instance, past work relied on predefined word lists to analyze character attributes (e.g. , Tausczik & Pennebaker, 2010 ; Mohammad & Turney, 2013) . Flekova & Gurevych (2015) used such lists to predict personality traits of fictional characters with above chance accuracy. Other researchers have used extensions of this method by factoring in context for the word occurrences (e.g. , is a relevant verb performed by the character or towards the character; Bamman et al. , 2013 ; 2014) . While word-level features and count-based analyses have been of great use to literary researchers, they are limited in the types of annot

FAQs

What does LLM_annotate do for fiction character analysis?

LLM_annotate provides standardized workflows to annotate character behaviors, infer character traits, and compute character-level statistics and embeddings using large language models.

How does the package validate annotation and inference quality?

It uses a human-in-the-loop GUI to validate the quality of annotations and inferred character traits.

What inputs and components does LLM_annotate include?

It includes functions for text chunking, LLM-based annotation, character name disambiguation, quality scoring, and the computation of character-level statistics and embeddings.

Sep 17, 2026

Uploaded by Anna Hans