StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description

Jul 16, 2026

Upload by Ophelia

Long-form audio description (AD) for blind and low-vision audiences must preserve narrative meaning beyond frame-by-frame action, including who the characters are, how events relate, and how each scene connects to earlier context. Existing video–language models often describe moments independently and therefore miss story coherence. StoryTeller proposes a training-free framework that carries forward a verified narrative memory across scenes and can retrieve public movie metadata optionally. It avoids subtitles, transcripts, character banks, or fine-tuning, and improves narrative coherence and factual grounding on benchmarks via automatic, QA-based, and human evaluations.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages41

Reading Time1 h 43 min

Introduction

Problem: story-aware long-form audio description

Related work and limitations

Training-free approach

Proposed StoryTeller framework

Evaluation and StoryAD-QA benchmark

arXiv :2607 . 11798v1 [ cs .CV] 13 Jul 2026 StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description Seung Hyun Hahm, Minh T. Dinh, and SouYoung Jin Dartmouth College, USA {Seung.Hyun.Hahm.GR,Minh.T.Dinh.GR,[SouYoung.Jin}@dartmouth.edu](SouYoung.Jin}@dartmouth.edu) Abstract. Long-form audio description (AD) requires more than describing visible actions: it must preserve characters, events, relationships, and story context across scenes so that blind and low-vision (BLV) audiences can follow a film. Modern video–language models (VLMs) are effective on short clips, but they often treat each moment independently, producing descriptions that miss who characters are, why events matter, and how the current scene connects to earlier narrative context. We propose StoryTeller, a training-free framework for story-aware long-form AD. Instead of relying only on local visual cues, StoryTeller maintains a verified narrative memory that carries forward story-relevant information across scenes, enabling later descriptions to remain coherent, grounded, and contextually informative. Given only raw video and a movie title, StoryTeller can optionally retrieve public movie metadata to resolve names and story context, while accepting only facts that are supported by the video through semantic filtering and VLM verification. The method requires no subtitles, scripts, AD transcripts, aligned captions, character banks, precomputed face identities, or task-specific fine-tuning. To evaluate whether generated AD preserves narrative information, we introduce StoryAD-QA 1 , a question-answering benchmark that tests whether a language model can answer story-context questions using only the generated descriptions. Experiments on standard AD benchmarks and diverse long-form videos show that StoryTeller consistently improves narrative coherence, factual grounding, and story comprehension over strong baselines in automatic, QA-based, and human evaluations. Keywords: audio description · long-form video understanding · retrieval-augmented generation · narrative grounding · accessibility 1 Introduction Audio description (AD) enables blind and low-vision (BLV) audiences to access visual media by narrating important visual information between dialogue and other sounds. Effective AD goes beyond describing objects or actions in the current frame: it identifies who is present, explains what has just happened, 1 StoryAD-QA benchmark dataset and evaluation code: [https://github.com/SEE-AI-Lab/ECCV2026_StoryTeller_StoryAD_QA](https://github.com/SEE-AI-Lab/ECCV2026_StoryTeller_StoryAD_QA) 2 S. H. Hahm et al. Baseline AD Albus Dumbledore makes his wand move with a flick of his wrist while conversing with someone. Harry is hit by a teacher, while Ron and Hermione look on. Fig. 1: Story-level audio description requires narrative memory. Existing audio description (AD) systems typically generate descriptions using only local visual context or static character banks, which often fails to preserve long-range narrative relationships. Our StoryTeller instead maintains a persistent narrative state consisting of an identity graph that tracks characters across scenes and a salience-weighted memory that accumulates narrative facts over time. This enables consistent character references and long-range story reasoning across an entire film. The baseline example is the output of AutoAD-Zero. conveys why a moment matters, and connects the scene to the broader narrative. Professional AD writers are able to provide this narrative grounding because they watch the film and write with the whole story in mind. However, producing high-quality AD remains costly and time-intensive, and copyright restrictions make large publicly distributable AD datasets difficult to obtain. Recent advances in video–language models (VLMs) [2,19,30,36] and large language models have motivated research on automatic AD generation. State-of-theart identity-aware AD sy

FAQs

Why is long-form audio description more difficult than describing short video clips?

Long-form AD must maintain narrative continuity across scenes, preserving character identity, relationships, and event context. Many models treat each moment independently, so they can miss who characters are and how current events connect to earlier story information.

What is StoryTeller and how does it preserve narrative context?

StoryTeller is a training-free framework that maintains an explicit narrative state while processing clips in chronological order. It uses a verified narrative memory that carries forward story-relevant facts to keep later descriptions coherent and contextually grounded.

What resources does StoryTeller avoid using?

StoryTeller does not rely on subtitles, scripts, AD transcripts, aligned captions, character banks, precomputed face identities, or task-specific fine-tuning. It can optionally retrieve public movie metadata, but only accepts facts supported by the video through semantic filtering and VLM verification.

Jul 16, 2026

Upload by Ophelia