Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions

Oct 12, 2026

Uploaded by Noah

Multimodal large language models (MLLMs) can help blind and low vision (BLV) people access visual information, yet they frequently produce hard-to-detect errors that raise safety and social risks. This work studies whether surfacing systematic variations across multiple MLLM outputs enables BLV users to identify unreliable claims without visually inspecting images. The paper proposes a design space, a prototype with three variation styles, and a study with 15 BLV participants. Results show a 4.9x improvement in detecting unreliable information and lower perceived reliability.

LanguageEnglish

CategoryGeneral

FormatPDF

Pages17

Reading Time6 min

AccessFree to download

Abstract

1 Introduction

CCS Concepts

Keywords

ACM Reference Format

Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions Meng Chen The University of Texas at Austin Austin, Texas, USA [mengchen@utexas.edu](mengchen@utexas.edu) Akhil Iyer The University of Texas at Austin Austin, Texas, USA [akhil.iyer@utexas.edu](akhil.iyer@utexas.edu) Amy Pavel University of California, Berkeley Berkeley, California, USA [amypavel@eecs.berkeley.edu](amypavel@eecs.berkeley.edu) arXiv :2507 . 15692v1 [ cs .HC] 21 Jul 2025 Abstract Multimodal large language models (MLLMs) provide new opportunities for blind and low vision (BLV) people to access visual information in their daily lives. However, these models often produce errors that are difficult to detect without sight, posing safety and social risks in scenarios from medication identification to outfit selection. While BLV MLLM users use creative workarounds such as crosschecking between tools and consulting sighted individuals, these approaches are often time-consuming and impractical. We explore how systematically surfacing variations across multiple MLLM responses can support BLV users to detect unreliable information without visually inspecting the image. We contribute a design space for eliciting and presenting variations in MLLM descriptions, a prototype system implementing three variation presentation styles, and findings from a user study with 15 BLV participants. Our results demonstrate that presenting variations significantly increases users’ability to identify unreliable claims (by 4.9x using our approach compared to single descriptions) and significantly decreases perceived reliability of MLLM responses. 14 of 15 participants preferred seeing variations of MLLM responses over a single description, and all expressed interest in using our system for tasks from understanding a tornado’s path to posting an image on social media. CCS Concepts • Human-centered computing → Accessibility design and evaluation methods; Accessibility systems and tools. Keywords Accessibility, Image Descriptions, Multimodal Large Language Models, Variations, AI Errors, Trust ACM Reference Format: Meng Chen, Akhil Iyer, and Amy Pavel. 2025. Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions. In The 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’25), October 26–29, 2025, Denver, CO, USA. ACM, New York, NY, USA, 17 pages. [https://doi.org/10.1145/3663547.3746393](https://doi.org/10.1145/3663547.3746393) 1 Introduction Recent progress in generative AI including multimodal large language models (MLLMs) provides a transformative opportunity for millions of blind and low vision (BLV) people to access both digital This work is licensed under a Creative Commons Attribution 4 .0 International License. ASSETS’25, Denver, CO, USA © 2025 Copyright held by the owner/author(s) . ACM ISBN 979-8-4007-0676-9/2025/10 [https://doi.org/10.1145/3663547.3746393](https://doi.org/10.1145/3663547.3746393) and real-world visual information [12, 35, 77] . MLLMs are deep learning algorithms that can process and generate multiple types of content, including text, images, audio, and video [78] . BLV people can now use MLLMs through services such as Be My AI [2], Seeing AI [30], and Envision [3] to support daily activities such as exploring new spaces [24, 50], reading informative charts and diagrams, interpreting images on social media posts [19], or creating visual art [41]. MLLMs provide descriptions faster than human-powered access tools [5, 6, 22] and with more detail than traditional image captioning techniques [42] . While MLLMs produce fluent and persuasive responses, their responses can be factually incorrect or misleading [42]. For example, MLLMs erroneously fabricate content that is not in the image (e.g., state an empty frame contains a family picture), misinterpret content that is in the image (e.g., mistaking a cleaning product for

FAQs

Why are MLLM-generated image descriptions challenging for BLV users to trust?

MLLM responses can be fluent yet factually incorrect or misleading, and BLV users often lack ways to detect subtle errors or biases without comparing against the image.

What is the core idea of the proposed approach?

The approach systematically surfaces variations across multiple MLLM responses so BLV users can detect unreliable information without visually inspecting the image.

How effective was the approach in the user study?

In a study with 15 BLV participants, presenting variations increased users’ ability to identify unreliable claims by 4.9x compared to single descriptions and significantly decreased perceived reliability.

Oct 12, 2026

Uploaded by Noah