Does AI Understand Imaging? - A Systematic Benchmark of Agentic AI for Computational Imaging Tasks

Sep 21, 2026

Uploaded by Levi

Vision-language models and agentic AI excel on semantic visual tasks, yet their ability to handle physics and inverse problems in computational imaging remains uncertain. This work introduces ImagingBench, evaluating 20 computational imaging tasks across five categories: optics, image signal processing, inverse reconstruction, computational sensing, and calibration. Three settings—Expert, Planner, and Forward—test expert-guided inversion, planner-guided execution, and forward-system consistency. Results show agentic models are consistently weaker than specialized methods, with modest and inconsistent gains from planner guidance, revealing a gap between semantic competence and physically grounded fidelity.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

AccessFree to download

Introduction

ImagingBench overview and motivation

Evaluation settings (Expert, Planner, Forward)

Contributions and benchmark coverage

Does AI Understand Imaging? A Systematic Benchmark of Agentic AI for Computational Imaging Tasks arXiv :2607 .07 189v 1 [ cs .AI] 8 Jul 2026 Ethan Chung 1 ,∗ , Chuanjun Zheng 1 ,∗ , Jasper Tan2 , Jingxi Li2 , Haopeng Zhang 1 , and Huaijin Chen 1 Abstract—Vision–language models (VLMs) and agentic AI have shown strong performance on semantic visual tasks, but it remains unclear whether they can handle the physics and inverse problems that underlie computational imaging. We present ImagingBench, a benchmark of 20 computational imaging tasks spanning five categories: ray and wave optics, image signal processing, inverse reconstruction, computational sensing, and calibration. ImagingBench evaluates three complementary settings: Expert, fixed expert-guided inverse reconstruction; Planner, planner-guided inverse reconstruction; and Forward, forward-system simulation for consistency checking. We benchmark leading proprietary and open-source image-centric multimodal systems, including Gemini, GPT, and Qwen, and compare them with representative task-specific non-agentic baselines. Across tasks, agentic models remain consistently weaker than specialized methods, especially on computational sensing problems such as lensless imaging, event-based reconstruction, time-of-flight imaging, and holography. Planner guidance provides only modestand inconsistent gains over the fixed-prompt Expert baseline. Although the models often generate visually plausible outputs, their reference-based fidelity remains poor, revealing a substantial gap between semantic visual competence and physically grounded imaging performance. ImagingBench provides a unified testbed for measuring this gap and tracking progress in agentic AI for computational imaging. Index Terms—Computational Photography, Vision-Language Models, Generative AI, Inverse Problems, Benchmark. ~~ ~~ ✦ ~~ ~~ 1 Introduction BENCHMARKS have repeatedly reshaped progress in ma chine learning by turning broad capabilities into reproducible evaluation programs. In vision and multimodal reasoning, widely used suites now measure recognition, segmentation, captioning, visual question answering, tool use, physical reasoning, and controllable generation [1], [2],[3], [4], [5], [6], [7], [8], [9], [10], [11] . Yet these benchmarks do not directly test a central form of visual competence: whether agentic AI systems can reason about image formation and solve computational imaging problems. Computational imaging differs from semantic vision because success depends on optics, sensing, sampling, calibration, degradation, and reconstruction. Tasks such as denoising, deblurring, demosaicking, HDR recovery, lensless reconstruction, time-of-flight restoration, holography, calibration, and imaging-system design require models to reason about forward operators and solve ill-posed inverse problems, rather than only assign semantic labels. Classical and learning-based computational imaging has made substantial progress through physics-based modeling, task-specific optimization, and learned reconstruction pipelines [12], [13],[14], [15], [16], [17], [18], [19], but benchmark design has not kept pace with the emergence of multimodal foundation models and agents. This gap matters because visually plausible outputs can be wrong when performance is governed by sensor physics, optical aberrations, noise statistics, or undersampling. It also matters for agentic systems: a useful imaging agent should not only produce an image, but also diagnose degradations,• ∗ Equal contribution. 1 University of Hawaii at Manoa. 2 Glass Imaging. • Preprint / work in progress. Subject to revision; not the final version. propose physically sensible restoration strategies, and remain consistent with the forward imaging process. Existing general-vision, medical-imaging, multimodal-reasoning, and agent benchmarks do not jointly evaluate these abilities in a unified setting. To address this gap, we introduce Imagin

FAQs

What gap does the ImagingBench benchmark target?

It targets whether agentic AI systems can reason about image formation and solve computational imaging problems tied to optics, sensing, sampling, calibration, degradation, and ill-posed inverse reconstruction.

How does ImagingBench evaluate agentic imaging models?

It evaluates three complementary settings: Expert (expert-guided inverse reconstruction), Planner (planner-guided inverse reconstruction), and Forward (forward-system simulation for consistency checking).

What are the main findings about agentic models versus specialized methods?

Agentic models remain consistently weaker than specialized task-focused methods, especially on computational sensing tasks like lensless imaging, event-based reconstruction, time-of-flight imaging, and holography; planner guidance yields only modest and inconsistent improvements.

Sep 21, 2026

Uploaded by Levi