OmniFood-Bench: 评估VLM在营养推理与个性化健康建议中的表现

Jul 16, 2026

Upload by Aurelia

Rapid integration of large vision-language models (VLMs) into food and healthcare promises more accurate dietary management, yet real systems face systemic information asymmetry between visual appearance and intrinsic nutritional composition. Existing benchmarks mainly test coarse food recognition and miss the full reasoning chain from hidden ingredients to portion mass and safety-critical medical guidance. OmniFood-Bench is built from MM-Food-100K and evaluates progressive abilities: Basic Perception, Quantitative Reasoning, and Safety-Critical Advisory. Experiments on six VLMs reveal a Semantic-Physical Gap, including mass estimation failures and benign hallucinations for high-risk diabetes profiles, establishing a trustworthiness standard for public health agents.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages7

Reading Time18 min

Introduction

Systemic information asymmetry and visual-to-physical inference

Risks and need for safety alignment

Limitations of existing food benchmarks

Proposed progressive evaluation tasks

OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice Qian Jiang 1 , Zhecheng Shi2 , Jingpu Yang3 , Zirui Song4 , Miao Fang 1* 1Northeastern University at Qinhuangdao 2The Hong Kong University of Science and Technology (Guangzhou) 3BeiHang University 4Mohamed bin Zayed University of Artificial Intelligence arXiv :2607 .08423v 1 [ cs .AI] 9 Jul 2026 Abstract—The rapid integration of Large Vision-Language Models (VLMs) into critical infrastructure promises to revolutionize personalized healthcare and dietary management. However, in the domain of food systems, autonomous agents face a unique and persistent challenge: the “Systemic Information Asymmetry” between visual appearance and intrinsic nutritional composition. Existing benchmarks primarily focus on coarse-grained classification tasks, such as food category recognition, which fail to evaluate the intricate reasoning chain required for real-world dietary management—specifically, the ability to traverse from identifying hidden ingredients to estimating physical mass, and finally synthesizing safety-critical medical advice. In this paper, we introduce OmniFood-Bench, a comprehensive benchmark constructed from the MM-Food- 100K dataset. Unlike previous works, OmniFood-Bench evaluates VLMs across three progressive capabilities: Basic Perception (Ingredients & Cooking Methods), Quantitative Reasoning (Portion Size & Nutritional Profiling), and SafetyCritical Advisory (Disease-Specific Recommendations). We evaluate six state-of-the-art VLMs, including gpt-5.1, gemini-3-flash, and qwen3-vl-8B. Our extensive experiments reveal a startling “Semantic-Physical Gap”: while models achieve nearhuman accuracy in naming dishes, they exhibit catastrophic failure in mass estimation and frequently hallucinate benign advice for high-risk diabetic profiles. This work establishes a rigorous standard for trustworthiness in autonomous agents deployed for public health. The code and datasets are available in:[https://anonymous.4open.science/r/OmniFood-Bench-7D0B](https://anonymous.4open.science/r/OmniFood-Bench-7D0B) Index Terms—Vision-Language Models, Autonomous Agents, Food Computing, Nutritional Reasoning, Benchmark I. INTRODUCTION In the era of intelligent media, autonomous agents are increasingly tasked with interpreting high-dimensional multimodal data to assist in complex daily decision-making processes [1] . Among the myriad applications of these agents, AI-driven dietary management holds immense potential for combating the rising global burden of chronic diseases such as obesity, diabetes, and hypertension. [2], [3] A theoretical”AI Dietitian” agent should possess the capability to perceive a meal via a camera, analyze its nutritional composition with clinical precision, and offer personalized advice tailored to the user’s specific physiological state. The realization of such a system would democratize access to personalized nutrition, *Corresponding author: Miao Fang. moving beyond generic calorie counting to risk-aware health surveillance. However, the deployment of such autonomous agents is currently hindered by significant challenges regarding robustness, data trust, and safety [4], [5] . Unlike general object detection tasks where identifying a bounding box suffices,[6] food understanding requires a complex process of Visualto-Physical Inference. An agent must deduce the invisible from the visible: determining whether a dish is deep-fried or steamed (which drastically alters caloric density), estimating the physical mass of a steak from 2D pixels (resolving scale and depth ambiguity), and retrieving domain-specific medical knowledge to warn a diabetic user about potential hidden sugars in a glaze. This creates a ”Systemic Information Asymmetry” where the visual signal alone is often insufficient without robust reasoning and world knowledge. The stakes in this domain are exceptionally high. In typical vision tasks, a misclassif

FAQs

What core problem does OmniFood-Bench target?

It targets the “Systemic Information Asymmetry” between what food looks like and its intrinsic nutritional composition, which makes visual-only understanding insufficient for health-safe recommendations.

How does OmniFood-Bench evaluate VLM capabilities?

It uses a progressive pipeline with three capabilities: Basic Perception (ingredients and cooking methods), Quantitative Reasoning (portion size and nutritional profiling), and Safety-Critical Advisory (disease-specific recommendations).

What major failure pattern do experiments show?

Models show a “Semantic-Physical Gap”: they can name dishes with near-human accuracy but catastrophically fail in mass estimation and may hallucinate benign advice even for high-risk diabetic profiles.

Jul 16, 2026

Upload by Aurelia