[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-136436-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-136436-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","v-mage-a-game-evaluation-framework-for-assessing-vision-centric-capabilities-in-multimodal-large-language-models-paper","V-MAGE - A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models - Paper","","V-MAGE is a game-based evaluation framework for vision-centric capabilities in multimodal large language models (MLLMs), targeting shortcomings of static image-text benchmarks. The benchmark contains five distinct video games and 30+ carefully designed scenarios set in free-form, visually complex, continuous-space environments. Models must interpret dynamic game states and decide using visual input only. A dynamic ELO-based ranking system enables robust, interpretable comparisons across difficulties and task diversity, revealing persistent performance gaps versus human baselines, especially on complex reasoning and task orchestration.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/v-mage-a-game-evaluation-framework-for-assessing-vision-centric-capabilities-in-multimodal-large-language-models-paper/136436/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/v-mage-a-game-evaluation-framework-for-assessing-vision-centric-capabilities-in-multimodal-large-language-models-paper/136436.png","ImageObject",300,407,{"name":42,"@type":43},"Olivia Brown","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-21","2026-08-22",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",19,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"What problem does V-MAGE address in evaluating multimodal large language models?","Question",{"text":63,"@type":64},"V-MAGE targets the insufficiency of static image-text benchmarks by evaluating dynamic perception and interactive reasoning in continuous, visually complex environments.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"How is V-MAGE structured?",{"text":68,"@type":64},"V-MAGE includes five video games with 30+ evaluation scenarios designed to require decisions based solely on visual input.",{"name":70,"@type":61,"acceptedAnswer":71},"How are models compared within V-MAGE?",{"text":72,"@type":64},"V-MAGE uses a dynamic ELO-based ranking system that accounts for varying difficulty levels and task diversity to produce interpretable comparisons.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},136436,1787386800,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,120,123,127],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":116,"doc_module":4,"doc_module_name":25,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":25,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":25,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":80,"read_time":144},16904993612988,"https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd","V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models  \nXiangxi Zheng1 , Linjie Li2 , Zhengyuan Yang2 , Ping Yu1 , Alex Jinpeng Wang3†, Rui Yan4†, Yuan Yao1 , Lijuan Wang2  \n1Nanjing University, 2Microsoft Research,  \n3 Central South University, 4Nanjing University of Science and Technology  \n[zhengxx@nju.edu.cn](zhengxx@nju.edu.cn) , [jinpengwang@csu.edu.cn](jinpengwang@csu.edu.cn) , [ruiyan@njust.edu.cn](ruiyan@njust.edu.cn)  \narXiv :2504 .06148v3 [ cs .CV] 24 Apr 2026  \nAbstract  \nRecent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in visual-text processing. However, existing static image-text benchmarks are insufficient for evaluating their dynamic perception and interactive reasoning abilities. We introduce Vision-centric Multiple Abilities Game Evaluation (V-MAGE), a novel game-based evaluation framework designed to systematically assess MLLMs’ visual reasoning in interactive, continuous-space environments. V-MAGE features five distinct video games comprising over 30 carefully constructed evaluation scenarios. These scenarios are set in free-form, visually complex environments that require models to interpret dynamic game states and make decisions based solely on visual input, thereby closely reflecting the conditions encountered by human players. To ensure robust and interpretable comparisons across models, V-MAGE employs a dynamic ELO-based ranking system that accounts for varying difficulty levels and task diversity. Benchmarking state-of-the-art MLLMs against human baselines reveals that while leading models approach human-level performance in simple tasks, their performance drops significantly in complex scenarios requiring advanced reasoning and task orchestration. This persistent performance gap highlights fundamental limitations in current MLLMs’ ability to perform vision-grounded, interactive frameby-frame control in simulated continuous-time environments. Through extensive analyses, we demonstrate the utility of V-MAGE in uncovering these limitations and providing actionable insights for improving the visual and reasoning capabilities of MLLMs in dynamic, interactive settings. Code is publicly available at [https://github.com/CSU-JPG/V-MAGE](https://github.com/CSU-JPG/V-MAGE).  \n†Corresponding authors: Alex Jinpeng Wang, Rui Yan  \n1 Introduction  \nBuilding on the success of Large Language Models (LLMs) in text-based tasks (Bai et al., 2023 ; Caiet al., 2024 ; OpenAI, 2023a), researchers have extended their capabilities to visual-text multimodal tasks through Multimodal Large Language Models (MLLMs) (OpenAI, 2023b ; Liu et al., 2023a ; Team, 2023 ; Yang et al., 2023 ; Li et al., 2024 ; Wanget al., 2024 ; Bai et al., 2025a ; Zhu et al., 2025) . Various multimodal evaluation benchmarks, such as MME (Fu et al., 2023), MMBench (Liu et al., 2023b), SEED-Bench (Li et al., 2023) have driven improvements in MLLM performance. With improving model capabilities, researchers are shifting toward open-world, dynamic, multi-round tasks beyond static benchmarks with fixed image-text inputs, as these better reflect real-world interaction and reasoning challenges. Among the promising approaches for evaluating models in such dynamic settings, game-based evaluation has emerged as a promising alternative, offering a more natural and interactive assessment of a model’s perception and reasoning abilities.  \nWhile progress has been made in game-based MLLM benchmarks, current approaches predominantly rely on text-based (Costarelli et al., 2024 ; Hu et al., 2024 ; Duan et al., 2024) or gridbased (Zhang et al., 2024 ; Wang et al., 2025a ; Paglieri et al., 2024) games. In such settings, limited visual reasoning demands and static, fully textually renderable content restrict evaluation of spatial, temporal, and dynamic complexities crucial for real-world problem-solving. In contrast, the rich visual information inherent in video","cbCaihi2K3fDzbvN","https://ap.wps.com/l/cbCaihi2K3fDzbvN","pdf",7016641,52,"English","# Introduction\n## Motivation for vision-centric dynamic evaluation\n## Limitations of existing game benchmarks\n## Overview of V-MAGE and contributions","[{\"question\":\"What problem does V-MAGE address in evaluating multimodal large language models?\",\"answer\":\"V-MAGE targets the insufficiency of static image-text benchmarks by evaluating dynamic perception and interactive reasoning in continuous, visually complex environments.\"},{\"question\":\"How is V-MAGE structured?\",\"answer\":\"V-MAGE includes five video games with 30+ evaluation scenarios designed to require decisions based solely on visual input.\"},{\"question\":\"How are models compared within V-MAGE?\",\"answer\":\"V-MAGE uses a dynamic ELO-based ranking system that accounts for varying difficulty levels and task diversity to produce interpretable comparisons.\"}]","V-MAGE - A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models - Paper | PDF",131]