[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-565871-105":3,"detail-sidebar-cat-1-en-105":84,"doc-detail-565871-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":77,"head_meta":79,"extra_data":81,"updated_unix":83},105,"en","composition-grounded-data-synthesis-for-visual-reasoning-read-online-free","COMPOSITION-GROUNDED DATA SYNTHESIS FOR VISUAL REASONING - read online free","","Pretrained multi-modal large language models (MLLMs) perform well on many multimodal tasks, yet struggle with advanced reasoning where annotated query-answer data is hard to collect. This work targets artificial image domains like charts, rendered documents, and webpages. It introduces COGS, a data-efficient framework that decomposes a small seed set into primitive perception and reasoning factors, then recombines factors with new images to synthesize large question-answer collections. Reinforcement learning uses factor-level process rewards, improving transfer across datasets and extending beyond charts.",{"@graph":14,"@context":76},[15,34,55],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/template/","Template",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/template/general/","General",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/template/composition-grounded-data-synthesis-for-visual-reasoning-read-online-free/565871/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/composition-grounded-data-synthesis-for-visual-reasoning-read-online-free/565871.png","ImageObject",442,249,{"name":42,"@type":43},"Ophelia","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-22","2026-10-13",true,{"@type":52,"interactionType":53,"userInteractionCount":30},"InteractionCounter",{"@type":54},"ViewAction",{"@type":56,"mainEntity":57},"FAQPage",[58,64,68,72],{"name":59,"@type":60,"acceptedAnswer":61},"What problem does COGS address for multimodal models?","Question",{"text":62,"@type":63},"C OGS addresses the gap between strong general performance of MLLMs and limited reasoning ability in domains where collecting large-scale annotated reasoning datasets is difficult.","Answer",{"name":65,"@type":60,"acceptedAnswer":66},"How does COGS generate training data efficiently?",{"text":67,"@type":63},"COGS decomposes each seed question into primitive perception and reasoning factors, then recombines discovered factors with new images to synthesize large collections of question-answer pairs.",{"name":69,"@type":60,"acceptedAnswer":70},"What training signal does COGS use to improve reasoning?",{"text":71,"@type":63},"COGS fine-tunes a pretrained MLLM with Group Relative Policy Optimization (GRPO), augmented with process rewards derived from factor annotations for fine-grained supervision.",{"name":73,"@type":60,"acceptedAnswer":74},"Does the approach work only for charts?",{"text":75,"@type":63},"No. The work demonstrates that the same framework extends beyond chart reasoning to other domains such as webpage reasoning.","https://schema.org",{"og:url":32,"og:type":78,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":80,"canonical":32},"index,follow",{"doc_id":82,"site_id":7},565871,1792546217,{"code":4,"msg":85,"data":86},"success",[87,92,97,102,107,112,117,122,127],{"id":88,"doc_module":22,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},11,"Presentations",90,"presentations",{"id":93,"doc_module":22,"doc_module_name":25,"category_name":94,"show_sort_weight":95,"slug":96},12,"Resumes",80,"resumes",{"id":98,"doc_module":22,"doc_module_name":25,"category_name":99,"show_sort_weight":100,"slug":101},14,"Invoices",70,"invoices",{"id":103,"doc_module":22,"doc_module_name":25,"category_name":104,"show_sort_weight":105,"slug":106},15,"Posters",60,"posters",{"id":108,"doc_module":22,"doc_module_name":25,"category_name":109,"show_sort_weight":110,"slug":111},16,"Social Media",50,"social-media",{"id":113,"doc_module":22,"doc_module_name":25,"category_name":114,"show_sort_weight":115,"slug":116},17,"Forms",40,"forms",{"id":118,"doc_module":22,"doc_module_name":25,"category_name":119,"show_sort_weight":120,"slug":121},18,"Letters",30,"letters",{"id":123,"doc_module":22,"doc_module_name":25,"category_name":124,"show_sort_weight":125,"slug":126},21,"Paper Templates",5,"papers-templates",{"id":128,"doc_module":22,"doc_module_name":25,"category_name":29,"show_sort_weight":4,"slug":129},158,"general-158",{"code":4,"msg":85,"data":131},{"doc_id":82,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":22,"category_id":128,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":30,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":144,"read_time":145},7971461741311,"https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826","COMPOSITION-GROUNDED DATA SYNTHESIS FOR VISUAL REASONING  \nXinyi Gu 1 ∗ Jiayuan Mao 1 Zhang-Wei Hong 1 Zhuoran Yu2 Pengyuan Li3 Dhiraj Joshi3 Rogerio Feris3 Zexue He 1 ,3  \n1MIT 2UW-Madison 3MIT-IBM Watson AI Lab  \nABSTRACT  \nPretrained multi-modal large language models (MLLMs) demonstrate strong performance on diverse multimodal tasks, but remain limited in reasoning capabilities for domains where annotations are difficult to collect. In this work, we focus on artificial image domains such as charts, rendered documents, and webpages, which are abundant in practice yet lack large-scale human annotated reasoning datasets. We introduce COGS (COmposition-Grounded data Synthesis), a dataefficient framework for equipping MLLMs with advanced reasoning abilities from a small set of seed questions. The key idea is to decompose each seed question into primitive perception and reasoning factors, which can then be systematically recomposed with new images to generate large collections of synthetic questionanswer pairs. Each generated question is paired with subquestions and intermediate answers, enabling reinforcement learning with factor-level process rewards.  \nExperiments on chart reasoning show that COGS substantially improves performance on unseen questions, with the largest gains on reasoning-heavy and compositional questions. Moreover, training with a factor-level mixture of different seed data yields better transfer across multiple datasets, suggesting that COGS induces generalizable capabilities rather than dataset-specific overfitting. We further demonstrate that the framework extends beyond charts to other domains such as webpages. Project page: [https://cogsynthesis.github.io](https://cogsynthesis.github.io).  \n1 INTRODUCTION  \nPretrained multi-modal large language models (MLLMs) have achieved impressive performance across a wide range of multimodal tasks (Liu et al., 2023c ; Bai et al., 2025 ; Wang et al., 2025a ; Agrawal et al., 2024 ; OpenAI et al., 2024 ; Comanici et al., 2025 ; Anthropic, 2024), yet advanced reasoning capabilities remain underdeveloped, especially in domains where user reasoning-intensive query-answer data is difficult to collect. In this work, we consider reasoning capability over artificial image domains, including charts, tables, information graphs, rendered documents, webpages, etc. While such images are abundant on the web, datasets containing reasoning questions over them are scarce. However, developing MLLMs that can handle these reasoning queries is critical for downstream applications, such as building agents that interpret and edit documents or take actions in digital environments.  \nIn this paper, we aim to equip MLLMs with these missing capabilities using only a small set of seed questions in a target domain. Our goal is to bootstrap from these seed questions to generate a large, diverse dataset of synthetic question-answer pairs, leveraging additional unlabeled images such as online charts or webpages. The key insight of our approach is compositionality: although a seed question set may contain only a limited number of surface forms, each question can be decomposed into a set of smaller subquestions, which we call factors in this paper. Factors may capture primitive perception and reasoning steps, such as reading a number from a chart, comparing two entries ina table, or performing arithmetic. Crucially, these factors can be recombined systematically and compositionally to produce a much larger space of complex questions.  \nWe propose a data-efficient framework, COGS (COmposition-Grounded data Synthesis), that operationalizes this idea in three stages. First, given a seed dataset of questions in the target domain, we  \n∗Work done during an internship at MIT-IBM Watson AI Lab. Corresponding to gxy@mit .edu, [zexueh@mit.edu](zexueh@mit.edu).  \nSeed Dataset Factors COGS Synthetic Dataset  \n\n| \u003Cbr>|  |  |\n| --- | --- | --- |\n| | | |\n| \u003Cbr>|  |  |\n|  |  |  |\n\nFigure 1: COGS: Starting from a small","cbCailE4QEw6fj4J","https://ap.wps.com/l/cbCailE4QEw6fj4J","pdf",3528572,25,"English","# Abstract\n# Introduction\n## Motivation and problem setting\n## Compositional bootstrapping with seed questions\n## COGS framework and training stages\n# Related Work\n## Grounded perception and visual reasoning","[{\"question\":\"What problem does COGS address for multimodal models?\",\"answer\":\"C OGS addresses the gap between strong general performance of MLLMs and limited reasoning ability in domains where collecting large-scale annotated reasoning datasets is difficult.\"},{\"question\":\"How does COGS generate training data efficiently?\",\"answer\":\"COGS decomposes each seed question into primitive perception and reasoning factors, then recombines discovered factors with new images to synthesize large collections of question-answer pairs.\"},{\"question\":\"What training signal does COGS use to improve reasoning?\",\"answer\":\"COGS fine-tunes a pretrained MLLM with Group Relative Policy Optimization (GRPO), augmented with process rewards derived from factor annotations for fine-grained supervision.\"},{\"question\":\"Does the approach work only for charts?\",\"answer\":\"No. The work demonstrates that the same framework extends beyond chart reasoning to other domains such as webpage reasoning.\"}]","COMPOSITION-GROUNDED DATA SYNTHESIS FOR VISUAL REASONING - read online free | PDF",1791859701,9]