[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82547-en":3,"doc-seo-82547-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82547,549758146520,"Patrick","https://ap-avatar.wpscdn.com/avatar/80002397d8c0411e94?_k=1775819394049821470",8,"Research & Report","Continual Evolution of Vision Language Models via Mistake Notebook Learning","Vision Language Models (VLMs) excel at multimodal reasoning but repeatedly fail in ways like skipping crucial visual checks, misapplying domain rules, and hallucinating unsupported concepts. Existing Supervised Fine-Tuning and Reinforcement Learning are costly to iterate and often brittle under distribution shift. M2Note introduces a training-free continual evolution framework that converts failed trajectories into editable subject–guidance notebook notes. During inference, multimodal RAG retrieves relevant guidance, while batch post-verification with rollback stabilizes updates.","M2Note: Continual Evolution of Vision Language Models via Mistake  \nNotebook Learning  \nHaiwen Li* Jing Tang† Rui Chen Lei Sun Xiangxiang Chu  \nAMAP, Alibaba Group  \narXiv :2607 .00685v 1 [ cs .MA] 1 Jul 2026  \nAbstract  \nVision Language Models (VLMs) have demonstrated remarkable capabilities in multimodal reasoning tasks, yet they still suffer from recurring failures, such as skipping key visual checks, misapplying domain rules, and hallucinating unsupported concepts. Most existing solutions rely on supervised fine-tuning (SFT) and reinforcement learning (RL), which are expensive to iterate and can be brittle under distribution shift. To this end, we propose Multimodal Mistake Notebook Learning (M2Note), a training-free continual evolution framework that externalizes learning into an editable memory. M2Note transforms failed trajectories into compact subject–guidance notes: the subject summarizes the underlying domain and concept, while the guidance provides actionable verification steps that can be reused in future inference. At test time, M2Note retrieves relevant notes via multimodal retrievalaugmented generation (RAG) and appends them to the model context, steering reasoning away from previously observed pitfalls. To stabilize continual evolution, we adopt batch-level post-verification with rollback, which commits notebook edits only if they improve performance on the same batch, reducing noisy updates and preventing regressions. M2Note supports both self-evolving, where the same VLM acts as solver and supervisor, and cross-model evolving, where a stronger supervisor guidesa weaker solver, enabling capability transfer without weight updates. Experiments on six multimodal reasoning benchmarks show consistent improvements across domains and backbones, while achieving strong cost and sample efficiency and remaining complementary to Chain-of-Thought (CoT) prompting.  \n*Work done during the internship at AMAP, Alibaba.†Project Lead.  \n1 Introduction  \nVision Language Models (VLMs) (Bordes et al., 2024 ; Hurst et al., 2024 ; Bai et al., 2025 ; Guo et al., 2025 ; Liu et al., 2023) have become a general interface for multimodal tasks, including STEM reasoning (Yue et al., 2024 ; Lu et al., 2023 ; Wanget al., 2024 ; Qiao et al., 2025), chart and diagram understanding (Masry et al., 2022 ; Kembhavi et al., 2016), document OCR (Liu et al., 2024c ; Yang et al., 2025), visual question answering (Liu et al., 2024b ; Chen et al., 2024 ; xAI), and video understanding (Li et al., 2024 ; Zhao et al., 2025 ; Yuan et al., 2026) . Despite strong progress, VLMs in realistic settings still exhibit repeatable failure modes, such as missing key visual evidence or over-relying on superficial cues. Correcting such behaviors efficiently, without sacrificing robustness or requiring costly retraining, remains a practical challenge.  \nThe dominant adaptation paradigm is parameterbased post-training, such as supervised fine-tuning (SFT) (Liu et al., 2023 ; Wei et al., 2022a) and reinforcement learning (RL) (Schulman et al., 2017 ; Rafailov et al., 2023 ; Shao et al., 2024 ; Yu et al., 2025) . Although effective, these methods are expensive, slow to iterate, and prone to regression. More importantly, once model weights are updated, test-time behavior becomes fixed, making continual improvement difficult in dynamic environments. This motivates growing interest in training-free adaptation through in-context steering.  \nExisting training-free approaches mainly fall into two categories: ❶ prompt optimization methods (Zhou et al., 2022 ; Yang et al., 2023 ; Pryzantet al., 2023) refine global instructions, but often provide advice that is too coarse to target recurring errors across diverse multimodal tasks; ❷ memorybased methods (Zhao et al., 2024 ; Zhang et al., 2024a ; Shinn et al., 2023 ; Zhang et al., 2025a) store instance-level experiences for retrieval, but they often lack abstraction, leading to redundant  \nFigure 1: Overview of M2Note. (Left) The syst","cbCaifiYHsFfEjkz","https://ap.wps.com/l/cbCaifiYHsFfEjkz","pdf",2413218,1,19,"English","en",105,"# Introduction\n## Motivation and problem setting\n## Training-based adaptation limitations\n## Training-free in-context adaptation\n## Proposed approach: M2Note","[{\"question\":\"What recurring failure modes does M2Note target in VLMs?\",\"answer\":\"M2Note targets repeatable issues such as skipping key visual evidence, misapplying domain rules, and generating unsupported hallucinated concepts in multimodal reasoning.\"},{\"question\":\"How does M2Note adapt VLM behavior without retraining model weights?\",\"answer\":\"M2Note uses a training-free continual evolution framework that stores structured subject–guidance notes from failures in an external editable mistake notebook. At test time, it retrieves relevant notes via multimodal RAG and appends them to the model context to steer reasoning.\"},{\"question\":\"How does M2Note prevent unstable or noisy notebook updates during continual evolution?\",\"answer\":\"M2Note employs batch-level post-verification with rollback: notebook edits are committed only when they improve performance on the same batch, reducing noisy updates and avoiding regressions.\"}]",1784181463,48,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"continual-evolution-of-vision-language-models-via-mistake-notebook-learning","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/continual-evolution-of-vision-language-models-via-mistake-notebook-learning/82547/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What recurring failure modes does M2Note target in VLMs?","Question",{"text":75,"@type":76},"M2Note targets repeatable issues such as skipping key visual evidence, misapplying domain rules, and generating unsupported hallucinated concepts in multimodal reasoning.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does M2Note adapt VLM behavior without retraining model weights?",{"text":80,"@type":76},"M2Note uses a training-free continual evolution framework that stores structured subject–guidance notes from failures in an external editable mistake notebook. At test time, it retrieves relevant notes via multimodal RAG and appends them to the model context to steer reasoning.",{"name":82,"@type":73,"acceptedAnswer":83},"How does M2Note prevent unstable or noisy notebook updates during continual evolution?",{"text":84,"@type":76},"M2Note employs batch-level post-verification with rollback: notebook edits are committed only when they improve performance on the same batch, reducing noisy updates and avoiding regressions.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]