[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85714-en":3,"doc-seo-85714-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85714,4398048950312,"Violet","https://ap-avatar.wpscdn.com/avatar/400002538284de19e3c?_k=1778320343897328908",8,"Research & Report","Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models","A systematic review addresses continual self-supervised learning (CSSL) for vision models, motivated by deployments that require continuous adaptation from unlabeled data streams rather than labeled, task-annotated datasets. The survey analyzes evaluation protocols and exposes inconsistencies that impede fair comparisons. It explains improved robustness to catastrophic forgetting by linking self-supervised objectives to task-agnostic representations and smoother loss landscapes. A unified taxonomy organizes methods by forgetting-mitigation strategies and highlights open scalability and fast adaptability challenges, advocating continual pre-training for large-scale systems.","arXiv :2607 .09785v 1 [ cs .CV] 8 Jul 2026  \nLifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models  \nSergi Masip∗ , Alicja Dobrzeniecka†, Jonathan Swinnen∗ , Joachim Collin∗ , Bartłomiej Twardowski ‡, Szymon Łukasik† and Tinne Tuytelaars∗  \n∗ PSI, KU Leuven, Leuven, Belgium  \nEmail: {sergi.masipcabeza, jonathan.swinnen, joachim.collin, [tinne.tuytelaars}@kuleuven.be](tinne.tuytelaars}@kuleuven.be)[ ](tinne.tuytelaars}@kuleuven.be)†NASK -National Research Institute, Warsaw, Poland  \nEmail: {alicja.dobrzeniecka, [szymon.lukasik}@nask.pl](szymon.lukasik}@nask.pl)  \n‡ Computer Vision Center, Universitat Autonoma de Barcelona, Barcelona, Spain  \nIDEAS Research Institute, Warsaw, Poland  \nEmail: [btwardowski@cvc.uab.cat](btwardowski@cvc.uab.cat)  \nAbstract—Traditionally, continual learning has assumed access to labeled data, yet many real-world applications—such as lifelong robotics—require models to adapt continuously from unlabeled streams. This has led to the development of continual self-supervised learning (CSSL), a rapidly growing area that lacks a dedicated, systematic review. In this work, we present a comprehensive survey of CSSL for vision, with connections to emerging vision-language settings. First, we analyze existing evaluation protocols and highlight inconsistencies that hinder fair comparison. We then examine why self-supervised objectives exhibit improved robustness to catastrophic forgetting, relating this to task-agnostic representations and smoother loss landscapes. Next, we organize existing methods into a unified taxonomy based on their forgetting-mitigation strategies, including distillation, replay, regularization, architectural approaches, model merging, and objective-level adaptation. Finally, we identify open challenges such as scalability and the need for fast adaptability. We argue that advancing CSSL requires moving beyond small-scale benchmarks towards continual pre-training paradigms for large-scale systems. Index Terms—self-supervised learning, representation learning, continual learning, online learning, pretraining, computer vision  \nI. INTRODUCTION  \nMost machine learning models are trained under a static paradigm, assuming the entire training dataset is available upfront. However, many real-world deployment scenarios challenge this assumption. Consider, for example, a robot navigation policy powered by a vision foundation model backbone that has been trained for logistics and delivery in North America and Europe. These environments are well represented in current datasets [1] . However, when deployed in, for example, Sub-Saharan Africa, the model encounters new object appearances, road layouts and lighting conditions that were largely absent from its training data. The industry standard is to periodically retrain the model from scratch, which is wasteful. Yet the alternative–continually adapting the model– poses a fundamental challenge: how can new knowledge be incorporated without overwriting the old?  \nContinual Learning (CL) addresses this challenge by enabling models to learn from data as it arrives over time, while also reducing the risk of catastrophic forgetting. [2] . However,  \nFig. 1: A conceptual visualization of CSSL. The bottom part represents tasks T arriving in sequence, and the top part shows an illustration of an SSL objective (in this example, non-contrastive) .  \nmost existing studies on CL focus on supervised settings, such as class-incremental learning [3], thereby sidestepping a key practical issue: in real-world deployments, labels are scarce, expensive, and often unavailable. A robot adapting toa new environment cannot pause to wait for human annotators. A foundation model ingesting a new data stream cannot assume curated, task-annotated inputs. Unlabeled visual data, by contrast, is generated at massive scales wherever these systems are deployed. Self-supervised learning, which extracts supervisory signals from the structure of the dat","cbCaid22VpriVobW","https://ap.wps.com/l/cbCaid22VpriVobW","pdf",653629,1,10,"English","en",105,"# Introduction\n# Evaluation Protocols and Metrics\n# Forgetting in Self-Supervised Learning\n# Taxonomy of CSSL Methods\n## Distillation\n## Replay\n## Regularization\n## Architectural Approaches\n## Model Merging\n## Objective-Level Adaptation\n# Open Challenges and Future Directions","[{\"question\":\"What gap does the survey target in continual learning research?\",\"answer\":\"The survey targets the lack of a dedicated, systematic review for continual self-supervised learning in vision, where models adapt continuously from unlabeled streams without task labels.\"},{\"question\":\"Why does the survey argue that self-supervised objectives can be more robust to catastrophic forgetting?\",\"answer\":\"It links this robustness to task-agnostic representations and smoother loss landscapes, which help mitigate overwriting old knowledge during continual training.\"},{\"question\":\"How does the survey organize existing CSSL methods?\",\"answer\":\"It groups methods into a unified taxonomy based on forgetting-mitigation strategies, including distillation, replay, regularization, architectural approaches, model merging, and objective-level adaptation.\"}]",1784205758,25,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"lifelong-representations-a-survey-on-continual-self-supervised-learning-for-vision-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/lifelong-representations-a-survey-on-continual-self-supervised-learning-for-vision-models/85714/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What gap does the survey target in continual learning research?","Question",{"text":75,"@type":76},"The survey targets the lack of a dedicated, systematic review for continual self-supervised learning in vision, where models adapt continuously from unlabeled streams without task labels.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why does the survey argue that self-supervised objectives can be more robust to catastrophic forgetting?",{"text":80,"@type":76},"It links this robustness to task-agnostic representations and smoother loss landscapes, which help mitigate overwriting old knowledge during continual training.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the survey organize existing CSSL methods?",{"text":84,"@type":76},"It groups methods into a unified taxonomy based on forgetting-mitigation strategies, including distillation, replay, regularization, architectural approaches, model merging, and objective-level adaptation.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":21,"slug":133},"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]