[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122508-en":3,"doc-seo-122508-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122508,687197207919,"Theodora","https://ap-avatar.wpscdn.com/avatar/a000253d6f5f7c60be?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779446848396160552",6,"Technology","Capsule Network Projectors are Equivariant and Invariant Learners - Abstract","Learning invariant representations has been a core direction in self-supervised learning, while recent work increasingly targets equivariance but often requires highly prescribed network designs. This work introduces CapsIE, an invariant-equivariant self-supervised architecture based on Capsule Networks, leveraging their ability to model viewpoint equivariance. An entropy-minimisation objective adapts the architecture to CapsNets and improves downstream results on equivariant tasks with better efficiency. The approach achieves state-of-the-art performance on 3DIEBench rotation while remaining competitive with supervised learning, with code released publicly.","Capsule Network Projectors are Equivariant and Invariant Learners  \nMiles Everett [miles. everett@abdn. ac.uk](miles. everett@abdn. ac.uk)  \nDepartment of Computing Science University of Aberdeen, UK  \nAiden Durrant [aiden. durrant@abdn. ac.uk](aiden. durrant@abdn. ac.uk)  \nDepartment of Computing Science University of Aberdeen, UK  \nMingjun Zhong [mingjun.zhong@abdn. ac.uk](mingjun.zhong@abdn. ac.uk)  \nDepartment of Computing Science University of Aberdeen, UK  \nGeorgios Leontidis [georgios.leontidis@abdn. ac.uk](georgios.leontidis@abdn. ac.uk)  \nInterdisciplinary Institute Department of Computing Science University of Aberdeen, UK  \nReviewed on OpenReview: [https: // openreview. net/ forum? id= 7owCO3qskH](https: // openreview. net/ forum? id= 7owCO3qskH)  \nAbstract  \nLearning invariant representations has been the long-standing approach to self-supervised learning. However, recently progress has been made in preserving equivariant properties in representations, yet do so with highly prescribed architectures. In this work, we propose an invariant-equivariant self-supervised architecture that employs Capsule Networks (CapsNets), which have been shown to capture equivariance with respect to novel viewpoints. We demonstrate that the use of CapsNets in equivariant self-supervised architectures achieves improved downstream performance on equivariant tasks with higher efficiency and fewer network parameters. To accommodate the architectural changes of CapsNets, we introduce a new objective function based on entropy minimisation. This approach, which we name CapsIE (Capsule Invariant Equivariant Network), achieves state-of-the-art performance on the equivariant rotation tasks on the 3DIEBench dataset compared to prior equivariant SSL methods, while performing competitively against supervised counterparts. Our results demonstrate the ability of CapsNets to learn complex and generalised representations for large-scale, multi-task datasets compared to previous CapsNet benchmarks. Code is available at [https://github. com/AberdeenML/CapsIE](https://github. com/AberdeenML/CapsIE).  \n1 Introduction  \nEquivariance and invariance have become increasingly important properties and objectives of deep learning in recent times, with precedence being largely placed on the latter. The task of invariance, i.e., being able to classify a specific object regardless of the camera perspective or augmentation applied, has driven progress in modern self-supervised learning approaches, specifically those that follow a joint embedding architecture (Assran et al., 2022; Bardes et al., 2022; Chen et al., 2020) . Equivariance, on the other hand, is the task of capturing embeddings which equally reflect the translations applied to the input space in the latent space. Equivariance thus has become an important property to capture to enable the learning of high-quality representations in the real world, where transformations such as viewpoint are essential.  \n(a) Schematic overview of the CapsIE architecture.  \n(b) Generalised visualisation of the CapsNet projector.  \nFigure 1: Left: Schematic overview of the proposed CapsIE architecture. Representations are fed into a CapsNet projector, and the output embeddings Zact and Zpose correspond to invariant and equivariant embeddings, respectively. Right: Generalised view of a Capsule projection head. CNN feature maps are transformed via the primary capsules into poses ui , represented by cylinders, and activations ai , represented by circles. Poses are transformed to votes, which represent a lower-level capsule’s prediction for each of the higher-level capsules. The routing process then determines how well these votes match the concept represented by the upper-level capsule, thereby creating the coupling coefficients. Coupling coefficients inform uj and aj , the output of the capsule projector head.  \nSelf-supervised learning owes its success to invariant objectives, where all recent progress, whether that is by contr","cbCaieNkEndGA6Ig","https://ap.wps.com/l/cbCaieNkEndGA6Ig","pdf",2069205,1,18,"English","en",105,"# Introduction\n## Invariance and equivariance in self-supervised learning\n## Why Capsule Networks for invariant-equivariant learning\n## CapsIE architecture overview\n## Objectives and training approach","[{\"question\":\"What problem does CapsIE address in self-supervised learning?\",\"answer\":\"CapsIE targets learning representations that are simultaneously invariant and equivariant, without relying on overly prescribed architectures.\"},{\"question\":\"How do Capsule Network projectors contribute to invariance and equivariance?\",\"answer\":\"The CapsNet projector produces invariant embeddings for one output and equivariant embeddings for another, leveraging capsule routing to encode viewpoint-related transformations.\"},{\"question\":\"What training objective is introduced for CapsNets in this work?\",\"answer\":\"The paper introduces a new objective function based on entropy minimisation to accommodate architectural changes required by CapsNets.\"}]","Capsule Network Projectors are Equivariant and Invariant Learners - Abstract | PDF",1785811006,45,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"capsule-network-projectors-are-equivariant-and-invariant-learners-abstract","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/technology/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/capsule-network-projectors-are-equivariant-and-invariant-learners-abstract/122508/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does CapsIE address in self-supervised learning?","Question",{"text":75,"@type":76},"CapsIE targets learning representations that are simultaneously invariant and equivariant, without relying on overly prescribed architectures.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do Capsule Network projectors contribute to invariance and equivariance?",{"text":80,"@type":76},"The CapsNet projector produces invariant embeddings for one output and equivariant embeddings for another, leveraging capsule routing to encode viewpoint-related transformations.",{"name":82,"@type":73,"acceptedAnswer":83},"What training objective is introduced for CapsNets in this work?",{"text":84,"@type":76},"The paper introduces a new objective function based on entropy minimisation to accommodate architectural changes required by CapsNets.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,113,118,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":111,"slug":112},50,"technology",{"id":114,"doc_module":4,"doc_module_name":46,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":46,"category_name":120,"show_sort_weight":121,"slug":122},8,"Research & Report",30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]