[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85910-en":3,"doc-seo-85910-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85910,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","Emergent Generalization by Representation Learning in Artificial Neural Networks","Dimensionality reduction identifies neural manifolds—low-dimensional structures beneath high-dimensional activity—yet their biological relevance and functional role in learning remain disputed. This study shows that an explicit information bottleneck training a recurrent neural network to learn a low-dimensional representation is necessary for rotational and out-of-distribution generalisation in time-series prediction. Information-theoretic causal-emergence measures reveal non-monotonic representation dynamics across memorisation-to-generalisation, scaling with task complexity. Analogous non-monotonic dynamics in mouse CA1 activity track behavioural performance, linking compact emergent representations to generalisation advantage in cognition.","arXiv :2607 . 10430v1 [ q-bio .NC] 11 Jul 2026  \nEmergent Generalization by Representation Learning in Artificial Neural Networks  \nHardik Rajpal 1,2,* and Dan Goodman2  \n1 I-X Centre for AI in Science, Imperial College London, W12 0BZ, UK  \n2 Department of Electrical and Electronic Engineering, Imperial College London, SW7 2AZ, UK  \n* [h.rajpal15@imperial.ac.uk](h.rajpal15@imperial.ac.uk)  \nABSTRACT  \nDimensionality reduction has proven powerful for identifying neural manifolds, which are low-dimensional structures underlying high-dimensional neural activity. These low-dimensional representations have improved the interpretability of population-level coding. Yet whether such low-dimensional representations are biologically relevant and confer functional advantages in learning systems, or merely reflect neuron-level activity, remains contested in neuroscience. We show that an explicit information bottleneck forcing a recurrent neural network to learn a low-dimensional representation is necessary for rotational and out-ofdistribution generalisation in a time-series prediction task. Using information-theoretic measures of causal emergence, we characterise the dynamics of this representation across the memorisation-to-generalisation transition, finding a non-monotonic trajectory which shows an initial decrease, a minimum, and a subsequent rise to a maximum, even as prediction loss falls monotonically. This trajectory scales with task complexity, and the magnitude of emergent structure reliably predicts generalisation performance. Analysis of CA1 hippocampal activity in mice learning an alternating maze task reveals analogous non-monotonic emergence dynamics that track behavioural performance. Together, these findings indicate that the ability of neural networks to learn compact, distributed and emergent representations confers a functional advantage for generalisation, supporting a causal role for learned representations in cognition.  \nIntroduction  \nWith the advent of large-scale neural recordings, vast amounts of high-dimensional neural data recorded at high temporal resolution have become available from a variety of different brain regions and species. To interpret these complex datasets, dimensionality reduction techniques have been used to identify low-dimensional structures underlying high-dimensional neural activity 1 , which are referred to as neural manifolds2. With repeatable experimental paradigms, stable and persistent neural manifolds have been identified across different brain regions corresponding to different cognitive and behavioural tasks, such as olfaction3, 4 , motor control5, 6 , spatial navigation7 , and even intrinsic neural dynamics8. The presence of these low-dimensional manifolds across tasks and species has led to greater interpretability of neural dynamics and has unveiled the geometric structure of neural computations.  \nMore recently, experiments have sought to establish a direct causal relationship between neural manifolds and behaviour by perturbing neurons within them and observing the effects on behaviour. For example, head orientation of the fruit fly can be manipulated optogenetically by perturbing the neurons in the ring attractor manifold9. Similar perturbation experiments have been performed to impact complex behaviours such as motor timing 10 and adaptive task learning 11 in mice. Complementarily, studies using fixed linear decoders have shown that behavioural variables can be reliably decoded from low-dimensional manifolds in the same animal up to years after initial training in monkeys 12 and across twin C. Elegans 13. These studies suggest that the low-dimensional manifolds may not merely be descriptive epiphenomena of neural activity, but rather have quantifiable causal power in driving behaviour.  \nHowever, there still exists a strong and consistent debate about the methods, interpretation, and the necessity of the neural manifold view in neuroscience. On the methodological side, ther","cbCaivjYbCNPniln","https://ap.wps.com/l/cbCaivjYbCNPniln","pdf",6622840,4,1,14,"English","en",105,"# Abstract\n# Introduction\n## Neural manifolds and dimensionality reduction\n## Causal evidence from perturbation and decoding\n## Debates and methodological concerns\n## Functional perspective: information bottleneck and generalisation","[{\"question\":\"What role does an information bottleneck play in the results?\",\"answer\":\"The study shows that explicitly forcing a recurrent neural network to learn a low-dimensional representation via an information bottleneck is necessary for rotational and out-of-distribution generalisation in a time-series prediction task.\"},{\"question\":\"How do the learned representations behave during the memorisation-to-generalisation transition?\",\"answer\":\"Using causal-emergence, the representation dynamics follow a non-monotonic trajectory: an initial decrease, then a minimum, and later a rise to a maximum even while prediction loss decreases monotonically.\"},{\"question\":\"Does the work connect neural network findings to biological data?\",\"answer\":\"Yes. Analysis of CA1 hippocampal activity in mice learning an alternating maze shows analogous non-monotonic emergence dynamics that track behavioural performance.\"}]",1784207110,35,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"emergent-generalization-by-representation-learning-in-artificial-neural-networks","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/emergent-generalization-by-representation-learning-in-artificial-neural-networks/85910/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-26","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What role does an information bottleneck play in the results?","Question",{"text":75,"@type":76},"The study shows that explicitly forcing a recurrent neural network to learn a low-dimensional representation via an information bottleneck is necessary for rotational and out-of-distribution generalisation in a time-series prediction task.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How do the learned representations behave during the memorisation-to-generalisation transition?",{"text":80,"@type":76},"Using causal-emergence, the representation dynamics follow a non-monotonic trajectory: an initial decrease, then a minimum, and later a rise to a maximum even while prediction loss decreases monotonically.",{"name":82,"@type":73,"acceptedAnswer":83},"Does the work connect neural network findings to biological data?",{"text":84,"@type":76},"Yes. Analysis of CA1 hippocampal activity in mice learning an alternating maze shows analogous non-monotonic emergence dynamics that track behavioural performance.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]