[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85880-en":3,"doc-seo-85880-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85880,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Interpreting Learning Dynamics of Autoencoders: Transient Scaling and Emerging Concepts of the Ising Model","Study how unsupervised autoencoders trained on microscopic spin configurations from the Ising model extract macroscopic, theory-relevant variables without using domain knowledge. Learning is analyzed across multiple spatial coarse-graining scales, revealing two hyperparameter-controlled dynamical regimes: a magnetization-dominated regime and an energy-dominated regime with competing representation quality. Recursive-dynamic trajectories show prediction-error-induced flow fields that yield a shared topology across representation spaces. Training is framed as far-from-equilibrium dynamics driven by fluctuations from data and optimizer.","arXiv :2607 . 10285v 1 [ cs .LG] 11 Jul 2026  \nInterpreting learning dynamics of autoencoders: Transient scaling and emerging  \nconcepts of the Ising model  \nMax Weinmann 1, 2 and Miriam Klopotek2, 1, ∗  \n1 WIN-Kolleg of the Young Academy | Heidelberg Academy of  \nSciences and Humanities, Karlstraße 4, 69117 Heidelberg, Germany  \n2 Stuttgart Center for Simulation Science, Cluster of Excellence EXC 2075,  \nUniversity of Stuttgart, Universit¨atsstraße 32, 70569 Stuttgart, Germany  \n(Dated: July 14, 2026)  \nWe study how unsupervised autoencoders trained on microscopic spin configurations from the Ising model learn macroscopic, theory-relevant variables underlying the data-generating process. Without embedding domain knowledge, we mimic a typical discovery setting: We quantify learning across multiple spatial (coarse-graining) scales and reveal two distinct dynamical regimes controlled by main hyperparameters (model depth, width, and learning rate)—a magnetization-dominated regime and an energy-dominated regime characterized by trade-offs in their representation quality. The first regime is a transitory state exhibiting dynamical scaling and fluctuations that follow an ordering-to-scale; the second gradually shifts resolution towards smaller scales relevant for the energy representation. Deep models trained at moderate and fast rates become arrested before reaching these regimes. With a novel analysis of recursive-dynamic trajectories, we demonstrate that prediction errors induce flow fields that produce a common trajectory topology across all representation spaces. A dynamical viewpoint of learning is established in which intrinsic properties expose the effects of forced changes in representation during training. We utilize the intuition that learning operates as a process driven far from equilibrium by fluctuations from the training data and optimizer to provide an interpretive basis grounded in both the physical world and the machine models that represent it.  \nKeywords: Machine learning, Artificial neural networks, Self-organized systems, Non-equilibrium dynamics,  \nCritical dynamics, Dynamical phase transitions, Ising model, Artificial intelligence  \nI. INTRODUCTION  \nMachines automate many physically demanding tasks, such as farm or factory work; formal tasks, such as calculations and thought experiments using computer simulations; and, more recently, increasingly abstract or creative tasks common to the worlds of work and education. In particular, machine learning enables the automation of tasks without a formal scope – such as pattern recognition [1] – and, with the advent of large language models (LLMs), also creative tasks such as writing and planning, albeit often to an inadequate standard [2, 3] .  \nWhile we have a good understanding of linear methods like linear regression and related applications (e.g. , PCA [4], LDA [5], SVD [6]), we lack a comparably deep understanding of nonlinear (stochastic) methods. This includes the methods used to train the neural networks (NNs) that underpin more recent advancements. Encouragingly, many nonlinear systems in physics arise from many interacting constituents forming a collective, whose qualitative behavior we can describe well at the macroscopic scale. This motivates our attempt to use well-established methods and theoretical principles from physics to explain the nonlinear training dynamics of commonly used models.  \nWe focus on studying learning dynamics of a deterministic nonlinear auto-encoder [7], which is useful for find-  \n∗ [miriam.klopotek@simtech.uni-stuttgart.de](miriam.klopotek@simtech.uni-stuttgart.de)  \ning low-dimensional representations of high-dimensional data. Autoencoders can be seen as nonlinear analogs of principal component analysis (PCA) [8]; heuristically, they allow us to describe ‘principal components’ of the data or generic extentions of these using nonlinear relationships. The most informative (non-linear) principal components are selected when optim","cbCaijPWxRMHHNGi","https://ap.wps.com/l/cbCaijPWxRMHHNGi","pdf",35265569,4,1,60,"English","en",105,"# Introduction\n## Learning dynamics and motivation\n## Autoencoders and coarse-graining\n## Hyperparameter effects on concept formation","[{\"question\":\"What is the main goal of this work on autoencoders and the Ising model?\",\"answer\":\"The work studies how unsupervised autoencoders learn macroscopic, theory-relevant variables from microscopic Ising spin configurations, aiming to explain how concept representations form during training.\"},{\"question\":\"How many dynamical regimes are identified, and what distinguishes them?\",\"answer\":\"Two regimes are identified: a magnetization-dominated regime and an energy-dominated regime. They are controlled by key hyperparameters and differ in how well representations capture different qualities.\"},{\"question\":\"What does the recursive-dynamic trajectory analysis show about learning?\",\"answer\":\"It shows that prediction errors induce flow fields producing a common trajectory topology across all representation spaces, establishing a dynamical viewpoint where intrinsic properties reflect forced changes during training.\"}]",1784206912,151,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"interpreting-learning-dynamics-of-autoencoders-transient-scaling-and-emerging-concepts-of-the-ising-model","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":20},"https://docshare.wps.com/document/interpreting-learning-dynamics-of-autoencoders-transient-scaling-and-emerging-concepts-of-the-ising-model/85880/",{"url":52,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-25","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main goal of this work on autoencoders and the Ising model?","Question",{"text":75,"@type":76},"The work studies how unsupervised autoencoders learn macroscopic, theory-relevant variables from microscopic Ising spin configurations, aiming to explain how concept representations form during training.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How many dynamical regimes are identified, and what distinguishes them?",{"text":80,"@type":76},"Two regimes are identified: a magnetization-dominated regime and an energy-dominated regime. They are controlled by key hyperparameters and differ in how well representations capture different qualities.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the recursive-dynamic trajectory analysis show about learning?",{"text":84,"@type":76},"It shows that prediction errors induce flow fields producing a common trajectory topology across all representation spaces, establishing a dynamical viewpoint where intrinsic properties reflect forced changes during training.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,109,114,119,122,127,130,134],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":22,"slug":108},5,"Comic","comic",{"id":110,"doc_module":4,"doc_module_name":46,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]