[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125907-en":3,"doc-seo-125907-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},125907,2336474466712,"Maeve","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Information decomposition in complex systems via machine learning","Information decomposition in complex systems via machine learning proposes a general methodology to extract the variation in measured system states that is most predictive of specified macroscale behavior. The approach uses machine learning to jointly optimize a lossy compression of multiple measurements, guided by the distributed information bottleneck objective. Mutual-information structure is leveraged to characterize how information distributes across observables. Analysis highlights interpretable decomposition in a Boolean circuit and an amorphous material under plastic deformation.","Information decomposition in complex systems via machine learning  \nKieran A. Murphya and Dani S. Bassetta,b,c,d,e,f  \na Dept. of Bioengineering, School of Engineering & Applied Science, U. of Pennsylvania, Philadelphia, PA 19104, USA; b Dept. of Electrical & Systems Engineering, School of Engineering & Applied Science, U. of Pennsylvania, Philadelphia, PA 19104, USA; c Dept. of Neurology, Perelman School of Medicine, U. of Pennsylvania, Philadelphia, PA 19104, USA; d Dept. of Psychiatry, Perelman School of Medicine, U. of Pennsylvania, Philadelphia, PA 19104, USA; e Dept. of Physics & Astronomy, College of Arts & Sciences, University of Pennsylvania, Philadelphia, PA 19104, USA; fThe Santa Fe Institute, Santa Fe, NM 87501, USA  \narXiv :2307 .04755v2 [ cs .LG] 19 Mar 2024  \nThis manuscript was compiled on March 20, 2024  \nOne of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system’s components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of linking variation across scales of a system due to its independence of functional relationship between observables. However, characterizing the manner in which information is distributed across a set of observables is computationally challenging and generally infeasible beyond a handful of measurements. Here we propose a practical and general methodology that uses machine learning to decompose the information contained ina set of measurements by jointly optimizing a lossy compression of each measurement. Guided by the distributed information bottleneck as a learning objective, the information decomposition identifies the variation in the measurements of the system state most relevant to specified macroscale behavior. We focus our analysis on two paradigmatic complex systems: a Boolean circuit and an amorphous material undergoing plastic deformation. In both examples, the large amount of entropy of the system state is decomposed, bit by bit, in terms of what is most related to macroscale behavior. The identification of meaningful variation in data, with the full generality brought by information theory, is made practical for studying the connection between micro-and macroscale structure in complex systems.  \ninformation theory | machine learning for science | complex systems | amorphous plasticity  \nA complex system is a system of interacting components where some sense of order present at the scale of the system is not apparent, or even conceivable, from the observations of single components (1, 2) . A broad categorization, it includes many systems of relevance to our daily lives, from the economy to the internet and from the human brain to artificial neural networks (3, 4) . Before attempting a reductionist description of a complex system, one must first identify variation in the system that is most relevant to emergent order at larger scales. The notion of relevance can be formalized with information theory, wherein mutual information serves as a general measure of statistical dependence to connect variation across different scales of system behavior (5 , 6) . Information theory and complexity science have a rich history; information theory commonly forms the foundation of definitions of what it means to be complex (7–11) .  \nMachine learning is well-suited for the analysis of complex systems, grounded in its natural capacity to identify patterns in high dimensional data (12) . However, distilling insight from a successfully trained model is often infeasible due toa characteristic lack of interpretability of machine learning models (13 , 14) . Restricting to simpler classes of models, for example linear combinations of observables, recovers a degree of interpretability at the expense of functional expres-  \nsivity (15) . For the study of complex systems, such a trade-off is unacceptable if the complexity of the system is no longer faithfully represented. In this work, we do ","cbCairr2Mgm3hcWP","https://ap.wps.com/l/cbCairr2Mgm3hcWP","pdf",6079120,3,1,16,"English","en",105,"# Introduction\n## Information variation across scales\n## Machine learning for complex systems and interpretability\n# Method\n## Distributed information bottleneck and lossy compression\n# Applications\n## Boolean circuit example\n## Amorphous plastic deformation example\n# Significance","[{\"question\":\"What problem does the methodology target in complex systems?\",\"answer\":\"It targets identifying which variation in microscale measurements is most relevant to predicting emergent macroscale behavior.\"},{\"question\":\"How is the objective defined for decomposing information?\",\"answer\":\"The method uses the distributed information bottleneck as a learning objective, optimizing a lossy compression that preserves information about a relevance quantity.\"},{\"question\":\"Which examples are used to demonstrate the approach?\",\"answer\":\"It demonstrates on a fully specified Boolean circuit and on an amorphous material undergoing plastic deformation.\"}]","Information decomposition in complex systems via machine learning | PDF",1785901969,40,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"information-decomposition-in-complex-systems-via-machine-learning","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,49,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":48},"https://docshare.wps.com/document/","Document",2,{"item":50,"name":12,"@type":44,"position":20},"https://docshare.wps.com/document/research-report/",{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/information-decomposition-in-complex-systems-via-machine-learning/125907/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-24","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does the methodology target in complex systems?","Question",{"text":76,"@type":77},"It targets identifying which variation in microscale measurements is most relevant to predicting emergent macroscale behavior.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How is the objective defined for decomposing information?",{"text":81,"@type":77},"The method uses the distributed information bottleneck as a learning objective, optimizing a lossy compression that preserves information about a relevance quantity.",{"name":83,"@type":74,"acceptedAnswer":84},"Which examples are used to demonstrate the approach?",{"text":85,"@type":77},"It demonstrates on a fully specified Boolean circuit and on an amorphous material undergoing plastic deformation.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,120,123,128,131,135],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":48,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":30,"slug":119},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":47,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":47,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":47,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":47,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]