[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-122960-en":3,"doc-seo-122960-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},122960,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","Trajectory Ensembles and Machine Learning - From reinforcement learning for rare event sampling to training of neural network ensembles","This PhD thesis connects statistical mechanics with machine learning, aiming to bridge trajectory ensemble techniques and modern deep learning methods. It develops two main avenues: translating rare-event sampling into reinforcement learning to exploit deep RL for near-optimal sampling dynamics, and introducing a trajectory-sampling algorithm that jointly trains an ensemble of neural networks. The work derives analysis for finite and infinite time problems. It also proposes a coupled ensemble training method using Markov chain Monte Carlo with gradient-free updates, outperforming uncoupled neuroevolution baselines on linear analysis and empirical MNIST experiments.","Trajectory Ensembles and Machine Learning:  \nFrom reinforcement learning for rare event sampling to training of neural network ensembles  \nby  \nJamie F. Mair  \nA thesis submitted for fulfilment of the requirements for the degree of Doctor of Philosophy  \nin the  \nSchool of Physics & Astronomy University of Nottingham  \nSupervisors:  \nProf. Juan P. Garrahan  \nDr. Adam Moss  \nJune 2023  \nAbstract  \nThis thesis builds on the existing body of work connecting the fields of statistical mechanics and machine learning. Many advances in machine learning have found their roots in statistical mechanics, e.g. simulated annealing for heuristic optimisation. Primarily, we aim to build bridges between trajectory ensemble techniques and current advances in machine learning, and in particular, deep learning. We explore these connections in two avenues: the first connects the study of rare events with that of reinforcement learning (RL); the second introduces a trajectory sampling algorithm for jointly training an ensemble of neural networks. We derive a framework for translating a rare event sampling problem into the language of RL, offering a way to leverage modern deep RL algorithms to obtain near-optimal sampling dynamics. Our work showcases a plethora of these RL algorithms and provides analysis on examples, including both finite and infinite time problems. Furthermore, we present a novel neural network ensemble training method, facilitated via Markov chain Monte Carlo algorithms, to produce coupled ensembles using gradient-free updates. We show that our coupled ensembles perform better, and are easier to train, than their uncoupled counterparts trained via neuroevolution, providing both analytic results on a linear problem and empirical evidence on the MNIST problem.  \nContents  \nAcronyms i  \nIntroduction iii  \n1 Introduction to Machine Learning 1  \n1.1 Supervised Learning ............................. 2  \n1.2 Linear Models ................................. 3  \n1.3 Neural Networks ............................... 4  \n1.3.1 Activation Functions ......................... 5  \n1.3.2 Back-Propagation ........................... 5  \n1.4 Training as an Optimisation Problem .................... 8  \n1.4.1 Gradient Descent ........................... 8  \n1.4.2 Stochastic Gradient Descent ..................... 9  \n2 Reinforcement Learning 11  \n2.1 Mathematical Definition ........................... 12  \n2.2 Grid World .................................. 16  \n2.3 Dynamic Programming ............................ 19  \n2.3.1 Bellman Equations and Optimality ................. 19  \n2.3.2 Policy Evaluation ........................... 20  \n2.3.3 Policy Iteration ............................ 22  \n2.4 Model-Free Methods ............................. 24  \n2.4.1 Policy Evaluation through Interaction ................ 25  \n2.4.2 Monte Carlo Policy Iteration ..................... 27  \n2.4.3 On/Off Policy Methods ........................ 28  \n2.4.4 Bootstrapping: Updating Estimates with Estimates ........ 32  \n2.4.5 n-step Temporal Difference ..................... 32  \n2.5 Approximate Solution Methods ....................... 34  \n2.5.1 Large State Spaces and Generalisation ............... 35  \n2.5.2 The Deadly Triad ........................... 35  \n2.6 Policy Gradient Methods ........................... 36  \n2.6.1 Parameterising a Policy ....................... 36  \n2.6.2 Policy Gradient: REINFORCE ................... 38  \n2.6.3 Variance of Vanilla Policy Gradient ................. 40  \n2.6.4 REINFORCE with Value Baseline .................. 45  \n2.6.5 Actor-Critic .............................. 46  \n3 Large Deviation Theory 49  \n4 CONTENTS  \n3.1 Gärtner-Ellis Theorem ............................ 50  \n3.2 Moments of the SCGF ............................ 51  \n3.2.1 Example: Bernoulli Random Variables ............... 51  \n3.3 Dynamical Large Deviations ......................... 53  \n3.3.1 Doob Dynamics ............................ 56  \n4 Monte Carlo Sampling 57  \n4.1 Mo","cbCaiuEnmNZJTpBt","https://ap.wps.com/l/cbCaiuEnmNZJTpBt","pdf",9099069,1,163,"English","en",105,"# Introduction\n## Supervised Learning\n## Linear Models\n## Neural Networks\n## Training as an Optimisation Problem\n# Reinforcement Learning\n## Mathematical Definition\n## Grid World\n## Dynamic Programming\n## Model-Free Methods\n## Approximate Solution Methods\n## Policy Gradient Methods\n# Large Deviation Theory\n# Monte Carlo Sampling\n## Monte Carlo Integration\n## Importance Sampling\n## Markov-Chain Monte Carlo Methods\n# Supporting Computational Infrastructure\n## Course: High Performance Computing in Julia\n## Package: Experimenter . jl\n## Package: TransitionPathSampling . jl\n## Package: SimpleNNs . jl\n# A Reinforcement Learning Approach to Rare Trajectory Sampling\n# Training Neural Network Ensembles via Trajectory Sampling\n# Minibatch Training of Neural Network Ensembles via Trajectory Sampling\n# Conclusions & Outlook","[{\"question\":\"How does the thesis link rare event sampling with reinforcement learning?\",\"answer\":\"It derives a framework translating rare-event sampling into the language of reinforcement learning, enabling use of modern deep RL algorithms to obtain near-optimal sampling dynamics.\"},{\"question\":\"What is the second main direction of the thesis?\",\"answer\":\"It presents a trajectory sampling algorithm for jointly training an ensemble of neural networks, including methods for minibatch training.\"},{\"question\":\"How are coupled neural network ensembles trained, and what evidence supports their advantage?\",\"answer\":\"Coupled ensembles are trained using Markov chain Monte Carlo facilitated, with gradient-free updates; results show improved performance and easier training versus uncoupled ensembles, supported analytically on a linear problem and empirically on MNIST.\"}]","Trajectory Ensembles and Machine Learning - From reinforcement learning for rare event sampling to training of neural network ensembles | PDF",1785813906,411,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"trajectory-ensembles-and-machine-learning-from-reinforcement-learning-for-rare-event-sampling-to-training-of-neural-network-ensembles","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/trajectory-ensembles-and-machine-learning-from-reinforcement-learning-for-rare-event-sampling-to-training-of-neural-network-ensembles/122960/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05","2026-08-04",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"How does the thesis link rare event sampling with reinforcement learning?","Question",{"text":76,"@type":77},"It derives a framework translating rare-event sampling into the language of reinforcement learning, enabling use of modern deep RL algorithms to obtain near-optimal sampling dynamics.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What is the second main direction of the thesis?",{"text":81,"@type":77},"It presents a trajectory sampling algorithm for jointly training an ensemble of neural networks, including methods for minibatch training.",{"name":83,"@type":74,"acceptedAnswer":84},"How are coupled neural network ensembles trained, and what evidence supports their advantage?",{"text":85,"@type":77},"Coupled ensembles are trained using Markov chain Monte Carlo facilitated, with gradient-free updates; results show improved performance and easier training versus uncoupled ensembles, supported analytically on a linear problem and empirically on MNIST.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]