[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-83378-en":3,"doc-seo-83378-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},83378,1099514068365,"Aurelia","https://ap-avatar.wpscdn.com/avatar/10000253d8d9f28188e?_k=1776742907772140068",8,"Research & Report","Beyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks","Backpropagation remains the dominant deep-learning training approach, yet gradient-based learning suffers from vanishing and exploding gradients. A gradient-free alternative is demonstrated: a simple Monte Carlo mutation–optimization selection routine on a single GPU that mutates parameters, accepts updates only when loss decreases, and retries otherwise. The method trains deep networks without common architectural aids like batch normalization or residual connections, and extends to pruning training, discrete weights, Gaussian and other unconventional transfer functions, and analysis of deep-network redundancy. Experiments cover networks over 20 layers, wide single-hidden-layer models with up to 16,384 neurons, and a basic Transformer on MNIST and Tiny Shakespeare.","arXiv :2607 .08406v 1 [ cs .LG] 9 Jul 2026  \nBeyond Backpropagation: Monte Carlo Method Can Train Deep Neural Networks  \nHong Zhao 1 ,2  \n1 Department of Physics, Xiamen University, Xiamen 361005, China  \n2 Lanzhou Center for Theoretical Physics, Lanzhou University, Lanzhou 730000, China  \nEmail: [zhaoh@xmu.edu.cn](zhaoh@xmu.edu.cn)  \nAbstract  \nBackpropagation (BP) dominates deep learning training, but its reliance on gradients brings inherent troubles—vanishing and exploding gradients. The pursuit of gradient-free methods has long been agoal in the field of artificial intelligence. This paper shows that indeed the simplest Monte Carlo algorithm implemented on a single GPU—randomly mutate a parameter, keep it if the loss decreases, otherwise retry—can practically train deep networks. This gradient-free method does not even need common techniques such as batch normalization or residual connections to directly train sufficiently deep networks. More remarkably, its flexibility extends to several nontrivial scenarios: it enables pure pruning training, supports discrete weights, accommodates unconventional transfer functions such as Gaussian, and reveals the substantial redundancy of deep networks. We have demonstrated its feasibility on deep networks with more than 20 layers, single-hiddenlayer wide networks with up to 16,384 hidden neurons, and even a simple Transformer architecture trained on both image classification (MNIST) and character-level language modeling (Tiny Shakespeare) . This simple gradient-free method may offer a complementary perspective for understanding the self-organization and learning mechanisms of neural networks, and also provides an alternative route for building physically inspired deep learning systems.  \nKeywords: Monte Carlo method, deep neural networks, gradient-free optimization, unconventional transfer functions  \n1 Introduction  \nAlthough backpropagation (BP) is currently the main method for training learning machines [1, 2], it is still necessary to explore alternative algorithms. Representative works in this field include random forests [3, 4, 5],  \nkernel methods [6, 7], tensor network architectures and their corresponding algorithms [8, 9, 10] . In recent years, increasing attention has been paid to finding new paradigms that can replace or even surpass BP. For example, spiking neural networks have explored brain-like learning [13, 14, 15], and the local-learning-based Forward-Forward algorithm has attempted to directly replace BP [16] . However, these methods have so far been unable to challenge the dominant position of BP in mainstream deep learning.  \nThe Monte Carlo method is a fundamental computational tool in physics, successfully applied to a series of key problems in statistical physics, quantum field theory, particle transport, and other fields [17, 18, 19, 20, 21] . By introducing stochastic sampling, it breaks through the limitations of traditional deterministic prediction models and has been widely used in various branches of neural networks [22, 23, 24] . Among them, Monte Carlo Dropout is widely used in various deep networks [25, 26], and diffusion models are built upon Monte Carlo sampling [27] . Moreover, the application of Monte Carlo tree search is the key to the success of top artificial intelligence systems such as AlphaGo [28] .  \nIn the early days of artificial neural network research, the Boltzmann machine had already introduced the concepts of random sampling, thermal equilibrium distribution, and simulated annealing from statistical physics into the basic architecture and learning rules of neural networks [29, 30]; however, because such methods require repeated operation of the network to estimate equilibrium statistics, the learning process is very slow, and they therefore long remained a non-mainstream training paradigm [29, 31] . More than twenty years ago, this method was used to design asymmetric Hopfield networks [32], which have advantages over symmetric networ","cbCaifYPjVNAWuoF","https://ap.wps.com/l/cbCaifYPjVNAWuoF","pdf",6172912,1,22,"English","en",105,"# Introduction\n## Motivation for gradient-free training\n## Monte Carlo background and related work","[{\"question\":\"为什么提出“Beyond Backpropagation”的训练方法？\",\"answer\":\"文中指出反向传播依赖梯度会带来梯度消失与梯度爆炸等问题，因此需要探索能够替代或补充BP的训练范式。\"},{\"question\":\"该文的核心训练算法是什么？\",\"answer\":\"采用最基本的Monte Carlo随机变异参数—仅当损失下降就接受，否则重试的选择机制，并在单GPU上实现。\"},{\"question\":\"这种梯度无关方法能做哪些扩展应用？\",\"answer\":\"文中说明其可用于纯剪枝训练、离散权重、使用高斯等非常规激活/传递函数，并用于揭示深层网络存在显著冗余。\"}]",1784187092,55,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"beyond-backpropagation-monte-carlo-method-can-train-deep-neural-networks","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/beyond-backpropagation-monte-carlo-method-can-train-deep-neural-networks/83378/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"为什么提出“Beyond Backpropagation”的训练方法？","Question",{"text":75,"@type":76},"文中指出反向传播依赖梯度会带来梯度消失与梯度爆炸等问题，因此需要探索能够替代或补充BP的训练范式。","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"该文的核心训练算法是什么？",{"text":80,"@type":76},"采用最基本的Monte Carlo随机变异参数—仅当损失下降就接受，否则重试的选择机制，并在单GPU上实现。",{"name":82,"@type":73,"acceptedAnswer":83},"这种梯度无关方法能做哪些扩展应用？",{"text":84,"@type":76},"文中说明其可用于纯剪枝训练、离散权重、使用高斯等非常规激活/传递函数，并用于揭示深层网络存在显著冗余。","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]