[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82632-en":3,"doc-seo-82632-105":29,"detail-sidebar-cat-0-en-105":95},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82632,16904993612988,"Olivia Brown","https://ap-avatar.wpscdn.com/davatar_a8503ba1806abce46bf441b54a3ca4cd",8,"Research & Report","Fourier-Diagonalized Natural Gradients and Sobolev Mirror Descent","We study natural-gradient updates whose metric operators are Fourier-diagonal and connect them to Sobolev mirror descent. In translation-invariant Fisher geometries and Sobolev mirror geometries, the inverse-map structure aligns in the spectral domain. The Fisher metric corresponds to a positive Fourier symbol, while Sobolev mirror geometry uses the Bessel-potential symbol tied to the Sobolev norm. When symbols match, updates coincide; otherwise, Sobolev mirror descent yields a canonical spectral preconditioner for Fisher inverse geometry, motivating Spectral Natural Gradient via FFT.","arXiv :2607 .01634v1 [math .NA] 2 Jul 2026  \nFourier-Diagonalized Natural Gradients and Sobolev  \nMirror Descent  \nJeongbin Joa,∗  \na Yonsei University, Seoul, Republic of Korea  \nAbstract  \nWe study natural-gradient updates whose metric operators are diagonalized by the Fourier transform and relate them to Sobolev mirror descent. Translation-invariant Fisher geometries and Sobolev mirror geometries share a common inverse-map structure in the spectral domain. The Fisher metric is represented by a positive Fourier symbol, while Sobolev mirror geometry corresponds to the specific Bessel-potential symbol associated with the Sobolev norm. When these symbols coincide, the natural-gradient and mirror-descent updates are identical; otherwise, Sobolev mirror descent provides a canonical spectral preconditioner for the Fisher inverse geometry. This gives a mathematical lens through which spectral filtering and truncation techniques in PDE and operator learning can be viewed as natural actions of inverse metric geometry. We introduce Spectral Natural Gradient, an FFT-based implementation of these geometric updates.  \nKeywords: Natural Gradient Descent, Mirror Descent, Sobolev Space, Fourier Transform, Optimization, Fractional Laplacian  \n1. Introduction  \nOptimizing parameterized probability distributions and continuous spatial fields is a central problem across machine learning, physics-informed neural networks, and statistical mechanics. Ordinary first-order gradient descent assumes the parameter space is flat and Euclidean, often leading to slow convergence or unstable behavior in ill-conditioned landscapes, especially when  \n∗ Corresponding author  \nEmail address: [jeongbin033@yonsei.ac.kr](jeongbin033@yonsei.ac.kr) (Jeongbin Jo)  \noscillatory spatial modes are present [1, 2] .  \nTo account for the intrinsic curvature of these spaces, Amari introduced Natural Gradient Descent (NGD) [3 , 4 , 5] . NGD operates within the framework of information geometry, utilizing the Fisher Information Matrix (FIM), denoted as F (θ), as the Riemannian metric tensor. The exact NGD update rule requires evaluating θt+1 = θt − α F−1(θt) ∇L(θt) . By following the steepest descent direction in distribution space rather than Euclidean parameter space, NGD provides a geometrically meaningful scaling of the gradient. However, for a model with N parameters, explicitly forming a dense FIM requires O (N2 ) storage, and direct inversion or factorization can require O (N3 ) operations. This cubic scaling creates a formidable computational bottleneck for extensive spatial models.  \nIn this work, we establish a spectral dictionary between translationinvariant natural-gradient geometry and Sobolev mirror descent. Both frameworks share a common Fourier-diagonal inverse-map structure. While the translation-invariant Fisher metric takes an arbitrary positive Fourier symbol λF (ω), Sobolev mirror descent is the special case in which the spectral symbol is (1 + |ω|2 )s. When these symbols coincide, the two updates are identical; otherwise, Sobolev mirror descent provides a canonical spectral preconditioner for the Fisher inverse geometry. This dictionary gives an information-geometric explanation for spectral filtering mechanisms used in spatial optimization.  \nThis perspective is complementary to scalable natural-gradient and preconditioned optimization methods that impose tractable matrix structure, including truncated or approximate natural-gradient solvers [6], Kroneckerfactored curvature approximations [7], and tensor-structured preconditioners [8] . Rather than approximating a generic dense Fisher matrix, SNG isolates the case in which the relevant geometry is already diagonal in a known harmonic basis. It is also related to Sobolev-gradient methods for differential equations [9], but emphasizes the natural-gradient interpretation and the explicit FFT implementation of the inverse geometry.  \n2. Fourier-Diagonalized Geometry and Sobolev Mirror Descent  \nIn this sec","cbCaigSW4KHEZbGq","https://ap.wps.com/l/cbCaigSW4KHEZbGq","pdf",436345,1,10,"English","en",105,"# Introduction\n# Fourier-Diagonalized Geometry and Sobolev Mirror Descent\n## Mathematical Definition of Mirror Descent (Fenchel Duality)","[{\"question\":\"What is the core relationship between Fourier-diagonal natural gradients and Sobolev mirror descent?\",\"answer\":\"Both share a common inverse-map structure in the spectral domain when formulated with translation-invariant operators. Sobolev mirror descent can match natural-gradient updates when their spectral symbols coincide, and otherwise provides a canonical spectral preconditioner for the Fisher inverse geometry.\"},{\"question\":\"How is the Fisher metric represented in this framework?\",\"answer\":\"The Fisher metric is represented by a positive Fourier symbol, allowing the natural-gradient update direction to be expressed modewise in the Fourier domain.\"},{\"question\":\"What does the Sobolev mirror geometry correspond to spectrally?\",\"answer\":\"Sobolev mirror geometry corresponds to a specific Bessel-potential symbol associated with the Sobolev norm, i.e., the spectral weight (1+|ω|^2)^s.\"},{\"question\":\"Why does the paper introduce Spectral Natural Gradient?\",\"answer\":\"It provides an FFT-based implementation of the geometric updates, leveraging the Fourier-diagonal structure to make the inverse-metric action computationally tractable.\"}]",1784181920,25,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":90,"head_meta":92,"extra_data":94,"updated_unix":27},"fourier-diagonalized-natural-gradients-and-sobolev-mirror-descent","",{"@graph":35,"@context":89},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/fourier-diagonalized-natural-gradients-and-sobolev-mirror-descent/82632/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81,85],{"name":72,"@type":73,"acceptedAnswer":74},"What is the core relationship between Fourier-diagonal natural gradients and Sobolev mirror descent?","Question",{"text":75,"@type":76},"Both share a common inverse-map structure in the spectral domain when formulated with translation-invariant operators. Sobolev mirror descent can match natural-gradient updates when their spectral symbols coincide, and otherwise provides a canonical spectral preconditioner for the Fisher inverse geometry.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How is the Fisher metric represented in this framework?",{"text":80,"@type":76},"The Fisher metric is represented by a positive Fourier symbol, allowing the natural-gradient update direction to be expressed modewise in the Fourier domain.",{"name":82,"@type":73,"acceptedAnswer":83},"What does the Sobolev mirror geometry correspond to spectrally?",{"text":84,"@type":76},"Sobolev mirror geometry corresponds to a specific Bessel-potential symbol associated with the Sobolev norm, i.e., the spectral weight (1+|ω|^2)^s.",{"name":86,"@type":73,"acceptedAnswer":87},"Why does the paper introduce Spectral Natural Gradient?",{"text":88,"@type":76},"It provides an FFT-based implementation of the geometric updates, leveraging the Fourier-diagonal structure to make the inverse-metric action computationally tractable.","https://schema.org",{"og:url":51,"og:type":91,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":93,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":96},[97,101,105,109,114,119,124,127,132,135,138],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},"Exam",70,"exam",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},5,"Comic",60,"comic",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":117,"slug":118},6,"Technology",50,"technology",{"id":120,"doc_module":4,"doc_module_name":45,"category_name":121,"show_sort_weight":122,"slug":123},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":125,"slug":126},30,"research-report",{"id":128,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":130,"slug":131},9,"Religion & Spirituality",20,"religion-spirituality",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":130,"slug":134},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":21,"slug":137},"Lifestyle","lifestyle",{"id":139,"doc_module":4,"doc_module_name":45,"category_name":140,"show_sort_weight":110,"slug":141},19,"General","general"]