[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-123128-en":3,"doc-seo-123128-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},123128,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",6,"Technology","LossLens - Diagnostics for Machine Learning through Loss Landscape Visual Analytics","Modern machine learning optimizes neural network parameters by minimizing a loss function, yet understanding both local behavior near a solution and global structure across many local minima remains difficult. LossLens presents a visual analytics framework that explores loss landscapes at multiple scales by integrating global and local metrics into a unified representation. The approach improves model diagnostics and is validated using case studies on residual connections in ResNet-20 and parameter effects in physics-informed neural networks.","Lawrence Berkeley National Laboratory  \nLBL Publications  \nTitle  \nLossLens: Diagnostics for Machine Learning Through Loss Landscape Visual Analytics  \nPermalink  \n[https://escholarship.org/uc/item/9f42j92j](https://escholarship.org/uc/item/9f42j92j)  \nJournal  \nIEEE Computer Graphics and Applications, PP(99)  \nISSN  \n0272-1716  \nAuthors  \nXie, Tiankai  \nChen, Jiaqing Yang, Yaoqinget al.  \nPublication Date  \n2024  \nDOI  \n10.1109/mcg.2024.3509374  \nCopyright Information  \nThis work is made available under the terms of a Creative Commons AttributionNonCommercial-ShareAlike License, available at [https://creativecommons.org/licenses/by](https://creativecommons.org/licenses/by)nc-sa/4 . 0/  \nPeer reviewed  \n[eScholarship.org](eScholarship.org) Powered by the California Digital Library  \nUniversity of California  \narXiv :2412 . 13321v1 [ cs .LG] 17 Dec 2024  \nLossLens: Diagnostics for Machine Learning through Loss Landscape Visual Analytics *  \nTiankai Xie*, Jiaqing Chen*, Yaoqing Yang*, Caleb Geniesse*, Ge Shi, Ajinkya Chaudhari, John Kevin Cava, Michael W. Mahoney, Talita Perciano, Gunther H. Weber, and Ross Maciejewski  \nAbstract  \nModern machine learning often relies on optimizing a neural network’s parameters using a loss function to learn complex features. Beyond training, examining the loss function with respect to a network’s parameters (i.e., as a loss landscape) can reveal insights into the architecture and learning process. While the local structure of the loss landscape surrounding an individual solution can be characterized using a variety of approaches, the global structure of a loss landscape, which includes potentially many local minima corresponding to different solutions, remains far more difficult to conceptualize and visualize. To address this difficulty, we introduce LossLens 1 , a visual analytics framework that explores loss landscapes at multiple scales. LossLens integrates metrics from global and local scales into a comprehensive visual representation, enhancing model diagnostics. We demonstrate LossLens through two case studies: visualizing how residual connections influence a ResNet-20, and visualizing how physical parameters influence a physics-informed neural network (PINN) solving a simple convection problem.  \n1 Introduction  \nThe success of neural network models in natural language processing [1] and computer vision [2] is often attributed to progressively intricate model architectures and increasing volumes of data. A key step involves training a neural network to learn complex features from data. The objective used in this learning process is known as a loss function, written here as Ltrain(θ), which quantifies the mismatch between a network’s output and the ground-truth (or target) value. The loss function measures how good a model using the set of weights θ is at predicting the target, and model weights are adjusted during training, such that the loss function is minimized.  \nCharacterizing the loss function with respect to the weights of a neural network, i.e., as a loss landscape, has become an increasingly popular tool for interpreting and understanding the properties of neural network models [3] . Many papers have utilized loss landscapes to gain insights into model architecture, training dynamics, and robustness [16] . However, the construction of loss landscapes and the metrics used to analyze them vary significantly between different approaches and continue to evolve and improve. For example, linear interpolation is used to visualize the loss landscape by creating a randomized subspace [16], sharpness is used to measure the quality of the loss landscape [7], and Hessian spectral analysis is used to study the statistical properties and robustness of trained models [4] . While loss landscape research has proposed many methods and metrics for measuring model properties, and, e.g., loss landscape metrics such as those based on the Hessian can reflect things like the local flatness of the loss ","cbCaip33wyjvSV3O","https://ap.wps.com/l/cbCaip33wyjvSV3O","pdf",4605353,1,19,"English","en",105,"# Introduction\n# Loss landscape characterization and challenges","[{\"question\":\"What problem does LossLens address in machine learning research?\",\"answer\":\"LossLens addresses the difficulty of conceptualizing and visualizing the global structure of loss landscapes, beyond local analysis around a single solution.\"},{\"question\":\"How does LossLens support diagnostics of trained models?\",\"answer\":\"LossLens integrates metrics from global and local scales into a comprehensive visualization, enabling richer diagnostics than single-property or single-scale analyses.\"},{\"question\":\"What case studies are used to demonstrate LossLens?\",\"answer\":\"LossLens is demonstrated by visualizing how residual connections affect ResNet-20 and how physical parameters influence a physics-informed neural network solving a convection problem.\"}]","LossLens - Diagnostics for Machine Learning through Loss Landscape Visual Analytics | PDF",1785814766,48,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"losslens-diagnostics-for-machine-learning-through-loss-landscape-visual-analytics","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/technology/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/losslens-diagnostics-for-machine-learning-through-loss-landscape-visual-analytics/123128/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-04",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does LossLens address in machine learning research?","Question",{"text":75,"@type":76},"LossLens addresses the difficulty of conceptualizing and visualizing the global structure of loss landscapes, beyond local analysis around a single solution.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does LossLens support diagnostics of trained models?",{"text":80,"@type":76},"LossLens integrates metrics from global and local scales into a comprehensive visualization, enabling richer diagnostics than single-property or single-scale analyses.",{"name":82,"@type":73,"acceptedAnswer":83},"What case studies are used to demonstrate LossLens?",{"text":84,"@type":76},"LossLens is demonstrated by visualizing how residual connections affect ResNet-20 and how physical parameters influence a physics-informed neural network solving a convection problem.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,113,118,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":111,"slug":112},50,"technology",{"id":114,"doc_module":4,"doc_module_name":46,"category_name":115,"show_sort_weight":116,"slug":117},7,"Healthcare",40,"healthcare",{"id":119,"doc_module":4,"doc_module_name":46,"category_name":120,"show_sort_weight":121,"slug":122},8,"Research & Report",30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},"General","general"]