[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-117530-en":3,"doc-seo-117530-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},117530,1099514067438,"River Wang","https://ap-avatar.wpscdn.com/avatar/100002539ee87300030?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780474512215547542",8,"Research & Report","Invariance Testing For Machine Learning Models - Thesis Abstract and Contents","Machine learning models are expected to remain consistent when inputs are slightly modified while ground truth labels do not change, a property termed invariance. Existing invariance evaluations often depend on a limited set of accuracy metrics, restricting insight into behavior under transformations. The thesis proposes a systematic invariance testing framework that goes beyond aggregated scores, delivering automated and reproducible, visualization-rich analysis for more informative and explainable assessment of model performance.","Invariance Testing For Machine Learning Models  \nZukang Liao St Hugh’s College  \nUniversity of Oxford  \nA thesis submitted for the degree of Doctor of Philosophy  \nTrinity 2024  \nAbstract  \nMachine learning models are expected to produce consistent results when the input data objects are slightly modified whilst the ground truth remains unchanged. Such characteristics are also known as invariance qualities. However, previous studies assessing ML model invariance have primarily relied on one or a few accuracy metrics, which provide limited information on how models behave under different transformations. This thesis addresses this gap by proposing a novel, systematic invariance testing framework that moves beyond aggregated accuracy scores and offers more detailed visual information to enable more complex and explainable analysis of model performance.  \nOur framework introduces a generic and extendable methodology for evaluating invariance across various attributes, including simple transformations (e.g., object rotation) as well as complex environmental variations (e.g., background changes) . It is designed to be fully automated, enabling extensive and reproducible testing while ensuring consistent and explainable assessments of models’ invariance qualities. A key challenge in invariance testing is ordering and structuring test data, especially when testing an entire input space is infeasible. To address this, we develop a novel approach to sampling large information spaces, leveraging contextsensitive training data to construct context-free models. This method ensures that our framework can effectively assess invariance properties without requiring exhaustive test cases.  \nBy providing visualized patterns of model behavior, our approach enhances diagnostic capabilities, making it a valuable tool for both researchers and practitioners. Our findings demonstrate that the evaluation of ML models’ invariance qualities based on the visualized patterns can be: more informative, automatic, reliable and consistent. This thesis contributes to the broader effort of improving ML model evaluation, ensuring that claims of invariance are substantiated with comprehensive and reproducible evidence.  \nContents  \n1 Introduction 1  \n1.1 Research Gap and Problem Statement ................. 1  \n1.2 Definable Variation ............................ 3  \n1.3 Motivations ................................ 4  \n1.4 Objectives ................................. 6  \n1.5 Outline of the Thesis ........................... 8  \n1.5.1 Published Papers and Technical Reports ............ 9  \n2 Related Works 11  \n2.1 Overview of Related Works ....................... 11  \n2.2 Traditional definition ........................... 13  \n2.3 Variables .................................. 17  \n2.3.1 Basic data attributes ....................... 17  \n2.3.2 Variables of devices / data capturing .............. 22  \n2.3.3 Data Quality: Imprecision (errors) and Noise .......... 25  \n2.3.3.1 Imprecision and Errors ................. 25  \n2.3.3.2 Noise .......................... 26  \n2.3.4 Styles attributes .......................... 31  \n2.3.5 Variables associated with applications .............. 34  \n2.4 Input generation and mutation control ................. 34  \n2.4.1 Generative adversarial networks ................. 35  \n2.4.2 Variational Autoencoder ..................... 38  \n2.4.3 Coverage-based fuzzing testing .................. 39  \n2.4.4 Adversarial attacks ........................ 41  \n2.4.5 Ontology and Scene Understanding ............... 43  \n2.5 Invariance Testing Workflows ...................... 44  \n2.5.1 Formula-based invariance testing ................ 45  \n2.5.2 Visualization-based professional analysis ............ 46  \n2.5.2.1 Explainable AI (XAI) ................. 47  \n2.5.2.2 Symbolic testing .................... 47  \n2.5.2.3 Heatmap-based Analysis ................ 48  \n2.5.3 Theory-based professional analysis ............... 51  \n2.6 Conclusion ..................","cbCaic4ddxCU5RJU","https://ap.wps.com/l/cbCaic4ddxCU5RJU","pdf",62205906,1,193,"English","en",105,"# 1 Introduction\n## Research Gap and Problem Statement\n## Definable Variation\n## Motivations\n## Objectives\n## Outline of the Thesis\n## Published Papers and Technical Reports\n# 2 Related Works\n## Overview of Related Works\n## Traditional definition\n## Variables\n## Input generation and mutation control\n## Invariance Testing Workflows\n## Conclusion\n# 3 Generic Invariance Testing Framework for Basic Data Attributes\n## Introduction\n## Related work\n## Definitions and Motivation\n## Methodology: Testing Framework\n## Experimental Results and Analysis\n## Case Study: Invariance Testing using Sparse Linear Layers\n## Conclusions\n# 4 Invariance Testing for Advanced Data Attributes: Background Invariance Testing\n## Introduction\n## Related Work\n## Definition, Overview, and Motivation\n## Methodology","[{\"question\":\"What does invariance mean in machine learning model testing?\",\"answer\":\"Invariance means the model produces consistent results when input objects are slightly modified while the ground truth remains unchanged.\"},{\"question\":\"Why are existing invariance studies considered limited?\",\"answer\":\"They mainly rely on one or a few accuracy metrics, which restrict understanding of how models behave under different transformations.\"},{\"question\":\"How does the proposed framework evaluate invariance beyond accuracy scores?\",\"answer\":\"It introduces a generic, extendable, fully automated testing methodology that provides detailed visual information and structured analysis across different attributes and variations.\"}]","Invariance Testing For Machine Learning Models - Thesis Abstract and Contents | PDF",1785676724,486,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"invariance-testing-for-machine-learning-models-thesis-abstract-and-contents","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/invariance-testing-for-machine-learning-models-thesis-abstract-and-contents/117530/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05","2026-08-02",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What does invariance mean in machine learning model testing?","Question",{"text":76,"@type":77},"Invariance means the model produces consistent results when input objects are slightly modified while the ground truth remains unchanged.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"Why are existing invariance studies considered limited?",{"text":81,"@type":77},"They mainly rely on one or a few accuracy metrics, which restrict understanding of how models behave under different transformations.",{"name":83,"@type":74,"acceptedAnswer":84},"How does the proposed framework evaluate invariance beyond accuracy scores?",{"text":85,"@type":77},"It introduces a generic, extendable, fully automated testing methodology that provides detailed visual information and structured analysis across different attributes and variations.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]