[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120281-en":3,"doc-seo-120281-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120281,13056703019404,"Miles","https://ap-avatar.wpscdn.com/davatar_29158cc5080c5b710cf443261637dec0",8,"Research & Report","Specifying and Testing k-Safety Properties for Machine-Learning Models","Machine-learning models are increasingly used for tasks such as image classification and decision-making, making reliability a central concern. Beyond robustness and fairness, expressing general functional-correctness expectations remains difficult. This paper introduces k-safety properties, inspired by formal-methods specifications, to capture functional correctness via reasoning over k executions. It presents a first specification language and a framework using metamorphic testing to automatically validate these properties. Experiments show effective bug detection and evidence that found violations can guide training improved models.","Specifying and Testing k-Safety Properties for Machine-Learning Models  \nMaria Christakis 1 , Hasan Ferit Eniser2 , Jörg Hoffmann3 ,4 , Adish Singla2 and  \nValentin Wüstholz5  \n1TU Wien, Austria  \n2MPI-SWS, Germany  \n3 Saarland University, Saarland Informatics Campus, Germany  \n4 German Research Center for Artificial Intelligence (DFKI), Germany  \n5 ConsenSys, Austria  \n[maria.christakis@tuwien.ac.at](maria.christakis@tuwien.ac.at), {hfeniser, [adishs}@mpi-sws.org](adishs}@mpi-sws.org), [hoffmann@cs.uni-saarland.de](hoffmann@cs.uni-saarland.de),  \n[valentin.wustholz@consensys.net](valentin.wustholz@consensys.net)  \nAbstract  \nMachine-learning models are becoming increasingly prevalent in our lives, for instance assisting in image-classification or decision-making tasks. Consequently, the reliability of these models is of critical importance and has resulted in the development of numerous approaches for validating and verifying their robustness and fairness. However, beyond such specific properties, it is challenging to specify, let alone check, general functional-correctness expectations from models. In this paper, we take inspiration from specifications used in formal methods, expressing functional-correctness properties by reasoning about k different executions—so-called k-safety properties. Considering a credit-screening model of a bank, the expected property that \"if a person is denied a loan and their income decreases, they should still be denied the loan\" is a 2-safety property. Here, we show the wide applicability of ksafety properties for machine-learning models and present the first specification language for expressing them. We also operationalize the language in a framework for automatically validating such properties using metamorphic testing. Our experiments show that our framework is effective in identifying property violations, and that detected bugs could be used to train better models.  \n1 Introduction  \nDue to the impressive advances in machine learning and the unlimited availability of data, machine-learning (ML) models, e.g., neural networks, are rapidly becoming prevalent in our lives, for instance by assisting in image-classification or decision-making tasks. As a result, there is growing concern about the reliability of these models in performing such tasks. For example, it could be disastrous if an autonomous vehicle misclassifies a street sign, or if a recidivism-risk algorithm, which predicts whether a criminal is likely to re-offend, is unfair with respect to race. The research community is, of course, aware of these issues and has devised numerous  \ntechniques to validate and verify robustness and fairness properties of machine-learning models (e.g., [Huang et al., 2017; Gehr et al., 2018; Singh et al., 2019; Albarghouthi et al., 2017; Bastani et al., 2019; Urban et al., 2020; Carlini and Wagner, 2017; Goodfellow et al., 2015; Madry et al., 2018; Galhotra et al., 2017; Udeshi et al., 2018; Tramèr et al., 2017]) .  \nBeyond such specific properties however, it is challenging to express general functional-correctness expectations from such models, let alone check them, e.g., how can we specify that an image classifier should label images correctly? We take inspiration from specifications used in formal methods—so-called hyperproperties [Clarkson and Schneider, 2008]—capturing functional-correctness properties by simultaneously reasoning about multiple system executions. For example, consider a credit-screening model of a bank. The expected property that \"if a person is denied a loan and their income decreases, they should still be denied the loan\", or conversely\"if a person is granted a loan and their income increases, they should still be granted the loan\", is a 2-safety hyperproperty—we need two model executions to validate its correctness. In contrast, the property that \"a person with no income should be denied a loan\" is a standard (1-)safety property since it can be validated by individual model e","cbCaijMRnLlNpcHd","https://ap.wps.com/l/cbCaijMRnLlNpcHd","pdf",334699,1,10,"English","en",105,"# Abstract\n# 1 Introduction\n## Functional-correctness beyond robustness and fairness\n## Hyperproperties and k-safety\n## Examples across domains\n## Credit-screening model case study","[{\"question\":\"What problem does the paper address in machine-learning model validation?\",\"answer\":\"It addresses the challenge of specifying and checking general functional-correctness expectations, not just robustness and fairness properties.\"},{\"question\":\"What are k-safety properties and how do they relate to hyperproperties?\",\"answer\":\"k-safety properties generalize standard safety properties by requiring reasoning over k different executions, aligning with the hyperproperties view of simultaneous multi-execution analysis.\"},{\"question\":\"How are k-safety properties operationalized for automated checking?\",\"answer\":\"The paper proposes a first specification language for expressing k-safety properties and implements a framework that validates them automatically using metamorphic testing.\"}]","Specifying and Testing k-Safety Properties for Machine-Learning Models | PDF",1785729221,25,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"specifying-and-testing-k-safety-properties-for-machine-learning-models","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/specifying-and-testing-k-safety-properties-for-machine-learning-models/120281/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05","2026-08-03",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What problem does the paper address in machine-learning model validation?","Question",{"text":76,"@type":77},"It addresses the challenge of specifying and checking general functional-correctness expectations, not just robustness and fairness properties.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"What are k-safety properties and how do they relate to hyperproperties?",{"text":81,"@type":77},"k-safety properties generalize standard safety properties by requiring reasoning over k different executions, aligning with the hyperproperties view of simultaneous multi-execution analysis.",{"name":83,"@type":74,"acceptedAnswer":84},"How are k-safety properties operationalized for automated checking?",{"text":85,"@type":77},"The paper proposes a first specification language for expressing k-safety properties and implements a framework that validates them automatically using metamorphic testing.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":21,"slug":134},"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":107,"slug":138},19,"General","general"]