[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-124480-en":3,"doc-seo-124480-105":30,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},124480,1099514068365,"Aurelia","https://ap-avatar.wpscdn.com/avatar/10000253d8d9f28188e?_k=1776742907772140068",8,"Research & Report","Methodologies for Systematic Evaluation and Targeted Mitigation of Deficiencies in Critical Machine Learning Models - Dissertation","Despite the growing use of machine learning in healthcare, critical challenges remain unaddressed: models often fail to respond appropriately to life-threatening conditions, generalize poorly across real-world clinical settings, and produce unequal performance across patient subgroups. These limitations undermine reliability, safety, and equity of AI decision-making in high-stakes environments such as intensive care. The work proposes a comprehensive evaluation and mitigation strategy focused on responsiveness and fairness, including systematic testing with synthesized cases and domain-knowledge-guided modeling to reduce dangerous blind spots.","Methodologies for Systematic Evaluation and Targeted Mitigation of Deficiencies in Critical Machine Learning Models  \nTanmoy Sarkar Pias  \nDissertation submitted to the Faculty of the  \nVirginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of  \nDoctor of Philosophy  \nin  \nComputer Science and Applications  \nDaphne (Danfeng) Yao, Chair  \nT. M. Murali  \nPearl Chiu  \nIsmini Lourentzou  \nShalmali Joshi  \nJuly 28, 2025  \nBlacksburg, Virginia  \nKeywords: AI Trustworthiness, Responsiveness, Knowledge guided ML, Custom Loss,  \nHealthcare  \nCopyright 2025, Tanmoy Sarkar Pias  \nMethodologies for Systematic Evaluation and Targeted Mitigation of Deficiencies in Critical Machine Learning Models  \nTanmoy Sarkar Pias  \n(ABSTRACT)  \nDespite the growing use of machine learning in healthcare, critical challenges remain unaddressed—models often fail to respond appropriately to life-threatening conditions, exhibit poor generalizability in real-world clinical settings, and show unequal performance across patient subgroups. These limitations compromise the reliability, safety, and equity of AI-driven decision-making, especially in high-stakes environments like intensive care. In this work, we outline a comprehensive evaluation and mitigation strategy to address both responsiveness and fairness shortcomings.. We develop testing approaches to systematically assess models’ability to respond to serious medical emergencies. Using generated test cases, we found that statistical machine-learning models trained solely from patient data are grossly insufficient and have many dangerous blind spots. Specifically, we identified serious deficiencies in the models’ responsiveness, i.e. , the inability to recognize severely impaired medical conditions or rapidly deteriorating health. For in-hospital mortality prediction, the models tested using our synthesized cases fail to recognize 66% of the test cases involving injuries. In some instances, the models fail to generate adequate mortality risk scores for all test cases. We also applied our testing methods to assess the responsiveness of 5-year breast and lung cancer prediction models and identified similar kinds of deficiencies. To address the low responsiveness of machine learning models to critical health conditions, we integrated domain knowledge into the modeling framework using two complementary strategies: (i) a custom loss function that penalizes violations of medical constraints, and (ii) a rule-based decision  \ntree derived from clinical knowledge, aggregated with a data-driven model. The resulting knowledge-guided models demonstrated notable improvements in performance, particularly under critical scenarios. For instance, recall improved by 7% on the full glucose test set and by 27% for critically high glucose cases, achieving 94–99% accuracy in detecting patients with severely abnormal glucose levels. Similar trends were observed for other vital signs. Moreover, the decision tree-based hybrid model enhanced early sepsis detection accuracy by 4%, underscoring the benefit of combining clinical knowledge with statistical learning for high-stakes medical applications. In addition, we address a bias problem we identified in models predicting type 2 diabetes, which disproportionately impacts younger adults—a growing segment of diabetes patients. In this research, we identify this deficiency in traditional machine learning models and propose an algorithm to mitigate the bias towards the young population when predicting diabetes. Deviating from the traditional concept of one-model-fits-all, we train customized machine-learning models for each age group. Our proposed solution consistently improves recall of diabetes class by 26% to 40% in the young age group (30-44) . Moreover, our technique outperforms 7 commonly used whole-group sampling techniques such as random oversampling, SMOTE, and AdaSyns techniques by at least 36% in terms of diabetes recall in ","cbCaidQcG3wpKDZc","https://ap.wps.com/l/cbCaidQcG3wpKDZc","pdf",50812590,1,224,"English","en",105,"# Abstract\n## Responsiveness and fairness challenges in critical healthcare ML\n## Systematic evaluation with synthesized test cases\n## Domain-knowledge integration for targeted mitigation\n## Bias mitigation via age-group customized models","[{\"question\":\"What main problems does the dissertation address in critical healthcare ML models?\",\"answer\":\"It addresses inadequate responsiveness to life-threatening conditions, poor generalizability in real clinical settings, and unequal performance across patient subgroups, which harm safety and equity.\"},{\"question\":\"How does the work evaluate model deficiencies?\",\"answer\":\"It develops testing approaches that use synthesized/generated test cases to systematically probe whether models can recognize severe medical emergencies and produce reliable risk assessments.\"},{\"question\":\"What mitigation strategies are proposed to improve responsiveness and fairness?\",\"answer\":\"The work integrates domain knowledge through a custom loss enforcing medical constraints and a clinical rule-based decision tree combined with a data-driven model; it also proposes an age-group tailored modeling approach to mitigate bias for younger adults in type 2 diabetes prediction.\"}]","Methodologies for Systematic Evaluation and Targeted Mitigation of Deficiencies in Critical Machine Learning Models - Dissertation | PDF",1785822691,564,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":28},"methodologies-for-systematic-evaluation-and-targeted-mitigation-of-deficiencies-in-critical-machine-learning-models-dissertation","",{"@graph":36,"@context":86},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/methodologies-for-systematic-evaluation-and-targeted-mitigation-of-deficiencies-in-critical-machine-learning-models-dissertation/124480/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-05","2026-08-04",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"What main problems does the dissertation address in critical healthcare ML models?","Question",{"text":76,"@type":77},"It addresses inadequate responsiveness to life-threatening conditions, poor generalizability in real clinical settings, and unequal performance across patient subgroups, which harm safety and equity.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How does the work evaluate model deficiencies?",{"text":81,"@type":77},"It develops testing approaches that use synthesized/generated test cases to systematically probe whether models can recognize severe medical emergencies and produce reliable risk assessments.",{"name":83,"@type":74,"acceptedAnswer":84},"What mitigation strategies are proposed to improve responsiveness and fairness?",{"text":85,"@type":77},"The work integrates domain knowledge through a custom loss enforcing medical constraints and a clinical rule-based decision tree combined with a data-driven model; it also proposes an age-group tailored modeling approach to mitigate bias for younger adults in type 2 diabetes prediction.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":46,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":46,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":46,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]