[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-125844-en":3,"doc-seo-125844-105":31,"detail-sidebar-cat-0-en-105":92},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":28,"seo_description":14,"update_tm":29,"read_time":30},125844,1099523882367,"Hazel","https://ap-avatar.wpscdn.com/davatar_9964176cb1d06d4a9deccf72a44ae3dc",8,"Research & Report","Machine Learning - Evaluation - Cross-validation, Metrics, Importance Scores","Machine learning models perform differently depending on data characteristics and application goals, making model selection difficult among many available options. A structured evaluation process helps identify which supervised or unsupervised model fits a given task and user needs. This chapter covers evaluation techniques such as cross-validation and bootstrap, along with practical performance measures for real-world deployment. It explains optimal and acceptable ranges for each measure and clarifies how the measures relate to each other when interpreting results.","Machine Learning—Evaluation (Cross-validation, Metrics, Importance Scores...)  \nAbdulhakim Qahtan   \nAbstract  \nThe high performance of machine learning (ML) techniques when handling different data analytics tasks resulted in developing a large number of models. Although these models can provide multiple options for performing the task at hand, selecting the right model becomes more challenging. As the ML models perform differently based on the nature of the data and the application, designing a good evaluation process would help in selecting the appropriate ML model. Considering the nature of the ML model and the user’s interest, different evaluation experiments can be designed to get better insights about the performance of the model. In this chapter, we discuss different evaluation techniques that suit both supervised and unsupervised models including cross-validation and bootstrap. Moreover, we present a set of performance measures that can be used as an indication on how the model would perform in real applications. For each of the performance measures, we discuss the optimal values that can be achieved by a given model and what should be considered as acceptable. We also show the relationship between the different measures,  \nA. Qahtan (B)  \nDepartment of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands  \ne-mail: [a.a.a.qahtan@uu.nl](a.a.a.qahtan@uu.nl)  \nwhich can give more insights when interpreting the results of a given ML model.  \nKeywords  \nMachine learning · Training · Testing · Regression · Classification · Clustering  \nBefore discussing how to evaluate the Machine Learning (ML) models, we give a brief summary about the different models and how they work. Depending on the nature of the data and the task at hand, different machine learning models can be selected. These models are usually parameterized to automatically adjust their performance according to the data and the performance criteria through a set of tunable parameters. The values of the different parameters are learned and automatically adjusted during a training (fitting) stage of the model development. Learning the models’parameters can be achieved using one of three main approaches.  \n• Supervised learning: When the training set consists of labeled examples (exemplars), the algorithms use the labeled examples to learn how to generalize to the set of all possible inputs. Examples of techniques that belong to supervised learning category include logistic regression [3], support vector machines [6], neural networks [11] decision trees [22], random forest [6], etc.  \n© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023  \nF. W. Asselbergs et al. (eds.), Clinical Applications of Artificial Intelligence in Real-World Data, [https://doi.org/10.1007/978-3-031-36678-9_11](https://doi.org/10.1007/978-3-031-36678-9_11)  \n175  \n• Unsupervised learning: Refers to the set of algorithms that learn from a set of unlabeled examples. These algorithms learn the patterns that exist in the data according to a specific criterion that could be statistical, geometric or similarity criterion. Examples of unsupervised learning include k-means clustering [5] and kernel density estimation [17] .  \n• Reinforcement learning: In this set of algorithms, learning is achieved by iterative exploring the solution space and receiving a feedback on the quality ofthe solution. The exploration is repeated until a satisfactory performance measure value is reached.  \nThe decision on using supervised/unsupervised learning technique will depend mainly on the availability of the labeled examples in the training set. In this chapter, we focus on the evaluation of the different machine learning techniques.  \n1 Background  \nEvaluation is a key and challenging task when selecting a Machine Learning (ML) model for aspecific problem. There are lots of models that can be used, but which one will perform better than the others. This requires a systema","cbCainymaP5ACVbA","https://ap.wps.com/l/cbCainymaP5ACVbA","pdf",362796,2,1,13,"English","en",105,"# Background\n## Model selection and evaluation criteria\n## Regression, classification, and clustering measures\n# Evaluation techniques\n## Cross-validation\n## Bootstrap\n# Performance measures and interpretation\n## Optimal vs acceptable values\n## Relationships among metrics\n# Learning paradigms overview\n## Supervised learning\n## Unsupervised learning\n## Reinforcement learning","[{\"question\":\"Why is evaluation a key challenge when selecting a machine learning model?\",\"answer\":\"Many models can be used for the same task, but their performance varies. A systematic evaluation process is needed to determine which model performs better for the specific problem and data nature.\"},{\"question\":\"How should supervised and unsupervised models be evaluated differently?\",\"answer\":\"Supervised evaluation compares predicted target values with actual values, while unsupervised evaluation is based on statistical measures such as clustering quality indices (e.g., Silhouette score).\"},{\"question\":\"When should regression models versus classifiers be used?\",\"answer\":\"Use regression when the target variable contains continuous values. Use a classifier when the target variable has a small set of distinct class labels, and evaluate by matching predicted and actual labels.\"}]","Machine Learning - Evaluation - Cross-validation, Metrics, Importance Scores | PDF",1785901539,33,{"code":4,"msg":32,"data":33},"ok",{"site_id":25,"language":24,"slug":34,"title":13,"keywords":35,"description":14,"schema_data":36,"social_meta":87,"head_meta":89,"extra_data":91,"updated_unix":29},"machine-learning-evaluation-cross-validation-metrics-importance-scores","",{"@graph":37,"@context":86},[38,54,69],{"@type":39,"itemListElement":40},"BreadcrumbList",[41,45,48,51],{"item":42,"name":43,"@type":44,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":46,"name":47,"@type":44,"position":20},"https://docshare.wps.com/document/","Document",{"item":49,"name":12,"@type":44,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":44,"position":53},"https://docshare.wps.com/document/machine-learning-evaluation-cross-validation-metrics-importance-scores/125844/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":42,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-16","2026-08-05",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72,78,82],{"name":73,"@type":74,"acceptedAnswer":75},"Why is evaluation a key challenge when selecting a machine learning model?","Question",{"text":76,"@type":77},"Many models can be used for the same task, but their performance varies. A systematic evaluation process is needed to determine which model performs better for the specific problem and data nature.","Answer",{"name":79,"@type":74,"acceptedAnswer":80},"How should supervised and unsupervised models be evaluated differently?",{"text":81,"@type":77},"Supervised evaluation compares predicted target values with actual values, while unsupervised evaluation is based on statistical measures such as clustering quality indices (e.g., Silhouette score).",{"name":83,"@type":74,"acceptedAnswer":84},"When should regression models versus classifiers be used?",{"text":85,"@type":77},"Use regression when the target variable contains continuous values. Use a classifier when the target variable has a small set of distinct class labels, and evaluate by matching predicted and actual labels.","https://schema.org",{"og:url":52,"og:type":88,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":90,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":93},[94,98,102,106,111,116,121,124,129,132,136],{"id":21,"doc_module":4,"doc_module_name":47,"category_name":95,"show_sort_weight":96,"slug":97},"Story & Novel",90,"story-novel",{"id":20,"doc_module":4,"doc_module_name":47,"category_name":99,"show_sort_weight":100,"slug":101},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":47,"category_name":103,"show_sort_weight":104,"slug":105},"Exam",70,"exam",{"id":107,"doc_module":4,"doc_module_name":47,"category_name":108,"show_sort_weight":109,"slug":110},5,"Comic",60,"comic",{"id":112,"doc_module":4,"doc_module_name":47,"category_name":113,"show_sort_weight":114,"slug":115},6,"Technology",50,"technology",{"id":117,"doc_module":4,"doc_module_name":47,"category_name":118,"show_sort_weight":119,"slug":120},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":47,"category_name":12,"show_sort_weight":122,"slug":123},30,"research-report",{"id":125,"doc_module":4,"doc_module_name":47,"category_name":126,"show_sort_weight":127,"slug":128},9,"Religion & Spirituality",20,"religion-spirituality",{"id":127,"doc_module":4,"doc_module_name":47,"category_name":130,"show_sort_weight":127,"slug":131},"World Cup","world-cup",{"id":133,"doc_module":4,"doc_module_name":47,"category_name":134,"show_sort_weight":133,"slug":135},10,"Lifestyle","lifestyle",{"id":137,"doc_module":4,"doc_module_name":47,"category_name":138,"show_sort_weight":107,"slug":139},19,"General","general"]