[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-116986-en":3,"doc-seo-116986-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},116986,8796095360427,"Lucas Martin","https://ap-avatar.wpscdn.com/davatar_994ba38a5ba835b3df7d355c54d3ed8d",8,"Research & Report","Classic machine learning methods - Chapter 2","This chapter surveys classic machine learning methods with a clear progression in complexity, emphasizing supervised learning for classification and regression. It covers nearest-neighbor methods, linear regression, logistic regression, support vector machines, decision trees and random forests, and kernel methods for extending linear ideas to non-linear settings. It explains overfitting and mitigation strategies, then introduces unsupervised learning approaches for clustering and dimensionality reduction. Neural networks and deep learning are intentionally deferred to later chapters.","arXiv :2310 . 11470v1 [ cs .LG] 24 May 2023  \nChapter 2  \nClassic machine learning methods  \nJohann Faouzi*1 and Olivier Colliot2  \n1 CREST, ENSAI, Campus de Ker-Lann, 51 Rue Blaise Pascal, BP 37203 –  \n35172 Bruz Cedex, France  \n2 Sorbonne Universit´e, Institut du Cerveau-Paris Brain Institute-ICM, CNRS, Inria, Inserm, AP-HP, Hˆopital de la Piti´e-Salpˆetri`ere, F-75013, Paris, France  \n* Corresponding author: e-mail [address: johann.faouzi@gmail.com](address: johann.faouzi@gmail.com)  \nAbstract  \nIn this chapter, we present the main classic machine learning methodss. A large part of the chapter is devoted to supervised learning techniques for classification and regression, including nearest-neighbor methods, linear and logistic regressions, support vector machines and tree-based algorithms. We also describe the problem of overfitting as well as strategies to overcome it. We finally provide a brief overview of unsupervised learning methods, namely for clustering and dimensionality reduction. The chapter does not cover neural networks and deep learning as these will be presented in Chapters 3, 4, 5 and 6 .  \nKeywords: machine learning, classification, regression, clustering, dimensionality reduction  \n1. Introduction  \nThis chapter presents the main classic machine learning (ML) methods. There is a focus on supervised learning methods for classification and regression, but we also describe some unsupervised approaches. The  \nTo appear in  \nO. Colliot (Ed.), Machine Learning for Brain Disorders, Springer  \n2 Faouzi and Colliot  \nchapter is meant to be readable by someone with no background in machine learning. It is nevertheless necessary to have some basic notions of linear algebra, probabilities and statistics. If this is not the case, we refer the reader to chapters 2 and 3 of [1] .  \nThe rest of this chapter is organized as follows. Rather than grouping methods by categories (for instance classification or regression methods), we chose to present methods by increasing order of complexity. We first provide the notations in Section 2 . We then describe a very intuitive family of methods, that of nearest neighbors (Section 3) . We continue with linear regression (Section 4) and logistic regression (Section 5), the later being a classification technique. We subsequently introduce the problem of overfitting (Section 6) as well as strategies to mitigate it (Section 7) . Section 8 describes support vector machines (SVM) . Section 9 explainshow binary classification methods can be extended to a multi-class setting. We then describe methods which are specifically adapted to the case of normal distributions (Section 10) . Decision trees and random forests are described in Section 11 . We then briefly describe some unsupervised learning techniques, namely for clustering (Section 12) and dimensionality reduction (Section 13) . The chapter ends with a description of kernel methods which can be used to extend linear techniques to non-linear cases (Section 14) . Box 1 summarizes the methods presented in this chapter, grouped by categories and then sorted in order of appearance.  \n2. Notations  \nLet n be the number of samples and p be the number of features. An input sample is thus a p-dimensional vector:  \n􀀲  \nx = 􀀶  \n􀀴  \nx 1  \n.  \n.  \n.  \nxp  \n􀀳  \n􀀷  \n􀀵  \nAn output sample is denoted by y. Thus, a sample is (x, y) . The dataset of n samples can then be summarized as an n × p matrix X representing the input data and an n-dimensional vector y representing the target data:  \n􀀲 x (1) 􀀳 􀀲 x (1)1 . . . x(1)p􀀳 􀀲 y 1 􀀳  \nX = 􀀶 ... 􀀷 = 􀀶 ... . . . ... 􀀷 , y = 􀀶 ... 􀀷  \n􀀴x (n)􀀵 􀀴x 1(n)   xp(n)􀀵 􀀴yn􀀵  \nMachine Learning for Brain Disorders, Chapter 2  \nClassic machine learning methods 3  \nBox 1: Main classic ML methods  \n• Supervised learning  \n– Classification: nearest neighbors, logistic regression, support vector machine (SVM), naive Bayes, linear discriminant analysis (LDA), quadratic discriminant analysis, tree-based models (decision tree, rando","cbCairm0LixRBx3L","https://ap.wps.com/l/cbCairm0LixRBx3L","pdf",4111174,1,60,"English","en",105,"# Introduction\n# Notations\n# Nearest-neighbor methods\n# Linear regression\n# Logistic regression\n# Overfitting and mitigation strategies\n# Support vector machines (SVM)\n# Multi-class extension\n# Methods adapted to normal distributions\n# Decision trees and random forests\n# Unsupervised learning: clustering and dimensionality reduction\n# Kernel methods","[{\"question\":\"What classic machine learning topics does this chapter focus on?\",\"answer\":\"It focuses on supervised learning for classification and regression, including nearest neighbors, linear and logistic regression, support vector machines, and tree-based models, followed by brief unsupervised learning for clustering and dimensionality reduction.\"},{\"question\":\"Why is overfitting discussed, and how is it handled?\",\"answer\":\"The chapter introduces the overfitting problem and presents strategies to mitigate it, before moving on to methods such as SVM and tree-based algorithms.\"},{\"question\":\"Does the chapter cover neural networks and deep learning?\",\"answer\":\"No. Neural networks and deep learning are not covered here because they are presented in later chapters (Chapters 3–6).\"}]","Classic machine learning methods - Chapter 2 | PDF",1785672976,151,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"classic-machine-learning-methods-chapter-2","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/classic-machine-learning-methods-chapter-2/116986/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-02",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What classic machine learning topics does this chapter focus on?","Question",{"text":75,"@type":76},"It focuses on supervised learning for classification and regression, including nearest neighbors, linear and logistic regression, support vector machines, and tree-based models, followed by brief unsupervised learning for clustering and dimensionality reduction.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why is overfitting discussed, and how is it handled?",{"text":80,"@type":76},"The chapter introduces the overfitting problem and presents strategies to mitigate it, before moving on to methods such as SVM and tree-based algorithms.",{"name":82,"@type":73,"acceptedAnswer":83},"Does the chapter cover neural networks and deep learning?",{"text":84,"@type":76},"No. Neural networks and deep learning are not covered here because they are presented in later chapters (Chapters 3–6).","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,109,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":21,"slug":108},5,"Comic","comic",{"id":110,"doc_module":4,"doc_module_name":46,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]