[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-seo-290504-105":3,"detail-sidebar-cat-0-en-105":81,"doc-detail-290504-en":130},{"code":4,"msg":5,"data":6},0,"ok",{"site_id":7,"language":8,"slug":9,"title":10,"keywords":11,"description":12,"schema_data":13,"social_meta":74,"head_meta":76,"extra_data":78,"updated_unix":80},105,"en","practical-machine-learning-is-learning-on-small-samples","Practical machine learning is learning on small samples","","Practical machine learning is analyzed under the limitation of limited observations, where future dependence must be inferred without access to infinite training data or an infinite population. The work argues that learning is made possible by an implicit smoothness assumption: nearby data points yield similar feedback. A “Practical learning paradigm” is formalized, defining learner descriptions and rules, and interpreting popular methods such as local smoothing, k-NN, decision trees, Naive Bayes, and SVMs for classification and regression.",{"@graph":14,"@context":73},[15,34,56],{"@type":16,"itemListElement":17},"BreadcrumbList",[18,23,27,31],{"item":19,"name":20,"@type":21,"position":22},"https://docshare.wps.com","Home","ListItem",1,{"item":24,"name":25,"@type":21,"position":26},"https://docshare.wps.com/document/","Document",2,{"item":28,"name":29,"@type":21,"position":30},"https://docshare.wps.com/document/research-report/","Research & Report",3,{"item":32,"name":10,"@type":21,"position":33},"https://docshare.wps.com/document/practical-machine-learning-is-learning-on-small-samples/290504/",4,{"url":32,"name":10,"@type":35,"image":36,"author":41,"headline":10,"publisher":44,"fileFormat":47,"inLanguage":8,"description":12,"dateModified":48,"datePublished":49,"encodingFormat":47,"isAccessibleForFree":50,"interactionStatistic":51},"DigitalDocument",{"url":37,"@type":38,"width":39,"height":40},"https://docshare.wps.com/thumbnails/practical-machine-learning-is-learning-on-small-samples/290504.png","ImageObject",300,407,{"name":42,"@type":43},"Noah","Person",{"url":19,"name":45,"@type":46},"DocShare","Organization","application/pdf","2026-10-18","2026-09-17",true,{"@type":52,"interactionType":53,"userInteractionCount":55},"InteractionCounter",{"@type":54},"ViewAction",9,{"@type":57,"mainEntity":58},"FAQPage",[59,65,69],{"name":60,"@type":61,"acceptedAnswer":62},"Why does the paper claim machine learning can work with limited samples?","Question",{"text":63,"@type":64},"It argues that learning relies on an implicit assumption that the underlying dependence is relatively “smooth,” so similar features imply similar feedback even without infinite training data.","Answer",{"name":66,"@type":61,"acceptedAnswer":67},"What does the “Practical learning paradigm” formalize?",{"text":68,"@type":64},"It formalizes the idea of selecting a hypothesis that smoothly approximates the training set, including terminology and rules for describing learners.",{"name":70,"@type":61,"acceptedAnswer":71},"How are common ML methods related to the paradigm?",{"text":72,"@type":64},"Popular learners like local smoothing, k-NN, decision trees, Naive Bayes, and SVMs are presented as implementations of the same paradigm.","https://schema.org",{"og:url":32,"og:type":75,"og:title":10,"og:site_name":45,"og:description":12},"article",{"robots":77,"canonical":32},"index,follow",{"doc_id":79,"site_id":7},290504,1789641862,{"code":4,"msg":82,"data":83},"success",[84,88,92,96,101,106,111,115,119,122,126],{"id":22,"doc_module":4,"doc_module_name":25,"category_name":85,"show_sort_weight":86,"slug":87},"Story & Novel",90,"story-novel",{"id":26,"doc_module":4,"doc_module_name":25,"category_name":89,"show_sort_weight":90,"slug":91},"Literature",80,"literature",{"id":33,"doc_module":4,"doc_module_name":25,"category_name":93,"show_sort_weight":94,"slug":95},"Exam",70,"exam",{"id":97,"doc_module":4,"doc_module_name":25,"category_name":98,"show_sort_weight":99,"slug":100},5,"Comic",60,"comic",{"id":102,"doc_module":4,"doc_module_name":25,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":25,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":112,"doc_module":4,"doc_module_name":25,"category_name":29,"show_sort_weight":113,"slug":114},8,30,"research-report",{"id":55,"doc_module":4,"doc_module_name":25,"category_name":116,"show_sort_weight":117,"slug":118},"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":25,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":25,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":25,"category_name":128,"show_sort_weight":97,"slug":129},19,"General","general",{"code":4,"msg":82,"data":131},{"doc_id":79,"user_id":132,"nickname":42,"user_avatar":133,"doc_module":4,"category_id":112,"category_name":29,"doc_title":10,"doc_description":12,"doc_content":134,"file_id":135,"file_url":136,"file_type":137,"file_size":138,"view_count":55,"is_deleted":4,"is_public":22,"is_downloadable":22,"audit_status":22,"page_count":139,"language":140,"language_code":8,"site_id":7,"html_lang":8,"table_of_contents":141,"faqs":142,"seo_title":143,"seo_description":12,"update_tm":80,"read_time":109,"preview_page_count":4},8796095462418,"https://ap-avatar.wpscdn.com/avatar/80000253c1241d02b47?x-image-process=image/resize,m_fixed,w_180,h_180&k=1778826106357471780","arXiv :2501 .0 1836v 1 [ cs .LG] 3 Jan 2025  \nPractical machine learning is learning on small  \nsamples  \nMarina Sapir  \nAbstract  \nBased on limited observations, machine learning discerns a dependence which is expected to hold in the future. What makes it possible? Statistical learning theory imagines inde􀀌nitely increasing training sample to justify its approach. In reality, there is no in􀀌nite time or even in􀀌nite general population for learning.  \nHere I argue that practical machine learning is based on an implicit assumption that underlying dependence is relatively “smooth” : likely, there are no abrupt di􀀋erences in feedback between cases with close data points.  \nFrom this point of view learning shall involve selection of the hypothesis“smoothly” approximating the training set. I formalize this as Practical learning paradigm. The paradigm includes terminology and rules for description of learners. Popular learners (local smoothing, k-NN, decision trees, Naive Bayes, SVM for classi􀀌cation and for regression) are shown here to be implementations of this paradigm.  \n1 Givens, goals and assumptions  \nDenote 􀀊 the set of real life objects of interest. For example, this may be patients with skin cancer, or bank clients or engine failures. The objects have two types of properties: hidden (feedback) and manifested, which are called features. The properties areassigned numerical values. Denote X domain of numerical feature vectors for objects in 􀀊 . The hidden property has values in a numerical domain denoted by Y.  \nThe underlying dependence ϕ : X → Y is what we learn. It is expected to beinexact. There are several reasons for inexactness, for example:  \n• we do not know what the feedback, actually, depends on;  \n• numerical evaluations of features have intrinsic uncertainty;  \n• the objects in 􀀊 and the dependence ϕ are changing in time; new objects may somehow be di􀀋erent from what we observed before.  \nHypothesis is a function f : X → Y. Cases are pairs hx, yi, where x ∈ X, y ∈ Y. If f is a hypothesis, then cases hx, f(x)i are called hypothetical. Denote pfq all hypothetical cases for the hypothesis f. Observations are cases from direct inspections of objects in 􀀊 . A training set T is a 􀀌nite set o observations available for analysis. By default, no two cases in T have identical 􀀌rst component.  \nDenone M(h, T) = T ∪ phq set of all cases for the hypothesis h and the training set T : all these cases together are representation of the underlying dependence.  \nThe next assumptions summarizes our intuitive understanding of the problem:  \nImplicite learning assumptions  \n• Any two observations hx1 , y 1 i, hx2 , y2 i with “similar” features x1 , x2 likely have “similar” feedback y 1 , y2.  \n• For any two hypotheses h 1 , h2 , a hypothesis h ∈ {h1 , h2 } with lower “inconsistency” on the set of cases M(h, T) represents the underlying dependence ϕ better.  \n• The goal of learning is to 􀀌nd a hypothesis with lowest “inconsistency” on M (h, T) .  \nThe assumption explain why we undertake the learning without having an inde􀀌 -nitely increasing training set or a knowledge of all relevant features. I will show that the assumption is su􀀎cient to build e􀀎cient learners in many cases.  \nBelow, I de􀀌ne the concepts of these assumptions in general terms and show on examples of popular learners how they are interpreted.  \nI will overload the notation [·] to mean di􀀋erent things for di􀀋erent types of data.  \nFor:  \nVector x :  \nCase α :  \nSequence of cases T :  \n􀀂x 􀀃 i  \n􀀂α 􀀃 i  \n􀀂T􀀃 i  \nis i-th component of x;  \nis i-th component of α;  \nis sequence of i-th components of cases in T.  \n2 Traditional views on ML  \nIt is interesting to observe that ML problem does not have a general de􀀌nition. Here are some popular views on the problem.  \n2.1 Is ML an induction?  \nIt is a common belief that ML is an induction inference.  \nThe dictionary says that induction is a method of reasoning from a part to a whole, from particulars to generals, or from an ","cbCaihaUKFK64LOx","https://ap.wps.com/l/cbCaihaUKFK64LOx","pdf",178347,16,"English","# Abstract\n# Givens, goals and assumptions\n# Traditional views on ML\n## Is ML an induction?\n## Is ML a prediction problem?\n## Does Statistical Learning describe ML?","[{\"question\":\"Why does the paper claim machine learning can work with limited samples?\",\"answer\":\"It argues that learning relies on an implicit assumption that the underlying dependence is relatively “smooth,” so similar features imply similar feedback even without infinite training data.\"},{\"question\":\"What does the “Practical learning paradigm” formalize?\",\"answer\":\"It formalizes the idea of selecting a hypothesis that smoothly approximates the training set, including terminology and rules for describing learners.\"},{\"question\":\"How are common ML methods related to the paradigm?\",\"answer\":\"Popular learners like local smoothing, k-NN, decision trees, Naive Bayes, and SVMs are presented as implementations of the same paradigm.\"}]","Practical machine learning is learning on small samples | PDF"]