[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-119284-en":3,"doc-seo-119284-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},119284,7971461740909,"Levi","https://ap-avatar.wpscdn.com/davatar_155a257f0dc6eb9ab79c44ca47cae57d",8,"Research & Report","A Comprehensive Overview of Kernels in Machine Learning - Mathematical Foundations and Applications","Kernels are central to machine learning, allowing learning algorithms to work effectively in high-dimensional feature spaces without explicit transformations. This paper presents a comprehensive overview of regression kernels, covering their mathematical formulation, key properties, and theoretical basis, including Mercer's theorem and positive semidefinite (PSD) kernels. It reviews widely used kernel functions, then applies regression kernel methods to the bike sharing demand dataset, comparing performance across kernels. The study also addresses limitations and highlights current research trends and future directions in kernel-based learning.","A Comprehensive Overview of Kernels in Machine Learning: Mathematical Foundations and Applications  \nOmar El Khatiba * , Nabeel Alkhatibb  \na, bMath. and Computer Science Dept., Loyola University New Orleans, New Orleans, 70118 USA  \naEmail: [oelkhat@loyno.edu](oelkhat@loyno.edu)  \nbEmail: [nalkhati@my.loyno.edu](nalkhati@my.loyno.edu)  \nAbstract  \nKernels play a fundamental role in machine learning, enabling algorithms to operate efficiently and effectively in high-dimensional spaces. In this paper, we provide a comprehensive overview of regression kernels in machine learning, focusing on their mathematical foundations, properties, and practical applications. We begin with an introduction to the concept of regression kernels and their significance in machine learning. Then, we delve into the mathematical formulation of regression kernels, exploring Mercer's theorem and positive semidefinite (PSD) kernels. Next, we discuss popular kernel functions with their respective properties and applications. After that we apply regression kernel to the bike sharing demand dataset as a case study and compare the different kernel functions. Finally, we explore kernel limitations and current research trends and emerging directions in kernel-based learning, offering insights into the future potential of this powerful methodology. This work aims to serve as a resource for both researchers and practitioners seeking a thorough understanding of regression kernel-based approaches in machine learning.  \nKeywords: Kernel Applications; Mathematical Foundations of Kernels; Kernel Trick; Kernel Properties.  \n1. Introduction  \nMachine learning (ML) is a branch of artificial intelligence (AI) that focuses on learning from data to make predictions to unseen data. Traditionally, the theory and algorithms of machine learning and statistics have been very well developed for the linear case. Linear classifiers work best when the dataset classes are linearly separable [1] . However, real world data analysis problems often require nonlinear methods to detect the kind of dependencies that allow successful prediction of target of interest. For nonlinear decision boundary, one approach that allows us to still utilize linear classifiers is to apply a kernel [2] .  \nReceived: 11/16/2024  \nAccepted: 1/16/2025  \nPublished: 1/26/2025  \n* Corresponding author.  \nKernels play a fundamental role in machine learning, enabling algorithms to operate efficiently and effectively in high dimensional spaces. Kernels use the linear classifier to build non-linear decision boundaries by transferring the input dataset into high dimension space [1, 2, 3] .  \nIn this paper, we provide a comprehensive overview of kernels in machine learning, focusing on their mathematical foundations, properties, and practical applications. Section 2 review previous work and our study. Then section 3 introduces the concept of regression kernels. Next, we delve into the mathematical formulation of regression kernels in section 4. After that we explore Mercer's theorem and positive semi-definite (PSD) kernels in section 5. Next, we discuss popular kernel functions such as linear, polynomial, Gaussian (RBF), Laplacian and sigmoid kernels, along with their respective properties and applications in section 6. Then, we discuss applications of kernels in section 7. After that we showed a practical application of using linear regression kernels on the bike sharing demand dataset from Kaggle to find the minimum error and discuss the limitation of regression kernels in section 8. Finally, we conclude with a discussion on current research trendsand future directions in kernel-based learning in section 9.  \n2. Previous Work  \nMercer’s theorem [4] established the conditions under which a kernel function can be expressed as an inner product in a high-dimensional space. This theorem is the foundation for kernel methods, ensuring that the kernel matrix is positive semi-definite and valid for use in optimization probl","cbCailTnAoCzKYmD","https://ap.wps.com/l/cbCailTnAoCzKYmD","pdf",1236355,1,23,"English","en",105,"# Introduction\n# Previous Work\n# Regression Kernels\n# Mathematical Formulation of Regression Kernels\n# Mercer's Theorem and PSD Kernels\n# Popular Kernel Functions and Applications\n# Kernel Applications\n# Practical Case Study and Limitations\n# Conclusion and Future Directions","[{\"question\":\"What problem do regression kernels address in machine learning?\",\"answer\":\"Regression kernels provide a way to model nonlinear relationships by enabling kernel-based methods to effectively operate in high-dimensional feature spaces. This improves prediction when linear assumptions are insufficient.\"},{\"question\":\"What is Mercer's theorem’s role in kernel methods?\",\"answer\":\"Mercer's theorem establishes when a kernel function can be represented as an inner product in a high-dimensional space. It supports kernel validity by ensuring the resulting kernel matrix is positive semidefinite for optimization.\"},{\"question\":\"Which kernel functions are compared and how is the comparison evaluated?\",\"answer\":\"The paper discusses several common kernel functions, including linear, polynomial, Gaussian (RBF), Laplacian, and sigmoid kernels. It applies regression kernel methods to the bike sharing demand dataset and compares their outcomes to highlight differences in performance.\"}]","A Comprehensive Overview of Kernels in Machine Learning - Mathematical Foundations and Applications | PDF",1785723507,58,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"a-comprehensive-overview-of-kernels-in-machine-learning-mathematical-foundations-and-applications","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/a-comprehensive-overview-of-kernels-in-machine-learning-mathematical-foundations-and-applications/119284/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem do regression kernels address in machine learning?","Question",{"text":75,"@type":76},"Regression kernels provide a way to model nonlinear relationships by enabling kernel-based methods to effectively operate in high-dimensional feature spaces. This improves prediction when linear assumptions are insufficient.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is Mercer's theorem’s role in kernel methods?",{"text":80,"@type":76},"Mercer's theorem establishes when a kernel function can be represented as an inner product in a high-dimensional space. It supports kernel validity by ensuring the resulting kernel matrix is positive semidefinite for optimization.",{"name":82,"@type":73,"acceptedAnswer":83},"Which kernel functions are compared and how is the comparison evaluated?",{"text":84,"@type":76},"The paper discusses several common kernel functions, including linear, polynomial, Gaussian (RBF), Laplacian, and sigmoid kernels. It applies regression kernel methods to the bike sharing demand dataset and compares their outcomes to highlight differences in performance.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]