[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-160244-en":3,"doc-seo-160244-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},160244,2336474459895,"Gloria","https://ap-avatar.wpscdn.com/avatar/22000baeef7a5ed0655?x-image-process=image/resize,m_fixed,w_180,h_180&k=1786071322749376916",8,"Research & Report","Maximum Margin Principal Components","Maximum Margin Principal Components presents a class-focused alternative to Principal Component Analysis for dimensionality reduction when the prediction objective is not least-squares error. The method selects a projection that minimizes differences in margin distribution between the original data and its low-dimensional projection, targeting classification under zero-one loss. Experiments demonstrate competitive or improved classification error versus PCA, including broad performance relative to Partial Least Squares and Lasso, while sometimes matching them statistically.","arXiv : 1705 .06371v1 [ stat .ML] 17 May 2017  \nMaximum Margin Principal Components Xianghui Luo∗1 and Robert J. Durrant†2  \n1 Department of Computer Science  \n2 Department of Mathematics and Statistics  \nUniversity of Waikato Hamilton, New Zealand  \nAbstract  \nPrincipal Component Analysis (PCA) is a very successful dimensionality reduction technique, widely used in predictive modeling. A key factor in its widespread use in this domain is the fact that the projection of a dataset onto its 􀀌rst K principal components minimizes the sum of squared errors between the original data and the projected data over all possible rank K projections. Thus, PCA provides optimal low-rank representations of data for least-squares linear regression under standard modeling assumptions. On the other hand, when the loss function for a prediction problem is not the least-squares error, PCA is typically a heuristic choice of dimensionality reduction – in particular for classi􀀌cation problems under the zero-one loss. In this paper we target classi􀀌cation problems by proposing a straightforward alternative to PCA that aims to minimize the di􀀋erence in margin distribution between the original and the projected data. Extensive experiments show that our simple approach typically outperforms PCA on any particular dataset, in terms of classi􀀌cation error, though this di􀀋erence is not always statistically signi􀀌cant, and despite being a 􀀌lter method is frequently competitive with Partial Least Squares (PLS) and Lasso on a wide range of datasets.  \n􀀃 [xluo@xaut.edu.cn](xluo@xaut.edu.cn)[ ](xluo@xaut.edu.cn)†[bobd@waikato.ac.nz](bobd@waikato.ac.nz)  \n1 Background and Introduction  \nDimensionality reduction techniques are a core part of the Statistics and Machine Learning toolbox, widely used in predictive modeling to improve a range of measures including generalization performance, interpretability oridenti􀀌ability of models, and the time and space complexity of learning or prediction. There are many such techniques, see e.g. [7] for a survey, but chief amongst them are simple linear techniques and the most widely-used of these in practice is probably Principal Components Analysis (PCA) [15] and its variants. Applications of PCA include [27, 28, 33] .  \nPCA works as follows: Suppose we have a data matrix X ∈ RD􀀂N of N , D-dimensional observations. PCA works by linearly projecting our original D-dimensional data onto K uncorrelated (orthogonal) directions – the 􀀌rst K‘Principal Components’ – where typically K ≪ D. Denote by P ∈ RD􀀂K the matrix with the Principal Components as columns, then the Principal Components are chosen to maximize the orthogonal projection of the dataset X onto the column space of P, that is P is chosen to satisfy:  \nmPinkX − PPT X k22. (1)  \nAs a result, PCA gives the best K-dimensional representation of the original D-dimensional data in the least-squares sense or, equivalently, if the data are centered, using PCA means that (for a 􀀌xed dataset) we discard the smallest amount of the total sample variation of any linear dimensionality reduction scheme. For data analysis tasks other than Ordinary Least Squares Regression (OLS) using PCA is a heuristic which works well frequently but, as noted in [11, 20, 16, 12], it can work very poorly (even for a linear regression task) in some natural scenarios. For the task of classi􀀌cation, PCA often seems to work very well experimentally [13, 14], but it is trivial to construct examples for which PCA will work badly in a classi􀀌cation task1 . In other words since the PCA objective is disconnected from the classi􀀌cation task at hand and, in particular it does not take account of the class structure inherent in the problem, vanilla PCA is prone to under􀀌tting. In supervised settings where class labels are present, it would be a waste not to use the label information in the selection of useful features for classi􀀌cation, thus we propose  \n1 For example, when the most discriminative features have small","cbCaihWFtutjfxKM","https://ap.wps.com/l/cbCaihWFtutjfxKM","pdf",229242,1,28,"English","en",105,"# Background and Introduction\n## Dimensionality reduction in predictive modeling\n## PCA and its limitations for classification\n## Related supervised dimensionality reduction methods","[{\"question\":\"Why does the paper argue that PCA may be suboptimal for classification?\",\"answer\":\"Because PCA optimizes variance reconstruction for low-rank projection without using class structure, its objective is disconnected from classification. This can lead to underfitting and poor class separation in some scenarios.\"},{\"question\":\"What is the core idea of Maximum Margin Principal Components?\",\"answer\":\"It proposes a PCA variant that aims to minimize the difference in margin distribution between original and projected data. The goal is to better preserve classification-relevant margins when reducing dimensionality.\"},{\"question\":\"How does the proposed method perform compared with PCA and other approaches?\",\"answer\":\"Experiments show it typically outperforms PCA on classification error across datasets, though the improvement is not always statistically significant. The approach is also competitive with Partial Least Squares and Lasso, even though it remains a filter method.\"}]","Maximum Margin Principal Components | PDF",1788052692,71,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"maximum-margin-principal-components","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/maximum-margin-principal-components/160244/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-30",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why does the paper argue that PCA may be suboptimal for classification?","Question",{"text":75,"@type":76},"Because PCA optimizes variance reconstruction for low-rank projection without using class structure, its objective is disconnected from classification. This can lead to underfitting and poor class separation in some scenarios.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What is the core idea of Maximum Margin Principal Components?",{"text":80,"@type":76},"It proposes a PCA variant that aims to minimize the difference in margin distribution between original and projected data. The goal is to better preserve classification-relevant margins when reducing dimensionality.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the proposed method perform compared with PCA and other approaches?",{"text":84,"@type":76},"Experiments show it typically outperforms PCA on classification error across datasets, though the improvement is not always statistically significant. The approach is also competitive with Partial Least Squares and Lasso, even though it remains a filter method.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]