[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-121590-en":3,"doc-seo-121590-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},121590,7971461741311,"Ophelia","https://ap-avatar.wpscdn.com/avatar/74000253aff267980c6?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779345379180704826",8,"Research & Report","Hunting Tomorrow's Leaders - Using Machine Learning to Forecast S&P 500 Additions & Removal","This study applies machine learning to forecast S&P 500 membership changes, events that strongly influence investor behavior and market dynamics. Quarterly firm data from WRDS (2013 onward) is assembled using industry classification, financial fundamentals, market variables, and corporate governance indicators. A Random Forest model achieves a test F1-score of 0.85, outperforming logistic regression and SVC. SHAP-based interpretability and feature engineering support transparent modeling. Predicted Q3 2023 additions and removals are translated into a trading strategy to capture alpha and improve index-dynamics decisions.","Hunting Tomorrow's Leaders: Using Machine Learning to Forecast S&P 500 Additions & Removal  \nVidhi Agrawal1, Eesha Khalid2, Tianyu Tan3, Doris Xu4  \nColumbia University  \nNew York, NY, United States  \n[va2504@columbia.edu](va2504@columbia.edu1)[1](va2504@columbia.edu1), [ek3365@columbia.edu](ek3365@columbia.edu2)[2](ek3365@columbia.edu2), [tt2976@columbia.edu](tt2976@columbia.edu3)[3](tt2976@columbia.edu3), [jx2577@columbia.edu](jx2577@columbia.edu4)[4](jx2577@columbia.edu4)  \nAbstract— This study applies machine learning to predict S&P 500 membership changes—key events that profoundly impact investor behavior and market dynamics. Quarterly data from WRDS datasets (2013 onwards) was used, incorporating features such as industry classification, financial data, market data, and corporate governance indicators. Using a Random Forest model, we achieved a test F1-score of 0.85, outperforming logistic regression and SVC models. This research not only showcases the power of machine learning for financial forecasting but also emphasizes model transparency through SHAP analysis and feature engineering. The model’s real-world applicability is demonstrated with predicted changes for Q3 2023, such as the addition of Uber (UBER) and the removal of SolarEdge Technologies (SEDG). By incorporating these predictions into a trading strategy—buying stocks announced for addition and shorting those marked for removal—we anticipate capturing alpha and enhancing investment decision-making, offering valuable insights into index dynamics.  \nKeywords—S&P 500, Index Inclusion, Prediction, Machine Learning, Random Forest, SVC, Logistic Regression, SHAP  \nI. INTRODUCTION  \nThe S&P 500 index, a critical benchmark for the U. S. equity market, significantly influences investor behavior and portfolio strategies. Passive investing, which tracks indices like the S&P 500, accounts for 20-30% of the value of U. S. equities, amplifying the importance of changes to the index. When a company is added to the index, its stock price typically rises due to anticipated demand from index fund managers and speculative trading [8] . For instance, Tesla's inclusion in the S&P 500 in December 2020 led to extraordinary trading activity and price performance, underscoring the significance of such changes. Conversely, deletions often lead to price declines as portfolios are adjusted to reflect the updated index composition.  \nBetween December 13, 2019, and September 24, 2024, we analyzed the effects of S&P 500 additions and deletions using announcements from S&P's website and data from CRSP. Our findings show that stocks added to the index experienced  \nsignificant price surges, while those removed faced declines, reflecting the predictable impact of index fund adjustments [1] . However, outliers like Ingersoll-Rand (IR), which underwent a ticker change to TT after a spinoff, and Apartment Investment and Management Co. (AIV), affected by an unadjusted stock split, introduced distortions. Excluding these outliers provided a clearer understanding of the systematic impacts of index changes.  \nThe 1-day, 2-day, and 7-day price movements following S&P 500 announcements (excluding IR and AIV) highlight clear alpha capture opportunities.  \nTABLE I. STATISTICS FOR ADDITION AND REMOVAL ACTIONS  \nStocks announced for addition saw significant price increases due to higher demand, supporting a long strategy, while those marked for removal declined due to reduced demand and selling pressure, favoring a short strategy. Building on these insights, we turn to machine learning to predict future S&P 500 additions and deletions [2], [10] . By leveraging historical patterns and financial indicators, we aim to create models that identify potential changes early, allowing investors to capture alpha with greater precision.  \nII. DATA  \nA. Data Sources and Extraction Process  \nThe dataset used for this study was constructed by combining data from multiple sources, specifically the Wharton Research Data","cbCaihee679cpf2k","https://ap.wps.com/l/cbCaihee679cpf2k","pdf",1491539,1,6,"English","en",105,"# Introduction\n## Impact of S&P 500 additions and deletions\n## Empirical observations and outlier handling\n## Motivation for machine learning prediction\n# Data\n## Data sources and extraction process\n## CRSP data\n## Compustat fundamentals\n## IBES data\n## Audit Analytics\n## S&P 500 membership data","[{\"question\":\"What does the study aim to predict about the S\\u0026P 500?\",\"answer\":\"It predicts membership changes—additions and deletions—that affect investor behavior and market dynamics.\"},{\"question\":\"Which machine learning model performs best in the research?\",\"answer\":\"A Random Forest model achieves a test F1-score of 0.85, outperforming logistic regression and SVC models.\"},{\"question\":\"How is interpretability handled in the modeling approach?\",\"answer\":\"The study emphasizes transparency using SHAP analysis alongside feature engineering.\"}]","Hunting Tomorrow's Leaders - Using Machine Learning to Forecast S&P 500 Additions & Removal | PDF",1785736382,15,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"hunting-tomorrows-leaders-using-machine-learning-to-forecast-sp-500-additions-removal","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/hunting-tomorrows-leaders-using-machine-learning-to-forecast-sp-500-additions-removal/121590/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What does the study aim to predict about the S&P 500?","Question",{"text":75,"@type":76},"It predicts membership changes—additions and deletions—that affect investor behavior and market dynamics.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Which machine learning model performs best in the research?",{"text":80,"@type":76},"A Random Forest model achieves a test F1-score of 0.85, outperforming logistic regression and SVC models.",{"name":82,"@type":73,"acceptedAnswer":83},"How is interpretability handled in the modeling approach?",{"text":84,"@type":76},"The study emphasizes transparency using SHAP analysis alongside feature engineering.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,114,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":21,"doc_module":4,"doc_module_name":46,"category_name":111,"show_sort_weight":112,"slug":113},"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]