[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-120157-en":3,"doc-seo-120157-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":27,"seo_description":14,"update_tm":28,"read_time":29},120157,8796095461610,"Oliver","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Extending the Scope of Inference About Predictive Ability to Machine Learning Methods - arXiv","Modern machine-learning forecasts are widely evaluated with out-of-sample procedures, but classic inference theory for predictive ability does not transfer directly to high-dimensional learners such as Lasso. The study identifies conditions under which asymptotic out-of-sample inference remains valid, focusing on two requirements: a zero-mean score of the prediction loss and fast convergence of the machine learner. Monte Carlo simulations support the theory, and an out-of-sample test is proposed to evaluate the Martingale Difference Hypothesis for exchange rates at multiple frequencies.","Extending the Scope of Inference About Predictive Ability to Machine Learning Methods  \narXiv :2402 . 12838v2 [ econ .EM] 16 Apr 2024  \nJuan Carlos Escanciano∗ Universidad Carlos III de Madrid  \nRicardo Parra  \nMarket Risk Boadilla, Banco Santander CIB April 15, 2024  \nAbstract  \nThough out-of-sample forecast evaluation is systematically employed with modern machine learning methods and there exists a well established classic inference theory for predictive ability, see, e.g. , West (1996, Asymptotic Inference About Predictive Ability, Econometrica, 64, 1067-1084), such theory is not directly applicable to modern machine learners such as the Lasso in the high dimensional setting. We investigate under which conditions such extensions are possible. Two key properties for standard out-of-sample asymptotic inference to be valid with machine learning are (i) a zero-mean condition for the score of the prediction loss function; and (ii) a “fast rate” of convergence for the machine learner. Monte Carlo simulations confirm our theoretical findings. We recommend a small out-of-sample vs in-sample size ratio for accurate finite sample inferences with machine learning. We illustrate the wide applicability of our results with a new out-of-sample test for the Martingale Difference Hypothesis (MDH) . We obtain the asymptotic null distribution of our test and use it to evaluate the MDH of some major exchange rates at daily and higher frequencies.  \n∗ Corresponding author: Department of Economics, Universidad Carlos III de Madrid. Calle Madrid 126, Getafe, 28907, Spain. E-mail: jescanci@eco.uc3m.es. Web Page: [https://sites.google.com/view/juancarlosescanciano. Research](https://sites.google.com/view/juancarlosescanciano. Research)[ ](https://sites.google.com/view/juancarlosescanciano. Research)[funded by the Spanish Grant PID2021-127794NB-I00](funded by the Spanish Grant PID2021-127794NB-I00) , [MCIN/AEI /10.13039/501100011033-CEX2021-001181-M](MCIN/AEI /10.13039/501100011033-CEX2021-001181-M), [CAM](CAM)[ ](CAM)[grants EPUC3M11](grants EPUC3M11) ([V PRICIT](V PRICIT)) [and Mad-Econ-Pol-CM H2019/HUM-5891](and Mad-Econ-Pol-CM H2019/HUM-5891) .  \n1 Introduction  \nOut-of-sample evaluation of predictive models has become a key feature of modern machine learning methods (see, e.g., James, Witten, Hastie, Tibshirani and Taylor, 2023, for a textbook treatment, and Mullainathan and Spiess, 2017, for an applied econometrics perspective) . It also has a long tradition in time series econometrics, with important contributions by Diebold and Mariano (1995) and West (1996) . These existing asymptotic justifications for predictive inference provide a formal framework for prediction uncertainty quantification. However, these classical results do not directly carry over to modern machine learning applications.1 In this paper, we point out two key properties for making standard inference on predictive performance asymptotically valid with machine learning methods: (i) a zero-mean condition for the score of the prediction loss function; and (ii) a “fast rate” of convergence for the machine learner.  \nThe zero-mean condition of the score of the prediction loss function holds in a number of important examples, including the leading Mean Squared Prediction Error (MSPE) loss function with the Lasso estimator. This zero-mean property holds when the prediction loss function and the point forecast are suitably matched, as defined and recommended in Gneiting (2011) . The second property—the fast rates of convergence for the machine learners—is satisfied under suitable identifying conditions and uniform deviation bounds for the score of the estimation loss function. Checking these conditions for time series is not trivial. We exploit recent advances in Wong, Li and Tewari (2020) for checking these conditions in our predictive ability exercise for β-mixing processes with geometrically decaying coefficients and sub-Weibull marginals. As the main result of this paper, we ","cbCaituSxCJscMkf","https://ap.wps.com/l/cbCaituSxCJscMkf","pdf",494938,1,31,"English","en",105,"# Introduction\n## Key properties for valid predictive inference\n## Zero-mean condition\n## Fast-rate convergence conditions\n# Main implications and contributions\n## Standard inference without estimation-risk correction\n## Proposed out-of-sample test\n# Theoretical results and simulations\n## Monte Carlo validation\n## Application to exchange rates","[{\"question\":\"Why do classic predictive-ability inference results not directly apply to modern machine learning methods?\",\"answer\":\"Classic asymptotic justifications require conditions that can fail for modern learners, especially in high-dimensional settings. The paper specifies two properties needed to extend inference validity to such methods.\"},{\"question\":\"What are the two main conditions ensuring valid asymptotic out-of-sample inference with machine learning?\",\"answer\":\"The score of the prediction loss must satisfy a zero-mean condition, and the machine learner must converge at a fast rate. Together, these make the estimation-risk component asymptotically negligible.\"},{\"question\":\"How does the proposed framework affect confidence intervals for predictive risk?\",\"answer\":\"With the stated conditions, an approximate 95% asymptotic confidence interval keeps the usual form: out-of-sample empirical risk ± two times the standard error, without correction for estimation risk.\"}]","Extending the Scope of Inference About Predictive Ability to Machine Learning Methods - arXiv | PDF",1785728481,78,{"code":4,"msg":31,"data":32},"ok",{"site_id":24,"language":23,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"extending-the-scope-of-inference-about-predictive-ability-to-machine-learning-methods-arxiv","",{"@graph":36,"@context":85},[37,54,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/extending-the-scope-of-inference-about-predictive-ability-to-machine-learning-methods-arxiv/120157/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":23,"description":14,"dateModified":62,"datePublished":62,"encodingFormat":61,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-08-03",true,{"@type":65,"interactionType":66,"userInteractionCount":4},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"Why do classic predictive-ability inference results not directly apply to modern machine learning methods?","Question",{"text":75,"@type":76},"Classic asymptotic justifications require conditions that can fail for modern learners, especially in high-dimensional settings. The paper specifies two properties needed to extend inference validity to such methods.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"What are the two main conditions ensuring valid asymptotic out-of-sample inference with machine learning?",{"text":80,"@type":76},"The score of the prediction loss must satisfy a zero-mean condition, and the machine learner must converge at a fast rate. Together, these make the estimation-risk component asymptotically negligible.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the proposed framework affect confidence intervals for predictive risk?",{"text":84,"@type":76},"With the stated conditions, an approximate 95% asymptotic confidence interval keeps the usual form: out-of-sample empirical risk ± two times the standard error, without correction for estimation risk.","https://schema.org",{"og:url":52,"og:type":87,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":89,"canonical":52},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":46,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":46,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":46,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":46,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]