cv - Cross-Validating Regression Models in R - Journal of Statistical Software Volume 116 Issue 8

Sep 5, 2026

Upload by Xiajie

The cv package implements cross-validation for standard R regression models using a uniform, generic function interface. It provides methods for many commonly used model classes, including mixed-effects models, and can be extended by adding methods for additional classes. The cv() function also cross-validates complex model-selection procedures involving variable transformations, predictor selection, and comparisons among competing models. Supplied methods support parallel computations, and linear and generalized linear model methods exploit computational efficiencies.

LanguageEnglish

CategoryPresentations

FormatPDF

Pages41

Reading Time14 min

AccessFree to download

Introduction

Cross-validation motivation and data-division

cv: Cross-Validating Regression Models in R

k-fold and leave-one-out cross-validation

Bias and variance considerations

Using cv() in R

Journal of Statistical Software May 2026, Volume 116, Issue 8 . doi: 10.18637/jss.v116.i08 cv: An R Package for Cross-Validating Regression Models John Fox McMaster University Georges Monette York University Abstract We describe the cv package, which implements cross-validation for standard R regression models through a uniform and simple to use generic function. Methods for the cv() function are provided for many commonly employed classes of statistical models, including mixed-effects models, and it is straightforward to extend the cv package by writing methods for additional classes of models. The cv() function can also cross-validate complex model-selection procedures, such as those that include variable transformations, predictor selection, and selection among competing models. The cv() function generally supports parallel computations, and the supplied methods for linear and generalized linear models take advantage of computational efficiencies. Keywords: cross-validation, regression analysis, model selection, R. 1 . Introduction Cross-validation (CV) is an essentially simple and intuitively reasonable approach to estimating the predictive accuracy of regression models. CV is developed in many standard sourceson regression modeling and “machine learning” – we particularly recommend James, Witten, Hastie, and Tibshirani (2021, Sections 5.1, 5.3)– and so we will describe the method only briefly here before taking up computational issues and some examples. See Arlot and Celisse (2010) for a wide-ranging, if technical, survey of cross-validation and related methods that emphasizes the statistical properties of CV. Validating research by replication on independently collected data is a common scientific norm. Emulating this process in a single study by data-division is less common (see, e.g. , Barnard 1974): The data are randomly divided into two, possibly equal-size, parts; the first part is used to develop and fit (in the language of machine leaning, “train”) a statistical model; and then the second part is used to assess the adequacy of (“validate”) the model fit 2 cv: Cross-Validating Regression Models in R to the first part of the data. Data-division, however, suffers from two problems: (1) Dividing the data decreases the sample size and thus increases sampling error; and (2), even more disconcertingly, the results can vary substantially based on the random division of the data, particularly in smaller samples. See Harrell Jr. (2015, Section 5.3) for cogent remarks about data-division and cross-validation. Cross-validation speaks to both of these issues. In CV, the data are randomly divided as equally as possible into several, say k, parts, called “folds.” The statistical model is fit k times, leaving each fold out in turn. Each fitted model is then used to predict the response variable for the cases in the omitted fold. A CV criterion (also termed a “cost” or “loss”measure), such as the mean-squared error (MSE) of prediction, is then computed using these predicted values. In the extreme, k equals n, the number of cases in the data, thus omitting individual cases and refitting the model n times – a procedure termed “leave-one-out (LOO) cross-validation.” Because the n models are each fit to n−1 cases, LOO CV produces a nearly unbiased estimate of the prediction error compared with a model fit to n cases. The n regression models are highly statistically dependent, however, based as they are on nearly the same data, and so the resulting estimate of prediction error has larger variance than if the predictions from the models fit to the n data sets were independent. Because predictions are based on smaller data sets, each of size approximately n − n/k , estimated prediction error for k-fold CV with k = 5 or 10 (commonly employed choices) is more biased than estimated prediction error for LOO CV. It is possible, however, to correct k-fold CV for bias (see Appendix A.2) . The relative varia

FAQs

What problem does cross-validation address in regression modeling?

Cross-validation estimates predictive accuracy by validating models on held-out data partitions. It helps assess adequacy while highlighting issues related to sample size reduction and randomness from data division.

How does k-fold cross-validation differ from leave-one-out (LOO) cross-validation?

In k-fold CV, the data are split into k folds and the model is fit k times, each time leaving one fold out for prediction. In LOO CV, k equals the number of cases, omitting one case at a time and refitting the model n times.

What capabilities does the cv package provide for cross-validation in R?

The cv package automates cross-validation for standard R regression model classes through the cv() function. It includes methods for linear, generalized linear, mixed-effects models, model lists, and for cross-validating complex model-selection processes, with support for parallel computations in many cases.

Sep 5, 2026

Upload by Xiajie