The role of hyperparameters in machine learning models and how to tune them - Research Note

Aug 2, 2026

Upload by Ezra

Hyperparameters critically influence how well machine learning models perform on unseen, out-of-sample data. The study systematically compares different hyperparameter settings and evaluates how often hyperparameters and their tuning procedures are reported in political science research. Analysis of 64 manuscripts from three leading journals (2016–2021) shows that only 13 publications report both the hyperparameters and how they tuned them. The findings highlight risks of insufficient transparency and recommend making hyperparameter tuning and documentation a standard robustness-check component.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

AccessFree to download

Why care about hyperparameters?

Impact on generalization and research conclusions

Evidence from political science journal manuscripts

Documenting tuning as robustness practice

Replication code and transparency concerns

Boston University OpenBU [http://open. bu.edu](http://open. bu.edu) BU Open Access Articles BU Open Access Articles 2024-02-05 The role of hyperparameters in machine learning models and how to tune them This work was made openly accessible by BU Faculty. Please share how this access benefits you. Your story matters. | Version | Published version | | --- | --- | | Citation (published version): | Arnold C, Biedebach L, Küpfer A, Neunhoeffer M. The role of hyperparameters in machine learning models and how to tune them. Political Science Research and Methods. Published online 2024:1-8 .
doi:10.1017/psrm.2023.61 | [https://hdl.handle.net/2144/49373](https://hdl.handle.net/2144/49373)[ ](https://hdl.handle.net/2144/49373)Boston University [https://doi.org/10.1017/psrm.2023.61](https://doi.org/10.1017/psrm.2023.61) Published online by Cambridge University Press Political Science Research and Methods (2024), page 1 of 8 doi:10.1017/psrm.2023.61 RESEARCH NOTE The role of hyperparameters in machine learning models and how to tune them Christian Arnold 1 , Luka Biedebach2, Andreas Küpfer3 and Marcel Neunhoeffer4,5 1Department of Politics and International Relations, Cardiff University, Cardiff, UK, 2Department of Computer Science, Reykjavik University, Reykjavik, Iceland, 3Institute for Political Science, Technical University of Darmstadt, Darmstadt, Germany, 4Rafik B. Hariri Institute for Computing and Computational Science & Engineering, Boston University, Boston, MA, USA and 5Department of Statistics, LMU Munich, Munich, Germany Corresponding author: Marcel Neunhoeffer; [Email: marcel@marcel-neunhoeffer.com](Email: marcel@marcel-neunhoeffer.com)[ ](Email: marcel@marcel-neunhoeffer.com)(Received 25 January 2022; revised 30 June 2023; accepted 5 July 2023) Abstract Hyperparameters critically influence how well machine learning models perform on unseen, out-of-sample data. Systematically comparing the performance of different hyperparameter settings will often go along way in building confidence about a model’s performance. However, analyzing 64 machine learning related manuscripts published in three leading political science journals (APSR, PA, and PSRM) between 2016 and 2021, we find that only 13 publications (20.31 percent) report the hyperparameters and also how they tuned them in either the paper or the appendix. We illustrate the dangers of cursory attention to model and tuning transparency in comparing machine learning models’ capability to predict electoral violence from tweets. The tuning of hyperparameters and their documentation should become a standard component of robustness checks for machine learning models. Keywords: Best Practice; Hyperparameter Optimization; Machine Learning 1 Why care about hyperparameters? When political scientists work with machine learning models, they want to find a model that generalizes well from training data to new, unseen data.1 Hyperparameters play a key role in this endeavor because they determine the models’ capacity to generalize. Finding a good set of hyperparameters critically affects conclusions about a model’s performance. The failure to correctly tune and report hyperparameters has recently been identified as a key impediment to the accumulation of knowledge in computer science (e.g. Henderson et al., 2018; Melis et al., 2018; Bouthillier et al., 2019, 2021; Cooper et al., 2021; Gundersen et al., 2023) . Is political science making the same mistake? We examined 64 machine learning-related papers published between 1 January 2016 and 20 October 2021 in some of the top journals of our discipline—the American Political Science Review (APSR), Political Analysis (PA), and Political Science Research and Methods (PSRM) . Of the 64 publications we analyzed, 36 (56.25 percent) do not report the values of their hyperparameters, neither in the paper nor the appendix. Forty-nine publications (76.56 percent) do not share information about how they u

FAQs

Why are hyperparameters important in machine learning models?

Hyperparameters shape how well a model generalizes from training data to unseen data by determining the model’s capacity. Correct tuning is therefore central to reliable performance-based conclusions.

How often do political science papers report hyperparameters and tuning procedures?

Among 64 analyzed machine-learning-related publications (2016–2021), only 13 (20.31%) reported the hyperparameters and how they tuned them. Many also omitted either hyperparameter values or tuning information.

What risks arise from limited transparency about tuning?

When tuning choices are not documented, readers and reviewers cannot assess manuscript quality and cannot meaningfully evaluate or replicate the results. The paper argues this should be treated as a robustness-check issue.

Aug 2, 2026

Upload by Ezra