Applying Machine Learning Methods to Generate Understandings of Differential Item Functioning in a Flu Knowledge Assessment

Aug 3, 2026

Current influenza trends, including severity of the 2025 flu season and prevalence of H5 bird flu in livestock, require improved understanding of how to educate students about transmission. Although validated influenza knowledge assessments exist, their potential affective and demographic biases remain insufficiently evaluated. This work investigates differential item functioning (DIF) in four items about flu transmission using a flexible machine learning framework alongside traditional statistical methods. Results indicate the strongest DIF effects are affective, while demographic factors emerge more in random forest and neural networks.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages19

Reading Time48 min

AccessFree to download

Abstract

Introduction and Background

Influenza trends and educational need

Existing assessment instruments for flu knowledge

Differential item functioning (DIF) goals

Psychometric and modeling approaches

Computer Science and Engineering Faculty Publications Computer Science & Engineering 2025 Applying Machine Learning Methods to Generate Understandings of Differential Item Functioning in a Flu Knowledge Assessment William L. Romine Tanvi BanerjeeDerrick Cox Follow this and additional works at: https://corescholar.libraries.wright.edu/cse Part of the Animal Diseases Commons, Computer Sciences Commons, Engineering Commons, andthe Medical Education Commons This Article is brought to you for free and open access by Wright State University’s CORE Scholar. It has beenaccepted for inclusion in Computer Science and Engineering Faculty Publications by an authorized administrator ofCORE Scholar. For more information, please contact library-corescholar@wright.edu . # Applying Machine Learning Methods to Generate Understandings of Differential ItemFunctioning in a Flu Knowledge Assessment William Romine 1, Tanvi Banerjee2, Derrick Cox2 1Kairos Research, Dayton OH USA 2Department of Computer Science, Wright State University, Dayton OH USA ## Abstract Current influenza trends, includingthe severity of the 2025 flu season andthe prevalence of H5bird flu in livestock, necessitate efforts to better understand how to educate students about itstransmission. Although validated assessments of influenza knowledge exist, these have not beenevaluated for affective and demographic biases. We explore differential item functioning (DIF)effects in four items focused on specific aspects of flu transmission derived from a validatedinfluenza knowledge assessment. In doing so, we introduce and utilize a machine learningframework for exploration of DIF which offers greater flexibility than traditional statisticalapproaches in terms of studying generalizability of effects within and across study sites andacross different modeling approaches. Both statistical (logistic regression) and machinelearning approaches revealed that the largest DIF effects—perceived complications and barriersto preventative practice—were affective in nature. Demographic factors such as gender,ethnicity, and presence of health professionals in the students’ families, tended to emerge fromthe algorithmic models (random forest and neural networks), whereas the data models (likelogistic regression) tended to overlook these smaller effects. While not a direct replacement forstatistical approaches, we encourage researchers interested in understanding equity to treatmachine learning as an additional resource in our toolboxes. To better understand how to educatestudents about communicable diseases such as H5 bird flu, moving beyond model-specificinferential methods toward model-agnostic machine learning-based methods will enhance ourability to detect biases in our assessments, and to focus on those biases which persist acrossdifferent samples and diverse modeling paradigms. ## Introduction and Background Our current flu season has been the worst since the 2009 H1N1 swine flu pandemic (CDC,2025a) . This corresponds with record low rates of influenza vaccination, especially amongchildren (CDC, 2025b) . In addition, H5 bird flu has traveled across the globe via bird migrationand has spread to multiple domestic populations including poultry, cattle, and humans (CDC,2024) and has the potential to become a global pandemic much like the H1N1 swine flupandemic in 2009. As with swine flu andthe more recent coronavirus pandemic in 2019,effective educational programming is crucial to risk mitigation. Previous work has shown thatincreased knowledge of influenza in high school students led to reduced incidences of flu-likeillness through reduction in perceived barriers to taking preventative practices which in turnimproved handwashing quality (Romine, Folk, & Barrow, 2017) . If we areto reduce the spread of influenza through educational programming, we need tounderstand what knowledge students possess and what they are getting out of the instruction through reliable and

FAQs

What is the document’s main objective regarding flu knowledge assessments?

To evaluate differential item functioning (DIF) in four flu transmission items and identify whether affective and demographic biases exist in validated assessments.

Which methods are used to analyze DIF effects?

The study combines statistical approaches such as logistic regression with machine learning models including random forest and neural networks.

What kinds of factors show the largest DIF effects in the results?

The largest DIF effects relate to affective factors, such as perceived complications and barriers to preventive practice.

How should researchers use machine learning in fairness and equity investigations?

Treat machine learning as an additional resource rather than a direct replacement, and use model-agnostic approaches to detect biases that persist across samples and modeling paradigms.

Aug 3, 2026

Upload by Seraphina