A Meta-Analysis of the Impact of Generative Artificial Intelligence on Learning Outcomes

Q: What does the meta-analysis evaluate about generative artificial intelligence in education?

It evaluates how Generative Artificial Intelligence affects student learning outcomes across cognitive, competency, and affective dimensions, and whether discipline, duration, knowledge type, prior knowledge, and tool type change the effect.

Q: What is the overall impact of GAI on learning outcomes, according to the results?

GAI shows a significant positive overall effect on learning outcomes with a combined effect size of 0.68 (p<0.001).

Q: Which factors significantly moderate GAI effectiveness, and which do not?

Effectiveness varies by discipline type, showing the highest positive effects in mathematics, science, and humanities and lower but still significant effects in computer science and medical/nursing education. Instructional period, knowledge type, prior knowledge level, and tool type are not significantly associated with differences in the impact in the main moderation analyses.

Jul 8, 2026

Upload by Finn

Rapid technological advances have driven growing attention to integrating Generative Artificial Intelligence (GAI) in education, yet prior findings about learning effects remain inconsistent. This study meta-analyzes GAI’s impact on student learning outcomes across cognitive, competency, and affective dimensions and examines moderating factors, including discipline type, instructional duration, knowledge type, prior knowledge, and tool type. Results from 34 experimental and quasi-experimental studies show a significant overall positive effect (g=0.68, p<0.001), strongest in cognitive and competency outcomes. Moderation analyses indicate discipline differences, with other factors showing no significant effects. Practical conclusions emphasize matching GAI to subject needs, adapting tools by learner level, integrating with traditional teaching, and strengthening monitoring.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages21

Reading Time53 min

Introduction

Background and motivation

Summary of known findings

Controversies in learning outcomes

Where effects may vary

What this paper adds

Overall learning effects

Dimension-specific effects

Moderating factors and tool comparisons

Implications for practice and policy

Journal of Computer Assisted Learning REVIEW ARTICLE A Meta-Analysis of the Impact of Generative Artificial Intelligence on Learning Outcomes Nan Ma | Zhiyong Zhong School of Education, Minzu University of China, Beijing, China Correspondence: Zhiyong Zhong ([zzylsq@hotmail.com](zzylsq@hotmail.com)) Received: 6 November 2024 | Revised: 16 June 2025 | Accepted: 24 August 2025 Keywords: generative artificial intelligence | learning outcomes | meta-analysis ABSTRACT Background: With the rapid advancement of technology, the integration of Generative Artificial Intelligence (GAI) in education has gained considerable attention. Many studies have examined GAI's impact on learning outcomes, yet their conclusions are inconsistent, highlighting the need for a comprehensive review to clarify its overall effects and identify influential factors. Objectives: This study aims to conduct a meta-analysis of the effects of GAI on student learning outcomes across cognitive, competency and affective dimensions. Additionally, it seeks to explore how various moderating factors, including subject discipline, instructional duration, knowledge type, prior knowledge and tool type, influence GAI's effectiveness. Methods: A meta-analysis was performed on 34 experimental and quasi-experimental studies published internationally. Effect sizes were calculated for overall learning outcomes and categorised by dimension. Further analysis was conducted to assess the influence of moderating variables on the impact of GAI. Results: The meta-analysis indicates that Generative Artificial Intelligence has a significant positive impact on overall learning outcomes, with a combined effect size of 0.68 (p < 0.001). The impact is particularly pronounced in the cognitive dimension (g = 0.795) and the competency dimension (g = 0.711), while its effect on the affective dimension (g = 0.507) is moderate but still significant. The analysis of moderating variables reveals that the effectiveness of GAI is influenced by discipline type but is not significantly affected by instructional period, knowledge type, prior knowledge level, or tool type. Specifically, GAI exhibits the highest positive effects in mathematics, science and humanities, whereas its impact is relatively lower yet still significant in computer science and medical/nursing education. Additionally, GAI's effectiveness does not significantly differ across various instructional periods, different knowledge types, learners with varying prior knowledge levels, or different AI tool versions. Conclusions: To optimise GAI's use in education, the study suggests aligning GAI with specific subject needs, adapting tools for different student levels, integrating GAI with traditional teaching and establishing monitoring mechanisms. These strategies aim to maximise GAI's positive impact on learning efficiency and quality across educational settings. 1 | Introduction With regard to rapid development in education technology, generative AI has increasingly found its way into the education sector (O'Dea 2024) . Therefore, teachers need to pay more attention toward using generative AI in an efficacious manner for improving learning performance and abilities among students (Kurtz et al. 2024) . Generative artificial intelligence is a subgroup of AI technology that is founded upon deep learning and neural network-based technology, which serves mainly for creating new content—be that in textual, image, or other forms—that highly resembles human-made content after learning from vast amounts of training data (Kaffeeet al. 2022) . These tools have potential for intelligent analysis © 2025 John Wiley & Sons Ltd. Journal of Computer Assisted Learning, 2025; 41:e70117 1 of 21 [https://doi.org/10.1111/jcal.70117](https://doi.org/10.1111/jcal.70117) Summary • What is already known about this topic ○ The application of Generative Artificial Intelligence (GAI) in education is growing, but its impact on lear

FAQs

What does the meta-analysis evaluate about generative artificial intelligence in education?

It evaluates how Generative Artificial Intelligence affects student learning outcomes across cognitive, competency, and affective dimensions, and whether discipline, duration, knowledge type, prior knowledge, and tool type change the effect.

What is the overall impact of GAI on learning outcomes, according to the results?

GAI shows a significant positive overall effect on learning outcomes with a combined effect size of 0.68 (p<0.001).

Which factors significantly moderate GAI effectiveness, and which do not?

Effectiveness varies by discipline type, showing the highest positive effects in mathematics, science, and humanities and lower but still significant effects in computer science and medical/nursing education. Instructional period, knowledge type, prior knowledge level, and tool type are not significantly associated with differences in the impact in the main moderation analyses.

Jul 8, 2026

Upload by Finn