Evaluating AI Literacy of Secondary Students Framework and Scale

Jul 18, 2026

Upload by Asher

AI literacy (AIL) has become essential as AI tools such as ChatGPT become widely accessible and pervasive. Building on prior work that defines AIL as critical evaluation, effective communication/collaboration with AI, and responsible use across contexts, this research argues for a shift from “AI in education” to “education about AI” beginning in K-12. The study focuses on secondary students, proposes an AIL framework centered on high-order thinking and dimensional connections, develops a reliable and valid measurement instrument, and investigates gender differences in students’ AIL.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages22

Reading Time55 min

AccessFree to download

Introduction

Rationale for AI literacy in K-12

Existing AIL assessment limitations

Study aims and research questions

Computers & Education 227 (2025) 105230 Contents lists available at ScienceDirect Computers & Education journal [homepage: www.elsevier.com/locate/compedu](homepage: www.elsevier.com/locate/compedu) | Evaluating AI literacy of secondary students: Framework and scale development
Baichang Zhong , Xiaofan Liu *
School of Information Technology in Education, South China Normal University, No.55 West Zhongshan Avenue, Guangzhou, 510631, China | | | | --- | --- | --- | | A R T I C L E I N F O | | A B S T R A C T | | Keywords:
Evaluation methodologies Information literacy Secondary education 21st century abilities | | K-12 AI education not only equips students with AI literacy but also encourages underrepresented groups to pursue further studies or careers in this field. Secondary students were particularly wellsuited for comprehensive AI education due to their cognitive characteristics and developmental readiness. While most studies have focused on developing pedagogy, curriculum, and tools for secondary AI education, they have prioritized measuring students’ learning outcomes over literacy development. Referring to the empirical research on secondary AI education as well as Piaget’s Epistemology and Bloom’s Taxonomy, this study figured out a KAT framework that constitutes AI literacy: (1) AI Knowledge (AI fundamentals, elements of AI technology, application of AI technology),(2) AI Affectivity (AI and human, AI and society), and (3) AI Thinking (engineering design thinking, computational thinking). Based on this, a 57-item AI literacy scale (AILS) was developed, and 56 items were retained after expert judgement. Then, a large sample of Chinese secondary students was surveyed, resulting in 1392 valid samples, which were randomly divided into two sub-samples: 720 samples were used for item reduction through Rasch Analysis and Exploratory Factor Analysis; 672 samples were used for model validation and comparison through Confirmatory Factor Analysis. Results indicated the AILS with three-factor structure of 48 items has a good reliability and validity. Moreover, gender differences in AI literacy among secondary students were examined. Results indicated that boys had significantly higher AI Knowledge than girls, whereas girls had significantly higher AI Affectivity than boys. The implications, limitations and future research were also discussed. | 1. Introduction Nowadays, the rapid development of Artificial Intelligence (AI) like ChatGPT have provided free and low-threshold access to powerful AI applications for everyone (Yilmaz et al., 2023). The ubiquity of AI applications requires future generations to acquire sufficient AI literacy (AIL) to most effectively use such technologies (Chiu, 2021; Long & Magerko, 2020). Although scholars first proposed AIL in the 1970s, the focus was on professional literacy about AI (Agre, 1972). Until 2016, AIL emerged as a key component of 21st century abilities, which refers to the general literacy

FAQs

Why is AI literacy important for future generations?

The widespread availability of AI applications requires students to develop sufficient AI literacy to use these technologies effectively and responsibly in everyday and learning contexts.

What problem does the study address in current AI education and assessment?

Existing research often evaluates related aspects such as concepts, attitudes, ethics, or creativity without directly assessing AIL, and frameworks may lack emphasis on higher cognitive skills and dimensional connections.

What does this study aim to do for secondary students?

It proposes a new AIL framework emphasizing high-order thinking and dimensional connections, develops a reliable and valid instrument to evaluate secondary students’ AIL, and examines gender differences in AIL.

Jul 18, 2026

Upload by Asher