Evaluating 21st-Century Competencies in Postsecondary Curricula with Large Language Models - Performance Benchmarking and Reasoning-Based Prompting Strategies

Oct 17, 2026

Uploaded by Sarah

The study addresses how 21st-century competencies in postsecondary education are embedded in curricula and whether academic programs align with evolving workforce and societal demands. It uses curricular analytics to evaluate different curriculum document sources and benchmarks general-purpose large language models on mapping curricula to competencies. With 7,600 manually annotated alignment scores across multiple competencies and course types, it analyzes error patterns and introduces reasoning-based curricular chain-of-thought prompting to improve pedagogical inference reliability.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages30

Reading Time1 h 16 min

AccessFree to download

Abstract

Notes for Practice

Keywords

Introduction

Global labor market and GenAI impact

Shift toward transferable competencies

Volume 13(1), 7–30. [https://doi.org//10.18608/jla.2026.9127](https://doi.org//10.18608/jla.2026.9127) Evaluating 21st-Century Competencies in Postsecondary Curricula with Large Language Models: Performance Benchmarking and Reasoning-Based Prompting Strategies Zhen Xu 1 , Xin Guan2†, Chenxi Shi3†, Qinhao Chen4 , Renzhe Yu5* Abstract The growing emphasis on 21st-century competencies in postsecondary education, intensified by the transformative impact of generative artificial intelligence (GenAI) on the economy and society, underscores the urgent need to evaluate how they are embedded in curricula and how effectively academic programs align with evolving workforce and societal demands. Curricular analytics, particularly recent advancements powered by GenAI, offer a promising data-driven approach to this challenge. However, the analysis of 21st-century competencies requires pedagogical reasoning beyond surface-level information retrieval, and the capabilities of large language models (LLMs) in this context remain underexplored. In this study, we extend prior research on curricular analytics of 21st-century competencies across a broader range of curriculum documents, competency frameworks, and models. Using 7,600 manually annotated curriculum-competency alignment scores (38 competencies and 200 courses across five curriculum document types), we evaluate the informativeness of different curriculum document sources, benchmark the performance of general-purpose LLMs on mapping curricula to competencies, and analyze error patterns. We further introduce a reasoning-based prompting strategy, curricular chain-of-thought (CoT), to strengthen LLMs’pedagogical reasoning. Our results show that detailed instructional activity descriptions are the most informative typeof curriculum document for competency analytics. Open-weight LLMs achieve accuracy comparable to proprietary models on coarse-grained tasks, demonstrating their scalability and cost-effectiveness for institutional use. However, no model reaches human-level precision in fine-grained pedagogical reasoning. Our proposed curricular CoT yields modest improvements by reducing bias in instructional keyword inference and improving the detection of nuanced pedagogical evidence in long text. Together, these findings highlight the untapped potential of institutional curriculum documents and provide an empirical foundation for advancing AI-driven curricular analytics. |
Notes for Practice
• Existing curriculum documents in digital formats are a promising yet underutilized resource for postsecondary institutions to assess the integration of 21st-century competencies in course design. Institutions may benefit the most from combining multiple data sources to ensure adequate coverage of key pedagogical elements.
• Large language models (LLMs) perform reasonably well in coarse-grained competency analytics, with openweight models achieving performance comparable to proprietary ones. However, none of the models reach human-level precision, with two common error patterns: biased inference from instructional keywords in short texts and difficulty identifying nuanced pedagogical evidence in long, unstructured documents.
• The proposed curricular chain-of-thought (CoT) prompting yields modest performance gains and reduces hallucination by guiding LLMs to extract key instructional elements prior to reasoning, improving the reliability of pedagogical inference, particularly for more advanced models. | | --- | | Keywords
Curricular analytics, 21st-century competency, large language models (LLMs), prompt engineering, chain of thought (CoT) .
Submitted: 09/02/2025 —Accepted: 22/12/2025 —Published: 25/02/2026 | 1 Email: [zx2393@tc.columbia.edu](zx2393@tc.columbia.edu) Address: Teachers College, Columbia University, New York, New York, United States. ORCID iD: [https:// orcid.org/ 0009-0004-3131-910X](https:// orcid.org/ 0009-0004-3131-910X) 2 Email: [xg2413@tc.columbia.edu](x

FAQs

What problem does the paper focus on regarding 21st-century competencies?

It evaluates how 21st-century competencies are embedded in postsecondary curricula and how effectively programs align with workforce and societal needs amid the impact of generative AI.

How is the evaluation conducted in the study?

The study uses 7,600 manually annotated curriculum-competency alignment scores across 38 competencies, 200 courses, and five curriculum document types, then benchmarks LLM performance for curriculum-to-competency mapping.

What is the proposed contribution beyond benchmarking?

It introduces a reasoning-based prompting strategy, curricular chain-of-thought (CoT), to strengthen LLM pedagogical reasoning and reduce bias in instructional keyword inference and improve evidence detection in long texts.

Oct 17, 2026

Uploaded by Sarah