AIriskEval-edu-db2: 新的K–12教育解释风险评估数据集

Jul 16, 2026

AIriskEval-edu-db2 introduces a new dataset for training and evaluating auditors based on LLMs to perform explainable pedagogical risk assessment of K–12 instructional explanations. The dataset contains 1,639 explanations generated from 170 curated ScienceQA questions spanning science, language arts, and social sciences. Each question pairs a human teacher explanation with 11 LLM-simulated teacher-profile explanations tied to distinct pedagogical risks. A comprehensive rubric covers factual precision, depth/completeness, focus/relevance, student-level appropriateness, and ideological bias, with structured explainability annotations validated by experts. Validation compares proprietary frontier models against a local Llama 3.1 8B for risk detection and explainability assessment, examining privacy-preserving deployability via fine-tuning.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages6

Reading Time15 min

Introduction

Motivation and Problem Setting

Related Work and Limitations

Contributions and Extensions

Dataset Construction and Annotation

AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations Javier Irigoyen∗ , Roberto Daza∗†, Francisco Jurado†, Julian Fierrez∗ , Ruben Tolosana∗ , Alvaro Ortigosa†, Enrique Blas∗ , and Aythami Morales∗ ,‡ ∗ BiometricsAI, Universidad Auto´noma de Madrid (UAM), Spain †GHIA, Universidad Auto´noma de Madrid (UAM), Spain ‡ Universidad de Las Palmas de Gran Canaria (ULPGC), Spain Corresponding author: roberto.daza@uam.es arXiv :2607 .0 1934v 1 [ cs .CL] 2 Jul 2026 Abstract—This work introduces AIriskEval-edu-db2, a new dataset designed to train and evaluate auditors based on LLMs for an explainable pedagogical risk assessment in instructional content for grades K–12. The dataset comprises 1,639 explanations from 170 curated ScienceQA questions, covering science, language arts, and social sciences. For each question, the dataset includes an explanation written by a human teacher alongside 11 explanations generated by LLM-simulated teacher profiles associated with distinct pedagogical risks. We propose a comprehensive risk rubric aligned with established educational standards that covers five complementary dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias. A key contribution is the addition of 785 explanations with structured explainability annotations, including risk localization and risk description. The annotations are produced through a semi-automatic process with expert teacher validation. Finally, we present validation experiments comparing state-of-the-art proprietary models with a lightweight local Llama 3.1 8B model in both the pedagogical risk detection and the explainability assessment. These experiments evaluate whether supervised fine-tuning on AIriskEval-edu-db2 enables a locally deployable model to approach or outperform stronger frontier models while preserving privacy in educational auditing and assessment tasks. I. INTRODUCTION Large Language Models (LLMs) are increasingly being deployed in human-facing systems where raw capability must be complemented by oversight, automatic monitoring, and risk-aware quality assurance. In educational technology, these models can answer K–12 questions with high precision [1], which has raised growing interest in their use as both tutors and as automatic evaluators of instructional explanations produced by humans or AI. At the same time, because such systems can produce unsafe, unreliable, or misleading outputs, their deployment also raises concerns that are closely related to the themes of automatic monitoring, human factors, artificial intelligence, and the societal impact of security technology [2] . In this sense, educational AI can also be viewed as a securitytechnology problem: it requires monitoring mechanisms able This research was supported by Ctedra ENIA UAM-VERIDASen IA Responsable (NextGenerationEU PRTR TSI-100927-2023-2), M2RAI (PID2024-160053OB-I00, MICIU/FEDER), TRUST-ID (PID2025- 173396OB-I00, MICIU/AEI and the EU) and PowerAI+ (SI4/PJI/2024-00062, Comunidad de Madrid and UAM) . Javier Irigoyen is supported by an FPI fellowship from MINECO/FEDER. to detect harmful, untrustworthy, or otherwise risky language outputs before or during deployment. Recent studies show that LLMs, especially when taskadapted, can approximate key tutoring behaviors [3], while industrial efforts such as LearnLM [4] and studies on the adaptation of instructional roles [5] further support this trend. At the same time, LLMs introduce well-documented risks [6], which are especially consequential in K–12 settings. This motivates automatic pedagogical assessors capable of detecting pedagogical and epistemic risks under established educational frameworks [7] . Previous work suggests that fine-tuning LLMs in rubricbased educational datasets can improve the reliability of theevaluator [8], [9] . However, current public resources remain limited: few target evaluat

FAQs

What is AIriskEval-edu-db2 designed for?

AIriskEval-edu-db2 is designed to train and evaluate LLM-based auditors for explainable pedagogical risk assessment in K–12 instructional explanations.

How is the dataset structured around teacher explanations and pedagogical risks?

For each of the 170 curated ScienceQA questions, the dataset includes a human teacher explanation plus 11 explanations generated by LLM-simulated teacher profiles, each associated with distinct pedagogical risks.

What rubric dimensions are used to assess pedagogical risks?

The proposed rubric covers five dimensions: factual precision, depth and completeness, focus and relevance, student-level appropriateness, and ideological bias.

Jul 16, 2026

Upload by Ava Thompson