Defining and Evaluating Argumentation Quality in the Context of Design Thinking - Using High School Students’ Design Critiques from Foundational Engineering Courses

Oct 15, 2026

Uploaded by OmBimo

This research investigates students’ argumentation quality in engineering design thinking. Learning by Evaluating (LbE) was implemented using Adaptive Comparative Judgment (ACJ), where students compared item pairs and wrote rationales for their choices. Fifteen students evaluated backpack designs and produced 145 comments, analyzed with a Claim, Evidence, and Reasoning (CER) framework adapted to engineering design thinking. Quality was coded across empathy, ideation, insight, and related critique characteristics.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages25

Reading Time1 h 3 min

AccessFree to download

Abstract

Keywords

Introduction

Defining and Evaluating Argumentation Quality in the Context of Design Thinking: Using High School Students’ Design Critiques from Foundational Engineering Courses Wonki Lee, Purdue University, USA Nathan Mentzer, Purdue University, USA Andrew Jackson, University of Georgia, USA Scott Bartholomew, Brigham Young University, USA Amiah Clevenger, Purdue University, USA Abstract This research investigates students’ argumentation quality in engineering design thinking. We implemented Learning by Evaluating (LbE) using Adaptive Comparative Judgment (ACJ), where students assess pairs of items to determine the superior one. In ACJ, students provided rationales for their critiques, explaining their selections. Fifteen students participated in an LbE exercise before starting their backpack design projects, critically evaluating multiple backpack designs and producing 145 comments. Writing comments required students to discern and justify the superior design, fostering informed judgment and articulation of their reasoning. The study used the Claim, Evidence, and Reasoning (CER) framework, adapted for engineering design thinking, to analyse these critiques. The framework emphasized three aspects: Empathy (understanding user needs), Ideation (deriving design inspiration), and Insight (gaining valuable understanding from evaluated designs) . We employed both deductive and inductive content analysis to evaluate the argumentation quality in students’ critiques. High-quality argumentation was identified based on six codes: user-focused empathy, design inspirations, logical rationalizations, multi-criteria evaluations, aesthetic considerations, and cultural awareness. Poor-quality argumentation lacked these elements and was characterized by vagueness, uncertainty, brevity, inappropriateness, irrelevance, gender bias, and cultural stereotyping. By identifying critical elements of effective argumentation and common challenges students may face, this study aims to enhance argumentation skills in engineering design thinking at the secondary education level. These insights are intended to help educators prepare students for insightful and successful argumentation in engineering design projects. Keywords LbE (Learning by Evaluating), ACJ (Adaptive Comparative Judgment), CER (Claim, Evidence, and Reasoning) framework, Design thinking Introduction There is an urgent need to enhance young individuals’ comprehension of argumentation within a scientific context (Osborne et al., 2004) . Osborne and colleagues (2004) asserted that science, a field that prides itself on rationality, often falls short in teaching students about the epistemological foundations of belief (Driver et al., 2000; Duschl & Osborne, 2002; Osborne et al., 2003) . Oftentimes, this failure results in students’ holding simplistic perceptions of science (Driver et al., 1996, 2000) . Thus, one of the vital responsibilities for the current education is to cultivate argument construction and explanation. Such instruction enhances students’ ability to comprehend and employ scientifically sound argumentation (Duschl & Osborne, 2002) . Osborne (2004) underscored the significance of embedding scientific argumentation within classroom settings, suggesting it serves as a heuristic methodology that augments students’ability to navigate conceptual and epistemic objectives. This integration not only illuminates students’ scientific cognition and reasoning but also facilitates formative assessment opportunities for educators. Consequently, the pursuit of epistemic objectives, such as the formulation, assessment, and refinement of scientific arguments, is imperative to modern science education. Numerous studies have highlighted that fostering students’ ability to construct scientific arguments can be significantly enhanced by presenting them with contrasting theories or evidence (Keogh & Naylor, 2000; Osborne et al., 2004; Settlage & Sabik, 1997; Solomon et al., 1992) . This metho

FAQs

How was argumentation quality evaluated in this study?

The study used Learning by Evaluating (LbE) with Adaptive Comparative Judgment (ACJ), where students compared pairs of items and provided rationales. Critiques were analyzed using an engineering-design-adapted Claim, Evidence, and Reasoning (CER) framework.

What did students do before starting their design project?

Fifteen students completed an LbE exercise before beginning their backpack design projects. They critically evaluated multiple backpack designs and produced 145 written comments with explanations for their selections.

What distinguishes high-quality from poor-quality argumentation?

High-quality critiques contained elements such as user-focused empathy, logical rationalizations, multi-criteria evaluations, aesthetic considerations, and cultural awareness. Poor-quality critiques were characterized by vagueness, uncertainty, brevity, inappropriateness, irrelevance, gender bias, and cultural stereotyping.

Oct 15, 2026

Uploaded by OmBimo