Does Calculator Use and Test Format Mask Weakness in Basic Math Ability? Experimental Evidence in Principles of Economics

Sep 14, 2026

Uploaded by Maeve

This research examines whether a simple Algebra I assessment without calculators and in an open-ended format, given by Principles of Economics faculty, captures weaknesses in students’ basic math ability that the SAT Math test does not. The study tests a hypothesis that calculator access and multiple-choice formatting may enable correct answers without true understanding of underlying concepts. Using fall 2013 experimental data from 1400 students, it compares speed and performance across question types and identifies math skills critical for economics.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages18

Reading Time45 min

AccessFree to download

Abstract

Introduction

Algebra I assessment and prerequisites

SAT Math vs. Algebra I assessment findings

Does Calculator Use and Test Format Mask Weakness in Basic Math Ability? Experimental Evidence in Principles of Economics Melanie Allwine and Irene R. Foster1 Department of Economics, The George Washington University December 2013 Preliminary version – please do not cite without authors’permission2 Abstract This research examines if a simple Algebra I assessment (no calculators allowed and open-‐ended format) administered by the Principles of Economics faculty at this institution is capturing some weakness in students’ basic math ability that the SAT Math test (calculator use allowed and mostly multiple-‐choice format) is not. Our hypothesis is that the use of a calculator and a multiple-‐choice format may be allowing students to answer questions correctly they would otherwise not be able to work through and solve. In other words, students may know how to compute answers with calculators or reverse engineer multiple-‐choice questions while failing to understand underlying math concepts. This paper presents results from primary experimental data collected Fall 2013 on 1400 students registered for Principles of Economics for which the prerequisite is Algebra I. The dataset contains Algebra I assessment results by type of question (with and without the use of a calculator; and either multiple choice or open-‐ended format) and other student characteristics including SAT Math scores. Assessment questions were identical to those on recent SAT Math tests. We identify whether calculator use or test format has a greater effect overall on speed and performance on tests. We identify questions students are able to answer correctly with or without a calculator, and multiple choice or open-‐ended format. We then identify questions students are unable to answer without a calculator or if a test has an open-‐ended format, and whether type of calculator makes a difference. We compare performance on such questions to see which of these math skills are critical to the study of economics. Results are presented by student characteristics. Finally, we discuss how to develop assessments to test math skills needed for Principles of Economics. 1 Corresponding author: Irene R. Foster – Email: [fosterir@gwu.edu](fosterir@gwu.edu) 2 Institutional Review Board permission was granted to conduct this research at The George Washington University (IRB \#081330) . 1. Introduction Each year, students’ college admissions are based in part on their SAT Math scores. The expectation of Economics faculty is that if students have been admitted into college, they should have the math skills necessary to navigate a Principles of Economics course. However, anecdotal evidence from faculty suggests that this isnot the case. Evidence from research in this area suggests that many students fail to comprehend foundational material in economics due to gaps in their understanding of basic math concepts (Mallik and Lodewijks 2010; Owen 2012) and that in addition to grades in advanced math classes and strong standardized test scores, a student’s score on a math quiz given at the beginning of an economics class has a statistically significant positive impact on performance in the course (Arnold and Straten 2012, Ballard and Johnson 2004, Benedict and Hoag 2002, Benedict and Hoag 2012, Schuhmann et al 2005) . To address this issue, Principles faculty at this institution administer a common Algebra I assessment at the start of each Fall semester. The questions are simple and cover only those skills required for a Principles class – percentage change, ratios, proportions, fractions and decimals, order of operations, place value, the area of a triangle, simple exponents and the graph of a straight line. Students are informed two weeks before class that an Algebra I assessment worth 10% of the course grade will be administered in Week One. They are pointed towards learning resources and told that calculators will not be allowed on the assessment. Any

FAQs

What is the main research question of the study?

The study asks whether calculator use and test format can mask weaknesses in students’ basic math ability in Principles of Economics.

How does the Algebra I assessment differ from the SAT Math test?

The Algebra I assessment prohibits calculators and uses an open-ended format, while the SAT Math test allows calculators and is mostly multiple-choice.

What data and approach does the paper use to evaluate effects?

It uses fall 2013 primary experimental data from 1400 Principles of Economics students, comparing speed and performance across question types (calculator vs. no calculator, multiple choice vs. open-ended) and relating results to student characteristics and SAT Math scores.

Sep 14, 2026

Uploaded by Maeve