Athens Journal of Education - Volume 6 - Issue 2 - Response to "An Examination of Plausible Score Correlation from the Trend in Mathematics and Science Study"

Oct 15, 2026

Upload by Mimi

Response article addressing methodological concerns raised by Wang and Ma regarding correlating plausible values (PVs) from TIMSS using canonical correlation analysis. The text reviews the design of international large-scale assessments, especially TIMSS, and explains how PVs and sampling procedures affect statistical estimation. It argues the proposed CCA approach is unsuitable because PVs are not multivariate measures as assumed. Additional issues in Wang and Ma’s overall analysis approach are discussed.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

AccessFree to download

Introduction

Literature Review

Discussion

Athens Journal of Education-Volume 6, Issue 2 – Pages 145-158 Response to "An Examination of Plausible Score Correlation from the Trend in Mathematics and Science Study" By Plamen Vladkov Mirazchiyski 􀀍 This article is a response to an article written by Wang and Ma "An Examination of Plausible Score Correlation from the Trend in Mathematics and Science Study", published in the Athens Journal of Education. The purpose of this paper is to address issues with Wang’s and Ma’s suggestion to use analysis method for correlating plausible values from TIMSS. The paper reviews the design of international large-scale assessments, and TIMSS in particular, and its implications for data analysis in regard to the application of Canonical Correlation Analysis for calculating association between two sets of plausible values, as proposed by Wang and Ma. The conclusion is that, given the design ofTIMSS and other large-scale assessments, the method proposed by Wang and Ma is not appropriate for correlating two sets of plausible values because they are not multivariate measures as the suggested method would assume. Some other methodological issues related to the overall analysis approach used by Wang and Ma is discussed as well. Keywords: correlation, international large-scale assessments, methodology, multiple imputation, plausible values Introduction This paper is a response article that addresses some issues with an article by Wang and Ma (2016) published in the Athens Journal of Education. The original article published by the aforementioned authors represents an attempt to address methodological issues when correlating two sets of plausible values (PVs) from the Trends in International Mathematics and Science Study (TIMSS) that the authors claim to exist. The main argument of the authors is that if two sets of PVs are used in correlation analyses of TIMSS data with the current approach the study uses, this can inflate the chance for making Type I error due to the non-additive nature of the correlation coefficients. The solution of this assumed issue the authors suggest is Canonical Correlation Analyses (CCA) that can accommodate for the multidimensionality of the signals and avoid the dependency on the coordinate system in which the variables are described (Wang & Ma, 2016). However, most of the assumptions the authors have do not reflect the design and the methodology TIMSS, as well as other large-scale assessments (ILSA), as well as the analytical strategy and methods it uses to produce unbiased estimates from any statistical analysis being employed. The next section provides a review of the literature on generation of PV sin 􀀍Chief Executive Officer, Researcher, International Educational Research and Evaluation Institute (INERI), Slovenia. [https://doi.org/10.30958/aje.6-2-4](https://doi.org/10.30958/aje.6-2-4) doi=10.30958/aje.6-2-4 Vol. 6, No. 2 Mirazchiyski: Response to "An Examination of Plausible Score… general and as they are used in TIMSS (as well as in other ILSA), what is the necessity for their derivation as proficiency scores, what they are, how they are derived and how they shall be used in analysis reflecting their nature and statistical theory that stands behind their derivation. In addition, the literature review presents the sampling procedures and the derivation of the sampling weights which must be included in the computation of any estimates using ILSA data. This is necessary because Wang and Ma (2016) do not address the important issues of the unequal probabilities of selection and the subsequent issue of sampling variance when computing estimates with TIMSS data. The literature review ends with an overview of the CCA, its assumptions and application in situations where other correlation methods are not appropriate. The purpose of this review is to add more clarity on the design of TIMSS and ILSA in general, and thus, justify the current approaches in using them in analysis. These provide the

FAQs

What is the purpose of this response article to Wang and Ma (2016)?

It addresses issues with Wang and Ma’s suggestion for analyzing plausible values from TIMSS, focusing on whether their proposed correlation approach matches TIMSS design and methodology.

Why does the article argue Wang and Ma’s proposed method is not appropriate for correlating two sets of plausible values?

It concludes that, given the design of TIMSS and other large-scale assessments, the method assumes PVs are multivariate measures, which the article says they are not.

Which topics does the literature review cover before returning to the critique?

It reviews the generation and use of plausible values, including how proficiency scores are derived, the need for sampling weights, unequal selection probabilities, sampling variance, and an overview of canonical correlation analysis assumptions and application.

Oct 15, 2026

Upload by Mimi