Athens Journal of Education - Rejoinder - Response to An Examination of Plausible Score Correlation from the Trend in Mathematics and Science Study - Canonical Correlation Analysis

Oct 15, 2026

Upload by Mimi

Rejoinder analyzing statistical computing for correlating student achievement in mathematics and science using TIMSS plausible values. The authors argue that major TIMSS reports did not consistently present correlational results, and existing computing macros did not address pairwise correlation needs among plausible values. They introduce canonical correlation analysis to handle complex sampling via sampling weights and standard error adjustment, addressing inflated Type I error risk. The paper also highlights issues in TIMSS multiple imputation reporting and the need for more plausible value imputations.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

AccessFree to download

What Is the Issue?

Core concerns raised by Mirazchiyski

Context and comparison with other international assessments

Athens Journal of Education-Volume 6, Issue 2 – Pages 159-170 Rejoinder: Response to: "An Examination of Plausible Score Correlation from the Trend in Mathematics and Science Study" By Jianjun Wang􀀍 & Xin Ma† This rejoinder keeps the original focus on statistical computing pertaining to the correlation of student achievement between mathematics and science from the Trend in Mathematics and Science Study (TIMSS). Albeit the availability of student performance data in TIMSS and the emphasis of the inter-subject connection in the Next Generation Science Standards in the U.S., the major TIMSS reports did not consistently include the correlational findings of student achievements, nor did the TIMSS macro, JACKREGPV per Mirazchiyski’s recommendation, meet the need of correlation computing between each pair of plausible values (PV) in mathematics and science. In addition, an inflation of Type I error would be inevitable unless a feasible approach is taken to concurrently correlate the two sets of PVs from each subject. In this context, we adduced canonical correlation analysis (CCA) as a feasible approach that included considerations of both sampling weight and standard error adjustment due to complex sampling. More importantly, this rejoinder reveals two fundamental issues in the TIMSS literature: (1) Incompliance to the multiple imputation rule in reporting the TIMSS 1995 international findings (see Beaton et al., 1996a, b); (2) Need of substantially increasing the number of PV imputations as illustrated by the mainstream statistical computing in general (e.g., SAS Institute, 2015; STATA, 2017), and the recent international large scale assessments (ILSA) in particular. Keywords: International Studies, TIMSS Data Analysis It is our great honor to contribute this rejoinder to an article of our esteemed colleague, Plamen Vladkov Mirazchiyski, the former Deputy Head of the Research and Analysis Unit of the International Association for the Evaluation of Educational Achievement (IEA) . We appreciate his effort on writing an extensive response (we shall brief his article name as "response") that nearly doubled the space of our original article. We sincerely accept his criticism on the briefness of our report that was originally designed for a 20-minute presentation at the ninth Annual International Conference on Statistics (Athens, Greece) in June, 2015. In this rejoinder, we are given the opportunity to elaborate our research with a clear focus on the potential disagreements we seemed to have with Mirazchiyski. To keep it simple, we categorized this article into four parts. First, we will analyze the core issues that originated this track of in-depth inquiry. In the second section, we clarify what is not the issue. The third section is devoted to coverage of new literature beyond the foundation of multiple imputation method Mirazchiyski 􀀍Professor, California State University, Bakersfield, USA. †Professor, Department of Educational, School, and Counseling Psychology, University of Kentucky, USA. [https://doi.org/10.30958/aje.6-2-5](https://doi.org/10.30958/aje.6-2-5) doi=10.30958/aje.6-2-5 Vol. 6, No. 2 Wang & Ma: Rejoinder: Response to: "An Examination of Plausible… reviewed from the period of 1970s – 1990s. We conclude this rejoinder in the last section to support the momentum of knowledge advancement in comparative education. What Is the Issue? Mirazchiyski raised issues on several fronts, including (1) the incorporation of sampling weights and the adjustment of result variance due to stratified, cluster sampling, (2) the ignorance of using TIMSS tools, particularly the macro of JACKREGPV.SPS, in the statistical computing that involves plausible values, (3) the use of design effect that, albeit its candidacy as a viable method for handling data with complex sampling in many decades (Kish, 1965), might produce crude estimates of the standard errors, (4) the inappropriateness of using the canonical correlation

FAQs

What approach do the authors propose for computing correlations between mathematics and science plausible values?

They propose canonical correlation analysis (CCA) as a feasible approach that incorporates sampling weights and adjusts for standard errors under complex sampling.

Why do the authors argue existing TIMSS reporting and tools were insufficient?

They state that major TIMSS reports did not consistently include correlational findings, and TIMSS tools such as JACKREGPV did not meet the need for correlation computing between each pair of plausible values.

Which two fundamental issues in TIMSS literature does the rejoinder emphasize?

It highlights (1) non-compliance with the multiple imputation rule in reporting TIMSS 1995 international findings, and (2) the need to substantially increase the number of plausible value imputations.

Oct 15, 2026

Upload by Mimi