Thinking Skills and Admissions - A report on the Validity and Reliability - of the TSA and MVAT/BMAT Assessments

Sep 18, 2026

Uploaded by Patrick

This report documents UCLES work on Thinking Skills, evaluating the TSA and parts of the MVAT/BMAT admissions assessments developed through and beyond the late 1980s. It reviews assessment properties, focusing on validity and reliability, and examines how TSA and MVAT/BMAT scores relate to university or performance outcomes. Findings indicate the TSA and BMAT (Parts 1 and 2) are reasonably sound, with limited yet non-problematic predictive validity and a cautious outlook for future development.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages45

Reading Time1 h 53 min

AccessFree to download

Overview

Background

UCLES, Thinking Skills and MENO

A Growing Problem with Admissions

A Trial

A Further Development

The TSA Questions

The Six Skills of MENO

Other Developments

The TSA

Validity and Reliability

Some Properties of an Assessment

Validity

Reliability

The Reliability of the TSA Tests

Early Development of Thinking Skills

CTSC and CATS

TSA Pre-tests

TSA Tests

Other Uses of TSA

Issues of Validity

Early Findings

Use of TSA

TSA Scores as Predictors of Achievement

Critical Thinking and Problem Solving: One Skill or Two?

The Two Sub-scales

Early Results

TSA Analyses and Implications

The MVAT and the BMAT

Need for an Aid to Admissions in Medical Courses

Reliability of the MVAT and BMAT Assessments

Predictors of Performance and Use in Admissions

TSA and BMAT in the Future

Much Needs Doing

Comments and Conclusions

Suggestions for Further Work

Appendices

TSA Test Specification

Areas for Further Work

References

Thinking Skills and Admissions A report on the Validity and Reliability of the TSA and MVAT/BMAT Assessments Alan Willmott September 2005 Contents Section Contents Page List of Tables iv Overview vi Note on References in this Report vii 1 Background 1 1.1 UCLES, Thinking Skills and MENO 1 1.2 A Growing Problem with Admissions 3 1.3 A Trial 3 1.4 A Further Development 4 2 The TSA Questions 5 2.1 The Six Skills of MENO 5 2.2 Other Developments 5 2.3 The TSA 6 3 Validity and Reliability 7 3.1 Some Properties of an Assessment 7 3.2 Validity 7 3.3 Reliability 8 3.4 Validity and Reliability 8 4 The Reliability of the TSA Tests 10 4.1 The Early Development of Thinking Skills 10 4.2 CTSC and CATS 10 4.3 TSA Pre-tests 11 4.4 TSA Tests 12 4.5 Other Uses of TSA 13 4.6 The Reliability of TSA 13 5 Issues of Validity 14 5.1 Some Early Findings 14 5.2 The Use of TSA 14 5.3 TSA Scores as Predictors of Achievement 14 5.4 Prediction by TSA Score Sub-scale 16 5.5 The Use of TSA in the Admissions Process 16 5.6 Some Comments from Users 18 5.7 Bias in TSA Questions 19 Thinking Skills and Admissions ii Contents Section Contents Page 6 Critical Thinking and Problem Solving: One Skill or Two? 20 6.1 The Two Sub-scales 20 6.2 Early Results 21 6.3 Results for TSA Analyses 21 6.4 Sub-scale Inter-Correlations 22 6.5 The CT and PS Sub-Scales 23 6.6 The Implications of the Findings 24 7 The MVAT and the BMAT 25 7.1 The Need for an Aid to Admissions in Medical Courses 25 7.2 The Reliability of the MVAT and BMAT Assessments 25 7.2.1 MVAT and BMAT Part 1 25 7.2.2 MVAT and BMAT Part 2 26 7.2.3 MVAT and BMAT Part 3 26 7.3 MVAT and BMAT Scores as Predictors of Performance 26 7.4 The Use of MVAT and BMAT in the Admissions Process 27 7.5 The MVAT and BMAT Sub-Scores 29 7.6 The Sub-Scale Inter-Correlations 30 7.7 The Implications of the Analyses 30 8 TSA and BMAT in the Future 31 8.1 Much Needs Doing 31 8.2 Some Comments and Conclusions 33 8.3 Suggestions for Further Work 33 Appendix A TSA Test Specification 34 Appendix B Areas for Further Work 35 References 36 Thinking Skills and Admissions iii List of Tables Table Number Content Page 1.1 Applications and Admissions for the 3 University of Cambridge in 1973 and 2003 4.1 Reliability Estimates for CTSC and CATS 11 Assessments – 1996-2003 4.2 TSA Pre-test Reliability Estimates 11 4.3 TSA Reliability Estimates 12 4.4 Reliability Estimates for Other Uses of TSA Tests 13 5.1 Correlations of TSA Scores with First-Year Results – 15 2001-3 and 2002-4 5.2 Correlations of CT and PS Scores with First Year Results – 16 2002-4 5.3 TSA Score by Application Decision – 2003 Cohort 17 5.4 TSA Score by Application Decision: 17 Subject B Applicants – 2003 Cohort 5.5 Correlations between TSA Sub-scores and 18 Application Outcome 6.1 Estimated Reliability of TSA Sub-scores – Early Results 21 6.2 Summary of Sub-Scale Analyses for TSA tests 22 6.3 Correlations between the Sub-Scales of TSA Tests 23 7.1 Reliability Estimates for Part 1 of the MVAT/BMAT Tests – 26 1999-2004 7.2 Reliability Estimates for Part 2 of the MVAT/BMAT Tests 26 – 1999-2004 7.3 Predictive Validity of MVAT 2000 27 7.4 MVAT Scores by Application Decision – 2000 27 7.5 Correlations-MVAT Scores and Application 27 Decisions – 2000 7.6 MVAT Scores by Application Decision – 2001 28 7.7 Correlations-MVAT Scores and Application 28 Decisions – 2001 Thinking Skills and Admissions iv List of Tables Table Number Content Page 7.8 MVAT Scores by Application Decision – 2002 28 7.9 Correlations-MVAT Scores and Application 28 Decisions – 2002 7.10 BMAT Scores by Application Decision – 2003 28 7.11 Correlations-BMAT Scores and Application 29 Decisions – 2003 7.12 Summary of Sub-Scale Analyses for Part 1 of MVAT/BMAT 29 7.13 Correlations between the Sub-Scales 30 of Part 1 MVAT and BMA

FAQs

What does the report evaluate regarding admissions assessments?

The report evaluates the TSA and parts of the MVAT/BMAT tests, focusing on their validity and reliability and whether scores can predict achievement or performance in relevant admissions contexts.

How are validity and reliability discussed in the report?

It reviews properties of an assessment, then separately considers validity and reliability, including how validity issues and reliability estimates affect interpretation of test scores.

What conclusions does the report reach about the TSA and BMAT assessments?

It finds the TSA and BMAT (Parts 1 and 2) to be reasonably sound, notes limited predictive validity information so far, and suggests that future developments should be approached with caution while expanding data for further analyses.

Sep 18, 2026

Uploaded by Patrick