The Marketing of Canadian University - 2016 CELT Vol IX - Rankings: A Misadventure

Oct 15, 2026

Uploaded by jay

A university rankings study based on Maclean’s ranking data over the past 24 years summarizes statistical findings and examines the current status and consequences for student welfare. Using correlational and cluster analyses, the research reports largely nonsignificant, inconsistent, and difficult-to-interpret relations between university rank standings and Maclean’s measures, as well as between ranks and the indices used to generate them. The work argues that empirical characteristics show the system has highly limited practical or academic value and may create unintended negative effects.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Introduction

The exercise of ranking

Institutional rankings as a global challenge

UNESCO conference mandate

Evaluation of Maclean's Canadian ranking exercise

The Marketing of Canadian University 2016 CELT Vol IX Rankings: A Misadventure [Now](Now celt. uwindsor.ca)[ celt. uwindsor.ca](Now celt. uwindsor.ca) [www.stlhe.ca](www.stlhe.ca) 24 Years Old Kenneth M. Cramer, Stewart Page, Vanessa Burrows, Chastine Lamoureux, Sarah Mackay, Victoria Pedri, and Rebecca Pschibul University of Windsor Based on analyses of Maclean’s ranking data pertaining to Canadian universities published over the last 24 years, we present a summary of statistical findings of annual ranking exercises, as well as discussion about their current status and the effects upon student welfare. Some illustrative tables are also presented. Using correlational and cluster analyses, for each year, we have found largely nonsignificant, inconsistent, and uninterpretable relations between rank standings of universities and Maclean’s main measures, as well as between rank standings and the many specific indices used to generate these standings. In our opinion, when assessed in terms of their empirical characteristics, the annual data show generally that this system of ranking is highly limited in terms of its practical or academic value to students. Among other difficulties with the interpretation of ranks, we also discuss the possibility that ranking exercises have unintended, though potentially serious, negative consequences for the intellectual and personal welfare of students. Introduction The exercise entities is of rating or ranking life’s various commonplace today – from best weather and places to live, to restaurants and toasters, to recording artists and vacation spots. Forbes’ top 500 companies and public institutions such as hospitals and even centres of higher learning are no exception (Aghaz, Hashemi, & Atashgah, 2015; Amsler & Bolsman, 2012; Huang, Chen, & Chien, 2015; Page & Cramer, 2001; Page, Cramer, & Page, 2008, 2010) . Canadian publication Maclean’s similarly aims to aid consumers in reaching a sound decision on where to attend college or university. 1 However, with increased public demand for external accountability, the transparency of these institutions becomes paramount (Allen & Bresciani, 2003; Shavelson & Huang, 2003), and spurs a spirit for additional (albeit valid) assessment data. As tools intended for monitoring institutional reputation and performance, Van Dyke (2005) suggests these report cards or “league sheets” as found in Maclean’s are especially popular among students and parents, and are also becoming increasingly accepted in academia. Yet the exercise of ranking is not without its criticisms. They include (but are not limited to) the halo effect of reputation, arbitrary subjectivity, relative weightings of indices, lack of statistical rigor, and the limits of ranks as units of measure (Brooks, 2005; Clarke, 2002; Ferguson & Takane, 1989; 1 Since their initial publication, Maclean’s has instituted occasional minor adjustments to their university ranking procedures, although their main measures, component indices, and overall approach remain essentially unchanged. Huang et al., 2015; Page et al., 2010; Provan & Abercromby, 2000; Siegel, 1959) . As Salmi and Saroyan (2007, p. 52) note: Notwithstanding their controversial nature and methodological shortcomings, university rankings have become widespread and are unlikely to disappear. Possible reactions, in the face of this rapidly expanding phenomenon, are to ignore, dismiss or boycott any form of ranking. Another, less extreme response is one that seeks to analyse and understand the significance and limitations of ranking exercises. We are presently pursuing the latter avenue, and sought to analyze the 2011-2015 data in the Canadian arena, as provided by Maclean’s. To address concerns surrounding institutional rankings as a global challenge, a UNESCO conference concluded with the crossnational mandate to evaluate and rank institutions of higher learning (Dill & Soo, 2005), which could direct educational

FAQs

What data and time span does the study use to assess Canadian university rankings?

It is based on analyses of Maclean’s ranking data pertaining to Canadian universities published over the last 24 years, with a focus on examining 2011–2015 data in the Canadian arena.

What do the analyses find about the relationship between university ranks and Maclean’s measures?

Using correlational and cluster analyses, the study finds largely nonsignificant, inconsistent, and uninterpretable relations between rank standings and Maclean’s main measures, as well as between ranks and the specific indices used to generate them.

What potential negative consequences does the paper discuss regarding ranking exercises?

It discusses the possibility that ranking exercises can have unintended, though potentially serious, negative consequences for students’ intellectual and personal welfare.

Oct 15, 2026

Uploaded by jay