Measured Approach or Magical Elixir? - How to Tell Good Science from Bad

Oct 15, 2026

Uploaded by McGucket

The text argues that education is filled with conflicting goals, research disputes, and profit-driven claims that are often presented as scientific. It recounts a real case involving educational software marketed as consistent with “whole brain learning,” then uses this context to show why evidence-based judgment is difficult. The article proposes a shortcut for evaluating scientific credibility through steps: stripping claims, tracing the persuader, analyzing why belief is requested, and making an informed decision.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages10

Reading Time25 min

AccessFree to download

Measured Approach or Magical Elixir?

How to Tell Good Science from Bad

The Four-Step Shortcut to Evaluate Evidence

Apply “Trace It” and “Analyze It” to Claims

Measured Approach or Magical Elixir? how to Tell Good Science from Bad ILLUSTRATIONS BY jAmES YANG By Daniel T. Willingham B ernhard Dohrmann is a businessman and entrepreneur of wide-ranging interests. Unfortunately, he has also had his share of legal problems. In 1975, he was convicted of securities fraud for selling railroad cars that did not exist. In 1982, he was charged by the Federal Trade Commission with misrepresenting the prices of investment diamonds. The case was settled out of court, with Dohrmann’s company returning $6.7 million to investors. In 1991, Dohrmann was charged by the U.S. attorney’s office with 16 counts of criminal contempt; it seems he Daniel T. Willingham is a professor of cognitive psychology at the University of Virginia and the author of numerous articles, including his regular“Ask the Cognitive Scientist” column for American educator. To read more of his work on education, [go to www.danielwillingham.com. This](go to www.danielwillingham.com. This)[ ](go to www.danielwillingham.com. This)[article is excerpted from his new book](article is excerpted from his new book), When Can You Trust the experts? how to Tell Good Science from Bad in education, © 2012 Daniel T. Willingham. This material is reproduced with permission of John Wiley & Sons, Inc. lied about his company’s sales figures when selling bonds to investors. he was sentenced to prison for this crime in November 1995.1 With such a history of legal problems, what’s a troubled businessman to do? Why, go into the educational software business, of course! Dohrmann started a company called life Success Academy that marketed (and continues to market) Super Teaching. Super Teaching consists of a system that projects images to three screens; the central screen shows whatever images a teacher typically uses in a lesson plan. The flanking screens show “seemingly random” images of nature, or real-time footage of the teacher orthe students. This practice is said to be consistent with “whole brain learning.”2 Systems initially were to sell for $160,000 per classroom;3 the current price is down to $29,500 .4 Although Super Teaching had been around since at least 2002, things started to look really promising for the life Success Academyin December 2007, when the company signed an agreement with the University of Alabama in huntsville. The university would help test and refine the Super Teaching method and would in 4 AmEricAN EdUcATOr | FALL 2012 return share in profits from future sales. In early October 2008, the university unveiled Super Teaching with a ribbon-cutting ceremony. The president of the university attended, but the honor of cutting the ribbon went—not inappropriately—to Tony robbins, motivational speaker and late-night infomercial pitchman.5 Ayear and a half later, the University of Alabama in huntsville dissolved its relationship with Dohrmann and the life Success Academy.6 Things had heated up six months earlier. A blog that covers Alabama politics had posted a lengthy summary of Dohrmann’s criminal past, provocatively headlined“Why Is UAh Involved with ‘a Very Dangerous Con Man’?”7 A month later, the university’s student newspaper published an article titled “learning at the Speed of Con.”8 This maybe an extreme example, but it’s hardly news that an educational reform idea attracted serious attention despite the fact that there was no evidence supporting it. If that were uncommon, I would have had no reason to write this article or the book provided; for example, saying that ham is “90 percent fat free!”sounds quite different than saying it is “10 percent fat!” “Trace it” is applied not to the educational claim or program but to its inventor. Most of us use this step already and, in fact, overuse it. It means to pay attention to the qualifications and motivations of the person trying to persuade us. We are most convinced by people who are knowledgeable and impartial. Unfortunately, it’shard to judge whe

FAQs

What problem does the text highlight in education research and reform claims?

Education is presented as awash in conflicting goals, “research wars,” and profiteers, making it hard to tell which claims are truly supported.

What is the main warning about using credentials to judge scientific credibility?

Credentials can be faked, and even genuine credentials are not a reliable guide to believability in education.

What are the four steps of the shortcut for evaluating evidence?

The shortcut consists of strip it and flip it, trace it, analyze it, and then make your decision about whether to adopt the claim.

Oct 15, 2026

Uploaded by McGucket