Someone (not Will Rogers) once joked that “When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states.” This quirk of statistics has some surprising implications for cancer survival rates.
Hey, it’s time for another dive into the ways that people misinterpret statistics. Like the false positive paradox and Simpson’s paradox, the Will Rogers paradox isn’t actually a paradox. And Will Rogers didn’t even say it! But this phenomenon has some interesting implications in a number of fields, including how we judge the efficacy of medical treatments.
The original quotation, misattributed to early 20th century American humourist Will Rogers, is as follows:
When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states.
The implication is that the Okies were below-average in Oklahoma but above-average in California. Them moving from one place to the other removes the dullards from Oklahoma, and introduces some (relative) geniuses to California. Will Rogers, it will not surprise you to learn, was from Oklahoma. New Zealand prime minister Robert Muldoon once quipped the same:
New Zealanders who emigrate to Australia raise the IQ of both countries.
(But then, of course, Muldoon is the one who announced a snap election while drunk, so it is best not to take him too seriously.)
This quirk of statistics actually happens in real life. In sports statistics, for example, a sub-standard player from a high performing team might become a high-performing player in a sub-standard team. Them moving from one team to the other improves the performance of both.
But this paradox becomes truly important when it emerges in medical statistics. Consider the following scenario. Lung cancer patients are subdivided into Stage 1, Stage 2, and Stage 3 cancer depending on how far their disease has spread. Looking at the mortality rates for each stage, you can calculate patients’ average chance of survival.
You can compare these survival rates across time to see if there’s any improvement in treatments and outcomes. In one study (the second link below), the survival statistics for each stage were better in 1977 than they were from 1953 to 1964. But the actual survival rates were not.
What happened? Treatment had not improved, but diagnostics had. Modern imaging equipment could track cancer spread more accurately. Small metastases that otherwise would have gone unnoticed were now noticed. Doctors might classify the same cancer as Stage 2 in 1964, but Stage 3 in 1977.
Those formerly hidden cancers were some of the most deadly from the Stage 2 group. So the simple act of diagnostic reclassification improved the Stage 2 survival rate. But when you put them into Stage 3, they’re not as deadly as the rest of that group. So the Stage 3 survival rate improves as well. This is the Will Rogers paradox in action.
A quick skim of Google Scholar reveals dozens of papers about this effect: it has been observed in statistics for prostate cancer, breast cancer, stomach cancer, multiple sclerosis, and many more. And it’s yet another example of just how easy it is to be mislead by shoddy statistics.
The Generalist Academy 