Are COVID-19 vaccines the reason vaccinated people ages 10 to 59 in England are dying at twice the rate of the unvaccinated? No, that's not true: Age 10-59 is, statistically, a huge age group, so the claim uses a misleading "same age" comparison. Within that group are older adults, who are three times more likely to be vaccinated, with a death rate that is 48 times that of 10-year-olds.
Put simply, most of those who die in England are older and therefore vaccinated. The data demonstrate Simpson's Paradox, the name for a common statistical problem: A trend apparent in several groups of data disappears or reverses when the data are combined because you aren't considering an extra, and consequential, variable. In this case, that variable is the higher death rate of older people from all causes.
The claim appeared in a November 20, 2021, post in the Unreported Truths newsletter by spy novelist Alex Berenson titled "Vaccinated English adults under 60 are dying at twice the rate of unvaccinated people the same age" (archived here). It opened:
And have been for six months. This chart may seem unbelievable or impossible, but it's correct, based on weekly data from the British government ... I don't know how to explain this other than vaccine-caused mortality.
Users on social media only saw this title, description and thumbnail:
Vaccinated English adults under 60 are dying at twice the rate of unvaccinated people the same age
And have been for six months. This chart may seem unbelievable or impossible, but it's correct, based on weekly data from the British government.
Berenson, who uses his large social media following to spread false claims about COVID vaccines, did not know how to explain the apparent correlation between vaccines and higher death rates because he is not a statistician and did not ask a statistician to explain it to his readers.
Professor Barbra Richardson, a biostatistician in the Vaccine and Infectious Disease Division of Fred Hutchinson Cancer Center in Seattle, said she saw the claim -- which she called "a huge misrepresentation" -- bouncing around on Twitter and that the problem with his analysis was obvious. In a November 23, 2021, email to Lead Stories, Richardson wrote:
... this is purely due to the fact that the age range of 10-59 is huge and younger people are less likely to be vaccinated AND less likely to die of any cause. Thus the data graphed of vaccinated consists of a much older age group than the data graphed of unvaccinated and older age = higher mortality.
It's the same phenomenon Lead Stories found in testing false claims about health statistics from Israel, which an anti-vaccine film maker similarly oversimplified in his claim COVID vaccines are not working.
In both cases, collapsing complex data into a simple chart demonstrates Simpson's Paradox, the name for a common statistical problem: a trend disappears or flips when multiple sets of data are combined, if you haven't accounted for a consequential variable that isn't captured in the data. In this case, what's not factored into the chart is the higher mortality rate of older people.
Richardson reviewed Lead Stories' explanation, in a prior fact check, of how a person unschooled in statistics can fall prey to errors arising from Simpson's Paradox. Modified, by Richardson, to apply to the data on England, the explanation is this: Misleading results can sometimes be obtained from observational data in the presence of confounding factors. The reason for this paradoxical result is that both vaccination and all-cause mortality probability were systematically lower in one group (younger) and higher in the other group (older). This creates a specific type of confounding that can produce such a paradox.
Richardson said when Berenson failed to break the mortality data down by age groups, lumping healthy and unvaccinated children with vaccinated elders who have a higher death rate, he erred.
... if we do not stratify by the confounding factor (age), then the overall analysis gives a blatantly misleading result.
The following BBC chart of UK.gov data on vaccination rates by age in the United Kingdom illustrates how the 59-year-olds who have a 48 times higher rate of death from all causes than 10-year-olds are also more than three times as likely to be vaccinated against COVID. Put simply, more old people die than young people and old people are almost all vaccinated. So when you lump all the age groups together, you end up with a false correlation between vaccination and mortality rates.
Robert Anderson, Ph.D., chief of the mortality statistics branch of the (U.S.) National Center for Health Statistics, said in a November 23, 2021, email, it's a familiar statistical error.
Death rates naturally increase with age. It is likely that the vaccinated group skews older and so, will have naturally have a higher death rate. The unvaccinated population probably skews younger and will have a lower death rate as a result. I suspect that if one looks at smaller age groupings or if one adjusts for age distribution (age standardization or age adjustment), one would see the opposite, i.e., death rates are lower among the vaccinated.
