Does a new study from Germany show that a higher COVID-19 vaccination rate results in higher death rates? No, that's not true: A German politician had presented a paper in a state parliamentary debate on November 17, 2021, that allegedly found that German states with a higher vaccination rate showed a higher excess mortality during six weeks between September and October 2021. The authors of the study have since backpedaled on the results and claim that their paper had only been intended as an internal "note." A closer look at the paper revealed methodological flaws: After re-examining the data the authors conceded that there is no statistically significant correlation.
The claim originated in a Substack newsletter by Steve Kirsch on November 20, 2021, titled "New study from Germany confirms higher vax coverage --> higher excess mortality" (archived here). It opened:
The Harvard study showed vaccination makes things worse as far as cases goes. This new study from German shows that the more you vaccinate the more people get killed. Not a surprise to me.
Users on social media only saw this title, description and thumbnail:
New study from Germany confirms higher vax coverage --> higher excess mortality
The Harvard study showed vaccination makes things worse as far as cases goes. This new study from German shows that the more you vaccinate the more people get killed. Not a surprise to me.
The Substack post referred to a calculation commissioned by Ute Bergner, a member of the state parliament in the Eastern German state of Thuringia. Bergner, who is a member of a small party called Bürger für Thüringen known for its anti-vaccine and anti-mask stance, presented the three-page paper to the state's health minister in a parliamentary session on November 17, 2021, to counter debates on vaccine mandates in Thuringia. The paper was also published on Bergner's website the next day. In her speech, Bergner refers to official statistical data showing an excess mortality in Germany of about 10% above the median for September 2021, a figure that had sparked discussions on social media on whether these deaths were caused by the COVID vaccines.
The paper was written by two statisticians with a background in psychology, Gregor Kappler and Professor Rolf Steyer. Steyer is a renowned statistician; until his retirement in 2020, he was professor for methodology and evaluation research at the psychology department of Universität Jena. In the paper Bergner presented and Kirsch included in his Substack post with an English translation, Kappler and Steyer did the following: They compared mortality rates for the 16 federal states of Germany ("länder" in German, incorrectly translated as countries in Kirsch's post) with a baseline number. The baseline was calculated by taking the median of the mortality numbers of the years 2016 to 2020 and adjusting them for population size of the 16 states. For this, they drew on numbers published by Destatis, the Federal Statistical Office.
Deviation from the baseline was then examined in relation to the states' vaccination rates, drawing on numbers by the Robert Koch Institut, the country's official public health authority. The authors found a correlation of +0.31, meaning that for the short period of time investigated here, states with higher vaccination rates showed a slightly higher excess mortality. In their paper, the authors themselves write that this correlation is "amazingly high," while cautioning that one cannot infer causation from this correlation.
After a video clip of Bergner's presentation went viral on Twitter, the methodology used and the interpretation of the results were harshly criticized by experts. The official Destatis Twitter account commented on the tweet that had first made the clip of Bergner's speech public, pointing out the following: "Please note: Correlation does not mean causality. There is no evidence that the respective excess mortality in the federal states has anything to do with the vaccination rate. Apart from that, a correlation coefficient of 0.31 is not considered a high correlation."
1/2 Bitte beachten Sie: Korrelation bedeutet nicht Kausalität. Es gibt keinen Beleg, dass die unterschiedliche Übersterblichkeit in den Ländern mit der Impfquote zusammenhängt. Ein Korrelationskoeffizient von 0,31 gilt zudem nicht als hohe Korrelation.
-- Statistisches Bundesamt (@destatis) November 22, 2021
On November 25, 2021, public broadcaster ARD published a fact check on the by then viral paper. Felix zur Nieden, Destatis expert on demographics and death rates, told ARD that the study was not methodologically sound and harshly critized the authors, saying, "this is not a serious evaluation that should have been made public." According to zur Nieden, even a scatterplot visualization included in the paper showed that there was no systematic correlation between vaccination rates and excess mortality.
(Image taken from "Je höher die Impfquote, desto höher die Übersterblichkeit" by Rolf Steyer and Prof. Dr. Rolf Steyer und Dr. Gregor Kappler, published on November 16,2021)
Zur Nieden also pointed out that weighing the federal states' population sizes skewed the findings, since the correlation will depend only on the largest states and thus on too few data points to get a meaningful result. He also told ARD that it was surprising that the authors of the paper considered only the vaccination numbers as a possible reason for the higher mortality rates in September, leaving out all other possible factors.
After receiving a lot of backlash from social media users as well as their colleagues -- Professor Helmut Küchenhoff, who teaches statistics at LMU Munich, said the paper was on a level of claiming a correlation between "storks and births" -- the authors of the paper published a statement on Steyer's university website refuting the interpretations of what they now simply called "a note." Below is the full statement, translated from German:
On November 16, 2021, we authored a short note for the member of the Thuringia state parliament, Dr. Ute Bergner. The occasion was a special parliamentary session on current events dealing with coronavirus policies. The note was intended for this purpose only. We agreed to the paper being made available to the state health minister and, after their request, also to the members of parliament from the CDU. The reported and surprising correlation was intended to spark further discussions and analyses. We regret that this note has been so widely distributed. The publication and spread of the note on the internet and on social media was not authorized or even initiated by us.
For clarification: the note is neither a scientific publication nor a sound scientific study, which meets our own standards for quality. Our note does not prove in any way that a higher vaccination rate leads to a higher mortality. We do not want the study to be falsely interpreted this way. There are countless reasons which could explain the positive correlation we found, without suggesting a negative effect of the vaccination rate on excess mortality. More comprehensive scientific analysis is needed to explain the current excess mortality. When the now available data for calendar week 41 is included, the positive correlation we reported last week is close to zero.
Wherever our note is cited or even reproduced in full, we ask that this statement is made available.
24. Nov. 2021
Rolf Steyer and Gregor Kappler
The paper had such severe methodological flaws and drew on such limited data that even the authors had to concede there was no observable correlation between vaccination rates and excess mortality. As of December 2, 2021, Germany has a 68.7% vaccination rate.
