STORY UPDATED: check for updates below.
Does a reanalysis of Pfizer and Moderna mRNA COVID-19 vaccine data show an increased risk of serious adverse events of special interest? No, that's not true: The authors of this study created their own subset of serious adverse events of special interest (AESI), which give the appearance of there being a strong correlation between them and the vaccines when, in fact, there was very little. Also, both the Centers for Disease Control and Prevention (CDC) and the Food and Drug Administration (FDA) say the coronavirus shots are safe and effective.
The claim appeared in a YouTube video (archived here) published by John Campbell on December 30, 2022, under the title "Reanalysis of mRNA trial data." The description opened:
Swine flu vaccine (1976), 1 serious event per 100,000 vaccinees, Vaccine withdrawn
Rotavirus vaccine Rotashield, (1999),1 to 2 serious events per 10,000 vaccinees, Vaccine withdrawn
Covid mRNA vaccines, 1 serious event per 800 vaccinees, Vaccine officially promoted
This is what the video looked like on YouTube at the time of writing:
(Source: YouTube video screenshot taken on Tue Jan 3 19:07:17 2023 UTC)
The swine flu and rotavirus numbers come from the CDC website on "Historical Vaccine Safety Concerns." The numbers in the video are accurate but are not specific about the adverse event, as the CDC documented. For swine flu, the increased risk was for one additional case of Guillain-Barré Syndrome (GBS) following vaccination with a swine flu vaccine. For rotavirus, the CDC estimated that one or two additional cases of intussusception would be caused among every 10,000 infants vaccinated with the RotaShield® vaccine.
John Campbell is a popular YouTube host with more than 2.6 million subscribers to his channel. He is not a physician but a retired emergency department nurse. According to the "About" page on his YouTube channel, his Ph.D. "focused on the development of open learning resources for nurses nationally and internationally." Since starting his channel in 2007, he has posted thousands of videos on a variety of medical topics but has focused primarily on COVID since the start of the pandemic, promoting unproven treatments like ivermectin (see here, here, and here), spreading misleading death statistics minimizing the number of victims and misinterpreting data about vaccine safety
Campbell's video details a peer-reviewed article (archived here) published September 22, 2022, in the online publication Vaccine, titled "Serious adverse events of special interest following mRNA COVID-19 vaccination in randomized trials in adults."
The article looks at data that was included in the FDA clinical review memo, statistical safety review memo and other public documents used in the authorization of the Moderna and Pfizer (Pfizer partnered with BioNTech) vaccines. The "harm-benefit analyses" called for in the study have already been performed by the FDA. This type of analysis can't provide new information about product safety beyond what is already documented.
Additionally, the original clinical trials by Pfizer and Moderna are now nearly 3 years old and COVID variants have changed the playing field. New trials would have to be conducted to determine if there's a cause and effect between the vaccines and the serious adverse events of special interest, which are pre-specified potential side effects.
In its discussion section, the peer-reviewed version of the article comes to this conclusion:
The excess risk of serious adverse events found in our study points to the need for formal harm-benefit analyses, particularly those that are stratified according to risk of serious COVID-19 outcomes. These analyses will require public release of participant level datasets.
In Dr. David Gorski's September 5, 2022, article "Peer review fail: Vaccine publishes antivax propaganda disguised as 'reanalyses' of Pfizer and Moderna COVID-19 vaccine clinical trial data," he challenged the conclusion the study's authors came to, singling out Peter Doshi, the senior editor at the BMJ (published by the British Medical Association) and a faculty member at the University of Maryland School of Pharmacy:
If you really want to know how safe and effective the vaccines are now (and have been over time), the original clinical trials are far from the best dataset to use, and this 'reanalysis' failed to find even a signal that there might have been a severe design defect or fraud in the original clinical trials. I'm all for data transparency and have no objections to releasing patient level data, but Doshi knows that this is not a trivial thing to do, as all the data have to be carefully de-identified and curated.
In a January 3, 2023, email to Lead Stories, the CDC Media Office said the public health agency stands by its data, which it says are under constant review:
Statements that imply that reports of deaths following vaccination equate to deaths caused by vaccination are scientifically inaccurate, misleading, and irresponsible. COVID-19 vaccines are undergoing the most intense safety monitoring in U.S. history. To date, CDC has not detected any unusual or unexpected patterns for deaths following immunization that would indicate that COVID vaccines are causing or contributing to deaths. CDC recommends that everyone who is eligible should get vaccinated.
COVID vaccines are recommended for everyone ages 6 months and older, according to the CDC website.
In a separate January 5, 2023, email, the public health agency added:
The adverse event reporting rate is only one of multiple factors that go into the decision to withdraw a vaccine from use - and those reasons will differ among vaccines. The reasons the 1976-77 influenza vaccine campaign was suspended were different from the reasons RotaShield was withdrawn from the market... which were different from the reasons why mRNA COVID-19 vaccines were preferentially recommended over the Ad26.COV2.S (Janssen/J&J) vaccine.
The FDA also responded to a Lead Stories query on the video and the study it was based on. In a January 3, 2023, email, it provided this statement:
FDA disagrees with the conclusions in the paper from Doshi et.al. Based on the agency's thorough evaluation of the safety and effectiveness data for the mRNA COVID-19 vaccines, as well as the ongoing safety surveillance of the vaccines, we continue to find their benefits far outweigh their risks in preventing COVID-19, including its most serious outcomes of hospitalization and death.
VAERS data
The ongoing safety surveillance and data on adverse events come from the Vaccine Adverse Event Reporting System (VAERS). It is co-managed by the FDA and the CDC as a crude early warning system and not as a database for the quantification of specific outcomes.
Anyone with internet access can add a report to the VAERS list of reports. The public access link to it expressly warns against unwarranted conclusions based on VAERS material because the list only provides a tally of unverified notes about any health event people experience after they are vaccinated.
The list itself cannot be used to prove or quantify, since all it shows is a chronological correlation, not the causal link that would be more difficult to establish. It's the equivalent of a police precinct's running "blotter" reports that may serve as a starting point for police work, not an endpoint.
Adverse reactions
In addition to the VAERS data, the CDC tracks selected adverse events that are reported after vaccination. In its January 5, 2023, email, the agency said:
Local and systemic adverse reactions to mRNA COVID-19 vaccines (such as pain at the injection site, fatigue, and fever) are commonly observed after these vaccines, and typically resolve within a few days of vaccination.
With regard to adverse events of special interest (AESI): the different vaccine surveillance systems have very similar lists of specific conditions (e.g., death, myocarditis, anaphylaxis) they consider AESIs; please see this site for more information (Emergency Preparedness and Vaccine Safety | Vaccine Safety | CDC ). To date, the only AESIs causally associated with COVID-19 vaccines are anaphylaxis (which occurs at rates comparable to other vaccines), myocarditis after mRNA COVID-19 vaccines (most frequently after dose 2, and mostly among males ages 12-39 years), and thrombosis with thrombocytopenia syndrome after the Janssen/J&J vaccine (Continued monitoring has identified nine deaths causally associated with J&J/Janssen COVID-19 vaccination).
Pfizer
In a January 3, 2023, email, Pfizer Media Relations told Lead Stories the pharmaceutical company stands by its coronavirus shot:
- Regulatory agencies across the globe have authorized the use of our COVID-19 vaccine. These authorizations are based on robust and independent evaluation of the scientific data on quality, safety and efficacy, including our landmark phase 3 clinical trial. Data from real world studies complement the clinical trial data and provide additional evidence that the vaccine provides effective protection against severe disease.
- Over the course of this deadly pandemic, mRNA vaccines have saved hundreds of thousands of lives, tens of billions of dollars in health care costs, and enabled people worldwide to go about their lives more freely. As of November 27th, 2022, Pfizer delivered 4.3 billion vaccines to 181 countries and territories in every region of the world.
Lead Stories also reached out to Moderna but had not received a reply at the time of writing. If one is provided, it will be added to the story.
In its January 5, 2023, email, the CDC gave a broader endorsement of the coronavirus shots. The agency said:
More than 660 million doses of COVID-19 vaccine have been given in the United States from December 14, 2020, through December 14, 2022. COVID-19 vaccines are safe and effective. COVID-19 vaccines were evaluated in tens of thousands of participants in clinical trials. The vaccines met the Food and Drug Administration's (FDA's) rigorous scientific standards for safety, effectiveness, and manufacturing quality needed to support emergency use authorization (EUA).
The Pfizer-BioNTech, Moderna, and Johnson & Johnson/Janssen and Novavax COVID-19 vaccines will continue to undergo the most intensive safety monitoring in US history. This monitoring includes using both established and new safety monitoring systems to make sure that COVID-19 vaccines are safe.
So far, research shows that the COVID-19 vaccines used in the United States continue to protect against severe disease, hospitalization, and death from known circulating variants. CDC will continue to monitor vaccine effectiveness to see what impact, if any, variants have on how well COVID-19 vaccines work in real-world conditions.
Playing with numbers
Abram L. Wagner, a research assistant professor of epidemiology at the University of Michigan, provided this interpretation of the article for Lead Stories in a September 12, 2022, email for a different story that looked at the same study:
The [study] compares the number of adverse events in those vaccinated compared to those not vaccinated. Even taking the results at face value, there needs to be a cautious interpretation of them. The data they analyzed included almost 40,000 individuals enrolled in clinical trials, and yet their results include wide confidence (or 'compatibility') intervals. This speaks to the very low number of adverse events as well as uncertainty in their analysis.
I have the following issues with the study. For one, they do not have access to individual-level data, and so have to compute statistics based on aggregate data. This limits a comparison of their data with what vaccine companies previously published. So in this [article], one individual reporting multiple adverse events over time is counted as multiple events, whereas in previous reports, this would just be one case of an adverse event. Their analysis therefore could result in double counting of events if they derive from a single adverse reaction to a vaccine.
Additionally, they modified the adverse events included in the original Bright Collaboration list. They do include the original list as a sensitivity analysis, and this sensitivity analysis shows a much broader, and therefore higher level of uncertainty in their data.
The authors of the study effectively created their own made-to-order list of AESI. Adverse events can be either serious or non-serious but the study only looks at what it calls the serious ones. It's a category defined by the authors of the study and wasn't used by government agencies when putting vaccines through the authorization process.
Wagner's email points to issues known in statistical analysis as p-hacking/selective reporting and data dredging/data fishing. Basically, it's a way of manipulating a data pool. Doing so can unearth something that got lost in the broader context, but it can also be used to show results that really weren't backed by the data.
Susan Oliver, a scientist who studies nanomedicine at The University of New South Wales in Australia, made a video called "Antivaxxers fooled by P-Hacking and apples to oranges comparisons." It exposes the methods Doshi used to reach his conclusions. The video is posted below:
Additional fact checks related to COVID-19 vaccines can be found here.
Updates:
-
2023-01-05T17:54:49Z 2023-01-05T17:54:49Z Adds response and information from the Centers for Disease Control and Prevention -
2023-01-04T16:15:39Z 2023-01-04T16:15:39Z Adds response and information from Pfizer Media Relations
