STORY UPDATED: check for updates below.
Does a draft research paper refute the Food and Drug Administration (FDA) ban on using hydroxychloroquine to treat COVID-19, and does it prove the anti-malarial is a life-saving drug for COVID-19 patients? No, that's not yet true: The website where the "preprint" paper was released cautions that this early in the process, research reports may still contain errors and put forward information that has not yet been accepted or endorsed by the medical or scientific community. The author of a major study of the drug said the statistical analysis in this paper is wrong and will need to be revised. Even the leading U.S. advocate for hydroxychloroquine said making claims about the drug's effects from this kind of observational study is "rarely appropriate." He said the authors will need to seek the customary peer review of their work -- and revise it if mistakes or errors of analysis are spotted -- before the article will be ready for publication. The FDA first granted emergency approval for hydroxychloroquine as a COVID treatment, but then reversed that permission June 15, 2020, after investigators found evidence it caused serious heart problems. The ban had not been lifted as of June 8, 2021.
The claim originates from a video posted to the Laura Ingraham page (archived here) on June 2, 2021, where it was published under the title: "New study reveals success of #hydroxychloroquine as COVID treatmet" (sic).
It opened:
We have that data that Dr. Stephen Smith promised. In a new study, Smith and three other medical experts prove what he and this show have been telling you for more than a year, that hydroxychloroquine can save lives.
This is what the post looked like on Facebook at the time of writing:
(Source: Facebook screenshot taken on Tue Jun 8 23:55:51 2021 UTC)
The data referred to is a study of 255 COVID patients who were intubated, checking results for those who received hydroxychloroquine.
A hydroxychloroquine researcher with peer-reviewed work on the subject, Prof. David Boulware, M.D., Ph.D. of the Division of Infectious Diseases and International Medicine at the University of Minnesota Medical School, said in a June 7, 2021 email to Lead Stories that the statistical analysis done by Smith and his co-authors is invalid and that he would not expect it to be published in its current form, noting:
...Overall, 89% of persons received HCQ and many of those persons died in the first few days, yet those those deaths are not being counted in the >80mg/kg HCQ group (because they did not survive long enough). Thus, one is effectively comparing persons in the ICU who survive >10 days versus everyone else. Not surprisingly, those who survive >10 days inherently do better ... Absent a correct statistical analysis, it is impossible to know what these data mean.
Smith's "pre-print" report on his study was posted to medrxiv.org, an unedited, open site that allows researchers to post unchecked studies to seek reviews from fellow experts in their field. That is an important part of the certification process for scientific results. Called "peer review," the at-times-combative process weeds out poorly executed experiments, overly optimistic conclusions and faulty logic, making the final draft of the paper, if it holds up enough to be published in an edited and refereed journal, stronger than the preprint draft.
Despite medrxiv.org's warnings to journalists, Ingraham hyped the results as definitive.
"How many lives could have been saved, had people had an open mind about hydroxychloroquine," Fox News host Laura Ingraham asks her guest at two minutes, 57 seconds into the video.
"Ten thousand lives?" she asks Dr. Stephen Smith, one of the co-authors of the not-yet-accepted-for-publication article. "No, more than that," Smith replies.
"One hundred thousand?" she asks. At three minutes, 40 seconds in the video, Smith says:
Yeah, Laura. I hate to say things like that but yes. I mean the data. I just try and look at the data ... This study was really not just hydroxychloroquine we studied by any means. We looked at every medicine ever given, every lab value, every vital sign. We looked at prior visits to doctors, but hydroxychloroquine kept coming up.
Smith and his co-authors said they tracked 255 COVID-19 patients who were intubated to treat their symptoms: "Co-administered HCQ and AZM were associated with >100% increase in survival. According to our data, HCQ is not associated with prolongation."
Translated to common English, Smith and his co-authors say the combination of azithromycin and hydroxychloroquine doubled the survival rate of intubated COVID-19 patients and did not correlate to prolonged intubation. Their study looked particularly at heavier doses than had been suggested before, he told Ingraham during the tv interview.
But, MedRxIV.org cautions against overstating the significance of preprints, noting that science journals will only publish an article about new research once editors are satisfied the authors have addressed the criticisms and suggestions raised by expert peers to whom the research report has been referred for review:
Readers should therefore be aware that articles on medRxiv have not been finalized by authors, might contain errors, and report information that has not yet been accepted or endorsed in any way by the scientific or medical community.
We also urge journalists and other individuals who report on medical research to the general public to consider this when discussing work that appears on medRxiv preprints and emphasize it has yet to be evaluated by the medical community and the information presented may be erroneous.
The leading U.S. proponent of hydroxychloroquine as a treatment for COVID-19, Yale University School of Public Health professor Dr. Harvey Risch, raised those same questions when shown Smith and Ingraham's public statements about the study. In a June 8, 2021 email to Lead Stories, Risch said:
"...this study will evolve over time as I have heard that it has been submitted for publication and will get peer review criticisms back and will have to deal with them."
He said fact-checking the study itself was premature, since it has yet to be revised in response to critiques by experts.
This fact check addresses not the research methodology or conclusions, but the public claims made by Smith and Ingraham about the findings. Risch said those claims overstate what seems to have been documented by Smith and his peers, with Risch writing:
The bold statement is perhaps overstated in that it should have said that the weight-adjusted variables were associated with improved survival. Making statements of causation from single observational studies is rarely appropriate.
Smith has been, with Risch, a public proponent of treating COVID-19 with hydroxychloroquine, a long-in-use drug typically prescribed for malaria, arthritis, lupus and auto-immune diseases. Both have attributed some of the resistance to hydroxychloroquine to politics, since then-President Donald Trump retweeted claims that it would fight COVID-19 and his aides said he was taking it as a preventive.
In his email to Lead Stories, Risch said science journal publication is no longer the guarantee of validity it once was:
Publication today is a chaotic wild west and it's every reader for him/herself. If a reader cannot understand what the issues are in a preprint and figure out the paper's validity or lack, perhaps that person shouldn't be working in that scientific area in the first place.
With an M.D. from Yale Medical School and multiple post-doctoral fellowships and other training in infectious disease, Smith is the founder of the Smith Center for Infectious Diseases and Urban Health, a non-profit AIDS-focused clinic in East Orange, New Jersey.
Updates:
-
2021-06-10T18:22:16Z 2021-06-10T18:22:16Z Updated to include analysis by Dr. David Boulware.
