Did the headline on this Facebook post correctly describe a draft report about the effect of lockdowns on the spread of COVID-19 in the U.S. and other countries? No, that's not true: The paper is, by its own wording, still a working draft and has not yet been subjected to professional peer review. That's the customary process through which the methodology and conclusions in draft scholarly writings like this are challenged by professional peers, often causing authors to restate their findings or back away from unsupportable declarations.
In this draft paper, the authors note lockdowns were implemented at the same time as other policies and in countries with imperfect data collection, which makes it impossible to isolate the effect of shelter-in-place orders on the ultimate death toll. While their draft conclusion is that death data suggest lockdowns did not flatten the curve, they concede cause and effect may be hard to establish and clearly say this is a working draft, not a completed study.
The headline declaring lockdowns backfired appears over a Facebook post (archived here) published on the Unbiased America account on June 24, 2021, under the title "NEW STUDY: THE LOCKDOWNS BACKFIRED, INCREASING DEATHS INSTEAD OF DECREASING THEM." It opened:
A new study examining deaths in the 50 states and across 44 countries has found that government lockdown orders resulted in higher overall deaths.
This is what the post looked like on Facebook at the time of writing:
(Source: Facebook screenshot taken on Fri Jun 25 18:02:11 2021 UTC)
The paper, by four public policy scholars, is part of the National Bureau of Economic Research (NBER) Working Papers program, which circulates draft research for comment and discussion. At the top of the first page, NBER includes this disclaimer:
NBER working papers are circulated for discussion and comment purposes. They have not been peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications.
NBER also says the views in the paper are those of the authors and do not necessarily reflect the views of NBER, which is a private organization of economists focused on research into public policy. NBER is funded by grants from government agencies like the National Institutes of Health; private foundations like the Bill & Melinda Gates Foundation; corporations like Johnson & Johnson; oil companies like ExxonMobil; by Google, and by major banks.
While the authors write that they found "excess deaths" (the difference between past year's patterns and the studied year's observed deaths) did not drop as expected after shelter-in-place (SIP) orders were put in place, the "Discussion" section of the draft paper outlines possible problems with concluding that lockdowns failed:
... we rely on total mortality, instead of cause-specific mortality. However, total mortality can also suffer from measurement error. For example, there could be a lag in registering deaths and upward revision of mortality data is common in many countries. It is also possible that deaths are simply undercounted especially in developing countries or rural areas due to lack of resources.
Second, enforcement and implementation of SIP policies could vary across countries or U.S. states ... Third, it is possible that SIP policies were implemented with other policies related to the pandemic and we cannot completely isolate the causal effects of SIP policies.
Finally, the counterfactual trajectory of the pandemic in the absence of SIP policies is difficult to estimate and might vary across countries and states, which could bias estimates.
The Facebook post, in addition to citing the draft paper as a "study" before it has been reviewed, revised and edited, does not include any of the caveats the authors provide about the limitations of their methods, which rely on statistical databases and not empirical research into the questions of unintended consequences of shelter in place.
