The academic journal Nature recently published two papers on the effects of non-pharmaceutical interventions (NPIs) on COVID-19. The first, from Imperial College London, argues that NPIs saved 3.1 million lives in 11 European countries. The second, from a team at Berkeley, argues that NPIs “prevented or delayed” 62 million confirmed cases in 6 countries, corresponding to 530 million infections.
The devil, however, is in the details, and neither paper comes to any very definite conclusions about which NPIs are responsible for the lives saved and the cases prevented or delayed.
In the Imperial paper, the 3.1 million figure that’s making headlines combines all interventions, which range from voluntarily social distancing to mandatory lockdowns. This leaves us with the obvious result that decreasing interpersonal contact reduces the spread of infectious diseases, but gives us little guidance in choosing among various voluntarily and mandatory interventions.
The paper compares infection rates before and after interventions for 11 European countries, ascribing observed changes to interventions. The authors admit that they discount voluntarily responses that can lower the rate without mandates. Indeed, all countries in their study experienced such drop-offs, including Sweden, which implemented far lighter restrictions than Canada. The lives saved thanks to mandatory measures is therefore likely far lower than 3.1 million.
More seriously, the Imperial study lacks the granularity for insight into which policy, specifically, “worked.” For example, they conclude that “lockdowns” significantly reduced infection rates. But which part of lockdowns: shuttering drive-in movie theatres, or banning social visits to senior centres?
The Berkeley paper is more rigorous, and is a good start at disaggregating which policies were actually “worth it,” and which were not. The paper notably includes voluntary distancing, so the 530 million figure is with zero public action at all, including simply recommending voluntary social distancing or quarantining inbound travellers.
The authors compared before-and-after rates, day by day, looking at how interventions affected growth in infections in 1700 local areas of China, Korea, Iran, Italy, France, and the US. Setting aside data quality in China (which they concede in the paper), this “local area” approach controls for a lot of factors left uncontrolled in the Imperial paper.
The disaggregated correlations suggest more limited policy packages might work better than near-total lockdowns. For example, they ascribe a 16% drop in infections in France to the entire package of interventions, but the specific intervention of “cancel events, no gatherings, other social distancing” scores a 22% reduction, beating the entire package. Similarly, for the US they estimate a 32% drop from the entire package, but 22% (2/3) of that is from “other social distancing,” with an unreported proportion being voluntary.
Some individual interventions the Berkeley team reported on yield very surprising results. In the US, infection growth was statistically worsened by closing schools, closing churches, and banning public gatherings. In Korea, infection growth was statistically worsened by quarantining inbound travellers. In Italy, infection growth was statistically worsened by bans on public gathering and mandates on social distancing and working from home. While the approach is good, these results are so counterintuitive that they may call into question the quality of the data itself.
In sum, both papers say “we should do something” in response to a pandemic like this one, but neither is very clear on what, exactly, are the best things to do. This is important given the costs of mandatory restrictions, measured in lives destroyed and, indeed, deaths due to ensuing mass unemployment, poverty, bankruptcies, and suicides.
As more detailed and scientifically robust data does become available, it will be important for both current and future interventions to be targeted appropriately and to be based on, we hope, ever more rigorous research.