The perfect study for substack
Masks! Schools! Politics!
My impression of Substack is that it is a place where smart people rile up the public for money. I don’t want to rile you up, and this newsletter is free. Nevertheless, I think I have an idea of what sells: schools, masks, and politics.
Most people across the political spectrum have happily moved on from wearing or even thinking about masks. A few people, motivated by financial incentives or boredom or deeply-held convictions, have continued arguing relentlessly about masks online. Since the evidence generated has been spotty, these arguments don’t really go anywhere.
Now The New England Journal of Medicine has weighed in. A new study published this week in the venerable periodical looks at the potential effect of mask mandates on Boston-area public schools.
The study’s top-line conclusion is that mask mandates reduce the number of COVID cases in both staff and students. The study is not a randomized trial; it’s observational. That doesn’t mean it should be dismissed outright. There are many kinds of observational designs, and the conversation is much less interesting when our cynicism is so abrasive that it polishes away these nuances.
The Boston study uses a difference-in-difference model, a good method to detect causal associations when done properly. The time period of interest occurred in March-June 2022, just after Massachusetts rescinded a statewide school masking policy, allowing individual districts to put their own rules in place. Most schools lifted their mask mandates, although they did so at different times. A couple of districts kept the mandates.
Schools that chose to continue masking are going to be different in other ways from schools that chose not to. The study authors point out differences in demographics, building quality, and classroom density. There are surely others. The benefit of a diff-in-diff design is that these structural differences should be accounted for. What is more challenging to account for are time-varying confounders. That is, is there some other policy, behavior, or epidemiologic change that happened around the same time as the lifting of mask mandates—and did those changes happen differently across schools? The authors statistically adjust for a few important factors that can vary over time: testing rates, community case rates, and vaccination rates. I think seroprevalence (immunity from previous COVID exposure) would have been another good variable to consider.
The outcome is also worth dissecting a bit. Measured case rates are not a perfect reflection of true infection rates. Many COVID infections are undetected or unreported. One important concern raised about this study in the preprint phase was that some local policies recommended for or against testing exposed contacts depending on whether they were wearing masks. This rule could represent a major confounder that would be difficult to disentangle from the results. The authors address the issue in detail. They chose a time period in which this policy was no longer in widespread use. Based on the total number of COVID cases reported through this particular program, the researchers estimate that even in a worst-case scenario testing rules could only account for ~7% of the observed differences in COVID rates. But testing behavior is a specter hanging over any study that uses recorded cases as a measure of true infection level. Does masking influence personal testing choices? That could affect the results. At the same time, reported cases and true infection rate are clearly related, at least in a wealthy country with a lot of testing.
Taking a step back, we should acknowledge that even a sophisticated observational study such as this one will have a complicated analytic plan and vulnerability to idiosyncratic real-world conditions that make drawing firm conclusions impossible. Well-done studies that take place in other locations and time periods will add to our understanding of masking policy.
One such study comes out of Catalonia, Spain. This analysis took advantage of a natural experiment produced by an age-based masking rule. The authors found that school masking had no effect on COVID case rates. Pediatrician
contributed a provocative editorial that ran alongside the study. Munro comes down against masking in young kids. “For a group who has sacrificed so much already, despite being themselves at the lowest risk of adverse outcomes from infection,” he writes, “it is important to ensure a disproportionate burden is not placed on them any longer.”It is easy to say we need a randomized trial to resolve this question, but we needed a randomized trial. Schools are already a relatively well-regulated institution, at least compared to the chaos that takes place outside of their walls. They have the ability to institute and enforce policies, and many schools already had COVID testing programs in place. The design of such a study would not be trivial, but I don’t think it was insurmountable. If economists could find a way to randomize hundreds of thousands of Bangladeshis (that masking trial was considered positive, by the way), then schools seem like a more measured feat.
Masks would hardly be the riskiest intervention you could institute at schools. Consider the school iodine experiment performed by pathologist David Marine in the early 20th century:
Zimmermann MB. Research on iodine deficiency and goiter in the 19th and early 20th centuries. The Journal of Nutrition. 2008;138(11):2060-3.
Older studies, however, were unencumbered by many of the ethical rules we have in place today. A small matter.
Actually, this may rile you up…
Let me end with some political red meat in the NEJM masking study.
The authors spend their limited discussion space emphasizing the value of masking for health equity. That is, we know there are racial, social, economic, geographic, and disability characteristics that have caused some groups to bear a disproportionate burden of the bad pandemic outcomes. Masking can be a relatively easy way to reduce those disparities, the researchers argue.
Some readers will be instinctively offended by this open discussion of health disparities. It feels too “woke” or whatever. Addressing health disparities is good, though, especially when it is done in meaningful—and not just symbolic—ways. I would say it is even one of the highest callings of government and public health. (Incidentally, vaccines are the best way to objectively reduce COVID disparities.)
Yet the authors shoot themselves in the foot. Nestled within their discussion are a few unfortunate references to “capitalism.” Public health has already become so politicized during the pandemic. Why would you want to further contribute to that, undermining your own credibility as a scientific actor? Journalist
has referred to the habit public health experts have of subsuming tangentially-related political views into COVID policy as "stolen-base politics." This is a great example of that. A study of masking concludes by calling capitalism a “system of oppression.” I’m sure the authors believe it, but this venue doesn’t allow them the space they need to justify this controversial stance.After all, capitalism has brought billions of people out of poverty, improving their health and well being in concert. It can’t be dismissed outright in the same breath as racism and xenophobia like the authors attempt to do. And if your particular complaint is that schools have bad facilities and overcrowding, then your problem is not with the free market but with the function of government and democracy. Public schools are a lot closer to socialism than capitalism, at any rate. A well-designed study might help convince political leaders and regular Americans that mask mandates are effective, but positioning masks as anti-capitalist is going to cancel that right out.
A few other interesting studies and news stories:
Her response that it's important to raise awareness of structural racism was bizarre. Is there any NEJM reader in 2022 who isn't aware that leftists believe structural racism is a key issue?