School closures have been widely implemented as a non-pharmaceutical intervention (NPI) to reduce the spread of COVID-19 caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). By April 2020, one month after the World Health Organization (WHO) characterised COVID-19 as a pandemic, 173 countries had closed schools, affecting 84.3% of the world’s enrolled students.
Yet school closures have broad impacts, including learning loss (as evidenced by the decrease in time spent learning and test scores, future earnings loss, deterioration of physical health, for example, cancellation of school meals and increase in weight) as well as mental health, maltreatment and lower maternal labour force participation, including health-care workers. Notably, these costs are disproportionately imposed on disadvantaged families, thereby widening social and economic inequality. Furthermore, school closures will lead to even long-term macroeconomic damage.
Accordingly, it is imperative to know whether the benefits of school closures outweigh these costs. Nonetheless, scholars have not reached a consensus on the degree of benefit, if any, to closing (or not reopening) schools. Some simulation and empirical studies show that school closures are effective in mitigating the spread of COVID-19. However, others fail to establish such statistically significant effects.
A collaborative team argues, in a study published in Nature Medicine, that one of the reasons why the literature is equivocal is methodological. Simulation studies assume parameters in their models, whose values may not be correct. Most empirical works estimate parameters (including the effect of school closures) by using publicly available aggregated data, although these studies are not necessarily rigorous in terms of causal inference.
A typical research design is panel regression: using a dataset that spans across countries and days, researchers regress the number of cases on a dummy variable to indicate whether a country closes its schools on a given day, where the coefficient of the dummy represents the effect of school closures. In essence, researchers estimate the effect of school closures by measuring the difference in the number of cases between days when a country closes and opens its schools. Many articles do not control for any other variables, while others include only a few control variables.
Therefore, readers should be concerned about dozens of potential confounders that affect both school closures and the number of cases (for example, share of children in the population, medical preparedness and the government’s fiscal situation), which would bias estimates of the effect of school closures. Relatedly, we cannot rule out the possibility of reverse causality, namely, that governments close their schools exactly because of a high number of COVID-19 cases among their residents.
If this is true, naive regressions with few control variables would underestimate the effect of school closures on infection.
Moreover, in essence, panel regression exploits variation in school closures across space and time, but school closures usually coincide with other NPIs (such as stay-at-home orders and prohibitions on gatherings) and/or are introduced simultaneously nationwide. Thus, it is challenging to disentangle the effect of school closures from those of other NPIs and/or from other contemporary factors such as season, economy and weather, especially when the unit of analysis is as large as a country or state.
To estimate the causal effect of school closures on reducing the spread of COVID-19, the team used data from Japan, where some municipalities closed their schools, while others did not. This variation was exploited in school closures among hundreds of municipalities by utilising matching techniques to account for dozens of confounders.
Study details
No causal effect of school closures in Japan on the spread of COVID-19 in spring 2020
Kentaro Fukumoto (Gakushuin University, Tokyo); Charles McClean (Harvard University/University of Michigan) & Kuninori Nakagawa (Shizuoka University, Japan)
Published in Nature Medicine on 27 October 2021
Abstract
Among tool kits to combat the coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2, school closures are one of the most frequent non-pharmaceutical interventions. However, school closures bring about substantial costs, such as learning loss. To date, studies have not reached a consensus about the effectiveness of these policies at mitigating community transmission, partly because they lack rigorous causal inference.
Here we assess the causal effect of school closures in Japan on reducing the spread of COVID-19 in spring 2020. By matching each municipality with open schools to a municipality with closed schools that is the most similar in terms of potential confounders, we can estimate how many cases the municipality with open schools would have had if it had closed its schools.
We do not find any evidence that school closures in Japan reduced the spread of COVID-19. Our null results suggest that policies on school closures should be reexamined given the potential negative consequences for children and parents.
See more from MedicalBrief archives:
Schools guidance to reflect new evidence on distancing in classrooms — CDC
COVID-19 in children: No room for complacency
Asia’s COVID tsunami threatens to engulf the Olympic Games
Masks helped limit COVID transmission in unvaccinated students — US study