--- title: "Learning Losses during the COVID-19 Pandemic: Evidence from Mexico" authors_and_venue: "with Enrique Alasino, María José Ramírez, Norbert Schady, and David Uribe - Economics of Education Review 98, 2024" venue: "Economics of Education Review" year: 2024 doi: "10.1016/j.econedurev.2023.102492" abstract: "This paper presents evidence of large learning losses and partial recovery in Guanajuato, Mexico, during and after the school closures related to the COVID-19 pandemic. Learning losses were estimated using administrative data from enrollment records and by comparing the results of a census-based standardized test administered to approximately 20,000 5th and 6th graders in: (a) March 2020 (a few weeks before school closed); (b) November 2021 (2 months after schools reopened); and (c) June of 2023 (21 months after schools re-opened and over three years after the pandemic started). On average, students performed 0.2 to 0.3 standard deviations lower in Spanish and math after schools reopened, equivalent to 0.66 to 0.87 years of schooling in Spanish and 0.87 to 1.05 years of schooling in math. By June of 2023, students were able to make up for 60% of the learning loss that built up during school closures but still scored 0.08–0.11 standard deviations below their pre-pandemic levels (equivalent to 0.23–0.36 years of schooling)." pdf_url: "https://mauricio-romero.com/pdfs/papers/GTO_Learning_Loss.pdf" canonical_url: "https://mauricio-romero.com/pdfs/papers/GTO_Learning_Loss.pdf.md" source: research.qmd note: >- Machine-readable Markdown version of the paper, generated for LLMs. --- Contents lists available at [ScienceDirect](https://www.elsevier.com/locate/econedurev) ## Economics of Education Review journal homepage: [www.elsevier.com/locate/econedurev](http://www.elsevier.com/locate/econedurev) ## Note # Learning losses during the COVID-19 pandemic: Evidence from Mexico[✩](#page-0-0) Enrique Alasino [a](#page-0-1) , María José Ramírez [a](#page-0-1) , Mauricio Romero [b](#page-0-2),[c](#page-0-3),[∗](#page-0-4) , Norbert Schady [a](#page-0-1) , David Uribe [d](#page-0-5) - a *World Bank, USA* - *ITAM, Mexico* - c *J-PAL, USA* - d *Secretaría de Educación de Guanajuato, Mexico* ## A R T I C L E I N F O *Keywords:* COVID-19 School closures Learning loss Recovery ## A B S T R A C T This paper presents evidence of large learning losses and partial recovery in Guanajuato, Mexico, during and after the school closures related to the COVID-19 pandemic. Learning losses were estimated using administrative data from enrollment records and by comparing the results of a census-based standardized test administered to approximately 20,000 5th and 6th graders in: (a) March 2020 (a few weeks before school closed); (b) November 2021 (2 months after schools reopened); and (c) June of 2023 (21 months after schools re-opened and over three years after the pandemic started). On average, students performed 0.2 to 0.3 standard deviations lower in Spanish and math after schools reopened, equivalent to 0.66 to 0.87 years of schooling in Spanish and 0.87 to 1.05 years of schooling in math. By June of 2023, students were able to make up for ∼60% of the learning loss that built up during school closures but still scored 0.08–0.11 standard deviations below their pre-pandemic levels (equivalent to 0.23–0.36 years of schooling). ## **1. Introduction** The COVID-19 pandemic led to sharp increases in mortality around the world — by December 2021, it had caused 18.2 million excess deaths globally [\(Wang et al.,](#page-8-0) [2022](#page-8-0)). Lockdowns slowed economic activity worldwide and caused widespread unemployment and inactivity. Global GDP shrank by 4.3 percent in 2020, and 70 million more people were living in extreme poverty in 2020 compared to the previous year [\(World Bank,](#page-8-1) [2022](#page-8-1)). In addition, schools closed in most countries. In low- and middle-income countries, roughly a billion children missed at least one full year of in-person education ([Schady, Holla, Sabarwal,](#page-8-2) [Silva, & Yi Chang](#page-8-2), [2023](#page-8-2)). This paper adds to a small but growing literature on the effect of the pandemic on student achievement. We study learning losses during the pandemic and the pace of recovery among 5th and 6th graders in the Mexican state of Guanajuato. When the pandemic first hit the country, the government quickly closed schools to slow the spread of the virus. Schools in Mexico were particularly slow to reopen — school closures in Mexico lasted 66 weeks, compared to an average of 32 weeks in low- and middle-income countries and 25 weeks in high-income countries [\(UNESCO](#page-8-3), [2021b](#page-8-3)).[1](#page-0-6) Mexico is, therefore, a particularly suitable setting to study the impact of the pandemic on student learning in the context of prolonged school closures. Moreover, Guanajuato has high-quality and comparable achievement data for 5th and 6th grade students measured at three key points in time: (a) immediately before schools closed (March 2020), (b) two months after they reopened (November 2021), and (c) over a year and a half after schools re-opened (June 2023). We use these data to estimate learning losses and the pace of recovery. We show that shortly after schools re-opened (November 2021) test scores of 5th and 6th graders in Guanajuato were 0.22 to 0.32 standard deviations below their pre-pandemic level, depending on the grade and subject. This is equivalent to a loss of 0.66–0.75 school years in Spanish and 0.87–1.05 school years in math. We cannot establish whether the decline in test scores is only a result of the school closures or also a consequence of other negative effects of the pandemic on achievement (e.g., negative income or health shocks). Regardless, since learning ✩ This study was possible thanks to the support of the Secretar 'ia de Educación de Guanajuato. The views expressed here are those of the authors alone and do not necessarily reflect the World Bank's opinions or those of the Secteraría de Educación Pública of Guanajuato. Romero gratefully acknowledges financial support from the Asociación Mexicana de Cultura, A.C. All errors are our own. ∗ Corresponding author. *E-mail address:* [mtromero@itam.mx](mailto:mtromero@itam.mx) (M. Romero). 1 These numbers refer to simple averages across countries, but the student-weighted numbers are similar: 27 weeks of missed school for the typical student in all low- and middle-income countries and 26 weeks in high-income countries. We use the World Bank's definition of low-, middle-, and high-income countries. When UNESCO reports that schools were ''partly open'', we assume they were open 50 percent of the time. achievement, not just educational attainment, is a powerful predictor of earnings and economic growth ([Hanushek & Woessmann,](#page-7-0) [2008\)](#page-7-0), these declines in learning could have long-lasting negative consequences. Next, we estimate the pace of recovery. Over three years after the pandemic started, students still scored 0.08–0.11 standard deviations below their pre-pandemic levels (equivalent to 0.23–0.36 years of schooling). That is, after 19 months (between November 2021 and June 2023), students were able to make up for 63% and 57% of the learning loss in Spanish and math that took place during school closures. Our paper makes three contributions to the literature on how the pandemic affected learning. Most papers on this topic have focused on high-income countries. For example, a recent comprehensive review by [Betthäuser, Bach-Mortensen, and Engzell](#page-7-1) ([2023\)](#page-7-1) concludes that ''there is a dearth of studies [on learning losses] from middle-income countries and no studies from low-income countries''.[2](#page-1-0) However, it is not ex ante clear how the findings from high-income settings would translate to poorer countries, which often have limited internet connectivity, and where parents frequently have low schooling levels, which makes it difficult for them to help their children with online classes. Our paper adds to the limited evidence on learning losses in low- and middle-income countries. Second, our results highlight the importance of carefully considering the comparability of the populations being studied to avoid biases in the results. For example, among the four studies from middle-income countries ([Betthäuser et al.,](#page-7-1) [2023\)](#page-7-1) review (compared to 38 from highincome countries), only one has a low risk of bias according to the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) tool [\(Sterne et al.](#page-8-4), [2016\)](#page-8-4) — [Lichand, Doria, Leal-Neto, and Fernandes](#page-8-5) ([2022\)](#page-8-5), who study learning losses in Sao Paulo, Brazil, and report learning losses of 0.32 standard deviations, equivalent to 0.75 school years — and one has a moderate risk — [Ardington, Wills, and Kotze](#page-7-2) ([2021\)](#page-7-2) who report large learning losses among 2nd-grade students in South Africa. Our study avoids many of the pitfalls of studies with serious or critical risks of bias for several reasons.[3](#page-1-1) First, by using data from all students in six municipalities in Guanajuato, we mitigate the risk of selection bias in our sample, as neither schools nor students are selected for convenience nor allowed to self-select into the data. Second, we explicitly account for the fact that the pandemic could have affected student enrollment and attendance – not just test scores – and that differential attrition patterns could bias our estimates (mitigating the risk of bias due to missing data). Because approximately 6 percentage points fewer students took the test in 2021 than in 2020, we calculate [\(Lee](#page-8-6), [2009\)](#page-8-6) bounds on learning losses. Missing children are likely drawn primarily from the bottom of the achievement distribution, which would mean that our estimates of the effects of the pandemic on learning would be downward-biased. However, our estimates of learning losses are significant even when we use a very conservative lower-bound estimate assuming that the missing children are drawn from the top of the distribution. Third, by having comprehensive and directly comparable measures of student learning across time, we mitigate bias in outcome measurement. Finally, we explicitly account for the difference in the timing of the test (with respect to the school year) in our measures of learning loss, which mitigates the risk of bias due to confounding. Finally, we join the few papers that have studied the pace of recovery. Like India [\(Singh et al.,](#page-8-7) [2022\)](#page-8-7) and Brazil ([Lichand & Alberto Doria](#page-8-8), [2022\)](#page-8-8), we find evidence of some learning recovery. In India, students recovered 66% of the learning loss within 5–6 months after schools reopened; in Brazil, they recovered 25% of the losses after a year of returning to in-person classes. We document that over a year and a half after schools re-opened, students in Mexico recovered ∼60% of the losses during school closures. ### **2. Context** ### *2.1. Setting* Guanajuato, located in central Mexico, is the 17th poorest of its 32 states; it is divided into 46 municipalities. Over a quarter (28 percent) of its 6 million inhabitants live in rural areas. Children in Guanajuato enjoy universal access to primary education, reflected in high gross (102 percent) and net (97 percent) enrollment rates. The school system is organized into preschool (ages 3 to 5), primary (*primaria*, grades 1 to 6), lower secondary (*secundaria*, grades 7 to 9), and upper secondary (*media superior*, grades 10 to 12). A total of ∼1.2 million students are enrolled in ∼10,500 schools in the state, ∼80 percent of which are public ([Secretaría de Educación de Guanajuato](#page-8-9), [2021\)](#page-8-9). Seventy percent of students attend large urban schools, but ∼65 percent of schools are small rural multigrade schools that teach students from different grades in the same classroom. Despite universal access to primary education, Guanajuato struggles to meet national learning standards. Before the pandemic, learning levels in the state were below the minimum expected national standards (but similar to the national average). In the national standardized assessment (*Plan Nacional para la Evaluación de los Aprendizajes* or PLANEA) administered in 2018, 62 percent and 50 percent of 6th graders scored ''insufficient'' in math and language, respectively. The proportion of students with ''insufficient'' scores was larger among students from poorer municipalities, around 70 percent. Mexican primary school students outperformed their Latin American peers on the 2019 *Estudio Regional Comparativo Explicativo* (ERCE), a regional assessment organized by UNESCO [\(UNESCO](#page-8-10), [2021a\)](#page-8-10). However, Mexico was among the lowest-performing countries participating in the 2018 Program for International Assessment (PISA), an OECD assessment for secondary school students ([OECD,](#page-8-11) [2019](#page-8-11)). Mexico was among the countries most affected by the pandemic, with a per capita excess death rate twice as high as that of the median country ([Wang et al.](#page-8-0), [2022](#page-8-0)). The Mexican government closed schools soon after the first cases of COVID-19 to slow the spread of the virus. The country's school closures were among the longest in the world ([UNESCO,](#page-8-3) [2021b\)](#page-8-3). In Guanajuato, schools closed from March 2020 to September 2021 for 18 calendar months or 13 academic months (see [Fig.](#page-2-0) [1](#page-2-0)). Following national guidelines, Guanajuato shifted to distance education for the remainder of the 2019/20 school year and the 2020/21 school year. As with most school systems worldwide, Guanajuato (and, more broadly, Mexico) was unprepared to deal with such long school closures. The national government's response to school closures was to offer TV education (officially known as ''Aprende en Casa'', which translates to ''Learning at Home''). The program consisted of 1.5 h of televised lessons each day for primary grades. In comparison, 91% of 2 After the review by [Betthäuser et al.](#page-7-1) [\(2023](#page-7-1)) a few studies from low-income countries have appeared. For example, [Singh, Romero, and Muralidharan](#page-8-7) [\(2022](#page-8-7)) analyze learning losses due to COVID-19 school closures in rural Tamil Nadu, India, and estimate these to be 0.34 standard deviations in language and 0.7 standard deviations in math. Similarly, [Guariso and Björkman Nyqvist](#page-7-3) [\(2023](#page-7-3)) show learning loss using a panel of children in public schools in Assam, India. 3 In contrast, [Betthäuser et al.](#page-7-1) [\(2023](#page-7-1)) classify earlier work on learning losses in Mexico ([Hevia, Vergara-Lope, Velásquez-Durán, & Calderón,](#page-7-4) [2022\)](#page-7-4), as being at serious risk of bias. Specifically, [Hevia et al.](#page-7-4) ([2022\)](#page-7-4) report learning losses in two states in Mexico (Campeche and Yucatán) of 0.34–0.45 standard deviations in language and 0.62–0.82 standard deviations in math. However, their 2019 sample was drawn from the results of a survey representative at the state level, while the 2021 sample was drawn from a list of beneficiaries of a social welfare program. The latter are likely to be considerably poorer than the average family in both states, so the estimated ''learning losses'' they report, which are large relative to those found in other middle-income countries, could be partly because the students in the 2021 sample were poorer. Moreover, their learning estimates are based on short tests (four questions in Spanish, five in math) and therefore cover little of the curriculum. students in Guanajuato attend public schools with school days lasting between 4.5 and 8 h in the absence of the pandemic.[4](#page-2-1) The lessons were transmitted on the public TV channel (Canal 11) but were also available on YouTube [\(https://www.youtube.com/](https://www.youtube.com/c/aprendeencasa) [c/aprendeencasa](https://www.youtube.com/c/aprendeencasa)). In addition, the government gave teachers some resources to plan remote lessons.[5](#page-2-2) Teachers were expected to keep in contact with students through phone calls, text messaging, video calls, or other digital platforms. There was no other official response at the national level. Distance education had not previously been part of the school culture or curriculum. For example, compared to other OECD countries, Mexico is well below the average in terms of principals reporting having an online platform available in their schools or having effective resources for teachers to learn how to use ICT ([OECD,](#page-8-12) [2020](#page-8-12)). Teacher training and technological infrastructure in schools, and in teachers' and students' homes, were also insufficient. For example, only 55 percent of students' households had internet access during the pandemic ([Secretaría de Educación de Guanajuato](#page-8-13), [2022](#page-8-13)). More broadly, compared to other OECD countries, children in Mexico were the least likely to report having a computer they can use for school work or having a quiet place to study ([OECD,](#page-8-12) [2020\)](#page-8-12). Overall, evidence from parent, student, and teacher surveys performed during the pandemic suggests that many teachers did not keep in touch with their students and faced considerable challenges in continuing instruction (remotely) during school closures ([Cárde](#page-7-5)[nas, Lomelí, & Ruelas](#page-7-5), [2022](#page-7-5)). Not surprisingly, time use surveys suggest students spent 30% less time on their studies than before the pandemic ([Boruchowicz, Parker, & Robbins,](#page-7-6) [2022](#page-7-6)). #### *2.2. Data* To measure learning losses, we use learning assessment data from six municipalities in the central-south region of Guanajuato: Jaral del Progreso, Moroleón, Salamanca, Uriangato, Valle de Santiago, and Yuriria. The government piloted an annual, state-level standardized test known as the *Recopilación de Información para la Mejora de los Aprendizajes* (RIMA) in these six municipalities during the 2019/20 school year before the closures (March 2020). Appendix Figure A.1 shows where Guanajuato is located within Mexico, and where the six municipalities are within the state. The tests were designed to give teachers feedback on their students' learning levels rather than to serve as an accountability measure ([Secretaría de Educación de Guanajuato](#page-8-13), [2022\)](#page-8-13). The assessment was administered to over 20,000 students enrolled in 442 public and private schools in these municipalities and was later scaled up during the 2021/22 school year to test every student in the state (November 2021). Students in the whole state were tested again at the end of the 2022/23 school year (June 2023). Consequently, even though data are available for the whole state after schools reopened, only these six municipalities have data from both before and after the closures. On average, the 2021 and 2023 test scores of children in the six sample municipalities were slightly higher than in other municipalities in the state. However, the differences are generally modest, between 0.07 and 0.1 standard deviations (see Appendix Tables A.1 and A.2). Each assessment takes place in a different school year. Thus, different cohorts of students were tested in each round. The exams, however, **Fig. 1.** Timeline of school closures in the state of Guanajuato, Mexico. were identical across grades (5th and 6th) and across the first two rounds (2020 and 2021).[6](#page-2-3) In 2023, the exam for Grade 5 students had some overlapping questions with the previous assessments, allowing us to study the pace of recovery for students in this grade.[7](#page-2-4) The Spanish test measures reading comprehension and grammar, while the math test covers numbers, operations, measurement, geometry, and data analysis. In 2020 and 2021, each subject (math and Spanish) included 50 multiple-choice questions with four response options. In 2023, each subject included 40 questions with four response options, of which 15 were taken from the 2020/2021 assessment and served as anchoring items. Review panels ensured the tests were aligned with the national curriculum. The tests were in printed format (paper 4 Private schools have freedom to choose how long their school day lasts. The length of the day in public schools depends on whether the school operates two shifts (with each shift lasting 4.5 h) or a single shift (lasting 8 h) through the Programa de Escuelas de Tiempo Completo ([Padilla-Romo,](#page-8-14) [2022\)](#page-8-14). 5 See for more details of the ''Aprende en Casa'' program. Specifically, for the resources available for teachers, see and [https://](https://aprendeencasa.sep.gob.mx/fichas-de-clase/) [aprendeencasa.sep.gob.mx/fichas-de-clase/](https://aprendeencasa.sep.gob.mx/fichas-de-clase/). 6 Two school years separate the first two assessment rounds. Thus, most 5th grade students assessed in 2020 would have progressed to 7th grade by the time students were assessed in 2021. 7 The exam for Grade 6 students in 2023 has no overlapping questions with the previous exams for grades 5 and 6 or with the exam for grade 5 in 2023. Thus, we exclude grade 6 students in 2023 from our analysis since performance in this exam is not comparable to performance in any other exam in our data. and pencil). Students select their answers by filling in the associated circles on a separate machine-readable response sheet. Teachers were trained to administer the tests to their students following standardized procedures.8 The tests meet commonly accepted psychometric standards. However, we recomputed test scores after removing six questions from the 2020/2021 tests with poor psychometric properties (see Appendix B for more details). Given the multiple-choice nature of the questions, we used a three-parameter logistic item response theory (3PL IRT) model to compute test scores using the mirt package in R (Chalmers, 2012). IRT models were estimated across grades and time and normalized to have a mean of zero and a standard deviation of one for students in 5th grade in 2020. We also present, as robustness checks, results using an IRT model computed only over the 15 common items across all three rounds and the percentage of correct answers. In addition to the assessment results, we use enrollment rosters to track student enrollment and attendance on the day of the test. ### 3. Estimation strategy To measure learning losses, we use the following estimation equation: $$Y_{ist} = \alpha_0 + \alpha_1 Grade6_{it} + \alpha_2 \mathbb{1}_{2021} + \alpha_3 Grade6_{it} \times \mathbb{1}_{2021} + \alpha_4 \mathbb{1}_{2023} + \varepsilon_{ist} \quad (1)$$ where $Y_{ist}$ is the test score of student i enrolled in school s at time t, $Grade6_{it}$ is a dummy variable indicating that the student is enrolled in 6th grade (as opposed to 5th grade), $\mathbb{1}_{2021}$ is an indicator for whether the test was taken in 2021, $\mathbb{1}_{2023}$ is an indicator for whether the test was taken in 2023, and $\varepsilon_{ist}$ is an error term. Standard errors are clustered at the school level. In a variation of this model, we include school fixed effects to estimate learning losses using the differences in performance over time within the same school. In this specification, $\alpha_0$ is the average score for students in the 5th grade in 2019/20. The $\alpha_1$ coefficient indicates the normal progression between grades before schools closed, which measures how much students learn in a typical year of schooling. The $\alpha_2$ coefficient denotes the difference between the scores of 5th grade students in the 2021/22 school year and 5th grade students in 2019/20, while $\alpha_2 + \alpha_3$ indicates the comparison across the 2021/22 and 2019/20 academic years for students in the 6th grade. Finally, $\alpha_4$ denotes the difference between the scores of 5th grade students in the 2022/23 school year and 5th grade students in 2019/20. To accurately estimate learning losses, it is crucial to account for the time of the year in which the test was administered. In the 2019/20 school year, students were tested in March 2020, towards the end of the school year or after 7 months of instruction. In 2021/22, students were tested in November 2021, towards the beginning of the school year, after only 3 months of instruction. In 2022/23, students were tested in June 2023, at the end of the school year, after 9 months of instruction.9 Scores are, therefore, expected to be lower in 2021 compared to 2020, even without any learning loss. Since the school year lasts approximately 10 months, students tested in 2021/22 had 40 percent less instruction time before the test than those tested in 2019/20. Thus, under a linearity assumption, 2021/22 test scores should be roughly 60 percent of what they were in 2019/20. We calculate learning loss by taking the difference between the actual and expected scores. For 5th grade, the actual score is $\alpha_0 + \alpha_2$ and the expected score is $\alpha_0 - 0.4\alpha_1$ . Therefore, learning loss by the time schools re-opened in 5th grade can be expressed as $\alpha_2 + 0.4\alpha_1$ . The learning loss in 6th grade by the time schools re-opened can be similarly calculated with $\alpha_2+\alpha_3+0.4\alpha_1$ (the actual score is $\alpha_0+\alpha_1+\alpha_2+\alpha_3$ , while the expected score is $\alpha_0+0.6\alpha_1$ ). Similarly, in the absence of any learning loss, we expect scores in 2023 to be higher than those in 2020 since students received two more months of instruction before the test. We calculate the learning loss after three years by taking the difference between the actual and expected scores (under a linearity assumption). We can only do this for 5th grade since students in Grade 6 were not tested with a comparable assessment in 2023. For 5th grade, the actual score in 2023 is $\alpha_0+\alpha_4$ and the expected score is $\alpha_0+0.2\alpha_1$ . Therefore, learning loss in 5th grade, three years after the pandemic started, can be expressed as $\alpha_4-0.2\alpha_1$ . $^{10}$ We also benchmark learning losses against learning in a typical school year before the pandemic. We present these results in equivalent years of schooling (EYOS), as in Evans and Yuan (2019), by dividing the learning loss by the expected normal progression. Thus, the equation for calculating learning loss by the time schools re-opened in EYOS for 5th grade is $\frac{\alpha_2+0.4\alpha_1}{\alpha_1}$ and for 6th grade it is $\frac{\alpha_2+\alpha_3+0.4\alpha_1}{\alpha_1}$ . The learning loss three years after the pandemic started for 5th grade is $\frac{\alpha_4-0.2\alpha_1}{\alpha_1}$ . While there was no substantial change in the number of students enrolled across years, fewer students took the RIMA assessment in 2021, shortly after schools reopened. In contrast, by 2023, more students took the RIMA assessment. 12 In the 2019/20 school year, 92 percent of students enrolled were present on the day of the test, but in the 2021/22 school year, this proportion was 86 percent in the six municipalities in our sample (see Table 1). In 2022/23, 96% of 5th grade students enrolled were present on the day of the test. Thus, our measures of learning loss might be biased if the absent students were not randomly distributed along the learning achievement distribution. We compute (Lee, 2009) bounds to assess how this could affect our estimates. The lower bound loss is estimated by excluding the highest-performing students (by the differential attrition across years) from the 2020 and 2023 sample and the upper bound by excluding the lowest-performing students. #### 4. Results ## 4.1. Descriptive evidence Fig. 2 displays the changes in the test score distribution before and after school closures for the 5th and 6th grades. Overall, the distribution of scores in the 2021/22 school year is well below that of 2019/2020. Further, the distribution of scores for 6th graders is similar to that of 5th graders in the 2019/20 school year (panels C and F), suggesting that the learning losses were equivalent to approximately 8 State authorities conducted unannounced quality control visits in a random sample of 96 schools to verify that the tests were carried out as intended and found no major issues. $^{9}$ June corresponds to 9 months of instruction when school breaks are considered. <sup>10 Learning may not be "linear" throughout the school year. Instead, students may learn more in the first or the last few months of the school year. For this reason, we estimate learning losses under different assumptions of how much learning typically occurs (in the absence of the pandemic) in the months between the exams relative to the whole academic year. <sup>11 Presenting results in terms of standard deviations has some pitfalls, as the size of the effects depends heavily on the design of the test (Singh, 2015). However, presenting results in equivalent years of schooling has other pitfalls (Baird & Pane, 2019; Kraft, 2020). For transparency, we present results in various ways by using the percent of correct answers and the IRT score as performance measures and by presenting results in absolute performance, changes in standard deviations, and equivalent years of schooling. Further, we provide an extensive review of the psychometric properties of our test in Appendix B. In the 2019/20 school year, 11,761 students were enrolled in 5th grade and 11,774 were enrolled in 6th grade. In the 2020/21 school year these figures were 11,949 and 12,050. In the 2022/23 school year 11,508 students were enrolled in Grade 5. In 2020/21, compared to 2019/20, there are 0.25 more students per school. In 2022/23, compared to 2019/20, there are 0.57 fewer students per school. **Table 1** Descriptive statistics on enrollment, attrition from the test, and test performance, before and after school closures. | | (1)
2020 | (2)
2021 | (3)
2023 | (4)
2021–2020 | (5)
2023–2020 | (6)
2023–2021 | |--------------------|-------------|-------------|-------------|------------------|------------------|------------------| | | | | | | | | | Panel A: Grade 5 | | | | | | | | Spanish: % correct | 0.55 | 0.48 | 0.57 | −0.07*** | 0.02** | 0.08*** | | | (0.26) | (0.24) | (0.25) | (0.01) | (0.01) | (0.01) | | Math: % correct | 0.45 | 0.38 | 0.45 | −0.07*** | 0.00 | 0.07*** | | | (0.23) | (0.21) | (0.22) | (0.01) | (0.01) | (0.01) | | Spanish: IRT | 0.00 | −0.35 | −0.01 | −0.35*** | −0.01 | 0.34*** | | | (1.00) | (0.92) | (1.03) | (0.03) | (0.03) | (0.03) | | Math: IRT | 0.00 | −0.38 | −0.05 | −0.39*** | −0.05 | 0.33*** | | | (1.00) | (0.94) | (1.01) | (0.04) | (0.04) | (0.03) | | P(Tested) | 0.90 | 0.86 | 0.96 | −0.04*** | 0.06*** | 0.10*** | | | (0.30) | (0.35) | (0.21) | (0.00) | (0.00) | (0.00) | | Enrolled | 11,761 | 11,949 | 11,508 | 188 | −253 | −441 | | Tested | 10,590 | 10,246 | 11,001 | −344 | 411 | 755 | | Panel B: Grade 6 | | | | | | | | Spanish: % correct | 0.63 | 0.56 | | −0.08*** | | | | | (0.26) | (0.26) | | (0.01) | | | | Math: % correct | 0.52 | 0.42 | | −0.09*** | | | | | (0.24) | (0.22) | | (0.01) | | | | Spanish: IRT | 0.34 | −0.05 | | −0.39*** | | | | | (1.05) | (1.01) | | (0.03) | | | | Math: IRT | 0.31 | −0.14 | | −0.44*** | | | | | (1.05) | (1.00) | | (0.04) | | | | P(Tested) | 0.93 | 0.85 | | −0.08*** | | | | | (0.25) | (0.35) | | (0.01) | | | | Enrolled | 11,774 | 12,050 | | 276 | | | | Tested | 10,974 | 10,299 | | −675 | | | *Notes*: This table presents in the first eight rows of each panel the average scores and their standard deviation (in parenthesis) for students who took the test in March of 2020 (Column 1) and in November of 2021 (Column 2), as well as the difference (Column 3) and the standard error of the difference (in parenthesis). The ninth and tenth row have the probability of an enrolled student being tested, and the standard deviation in parenthesis. The last two rows have the number of enrolled students and the number of students tested. Panel A has data for Grade 5 students, while Panel B has data for Grade 6 students. Standard errors are clustered at the school level. Statistical significance at the 1, 5, 10% levels is indicated by \*\*\*, \*\*, and \*. one year of schooling. In both grades and subjects, the cumulative distribution of test scores in 2020 first-order stochastically dominates the 2021 distribution (see Appendix Figure A.2). However, the distribution of scores in 2022/2023 for 5th grade students – three years after the pandemic started – is similar to that of 2019/2020, suggesting a marked recovery a little under two years after schools reopened. ## *4.2. Estimates of learning losses and recovery* Before the school closures, 6th grade students scored 0.34 standard deviations higher in Spanish and 0.30 standard deviations higher in math than those in 5th grade. Shortly after schools reopened in 2021, students in the 5th grade scored 0.35 standard deviations lower in Spanish and 0.38 standard deviations lower in math than their counterparts in the same grade before schools closed. Furthermore, students in 6th grade scored 0.38 and 0.44 standard deviations lower in Spanish and math, respectively, shortly after schools reopened compared to those who were tested before the schools closed. However, there is a noticeable recovery by 2023. Students enrolled in Grade 5 in 2023 scored 0.0095 and 0.049 standard deviations below in Spanish and math than students in the same grade before schools closed (Columns 1 and 3, [Table](#page-6-0) [2\)](#page-6-0). When the difference in the time of the year in which the tests were administered is taken into account, the learning loss in 5th grade by November of 2021 (20 months after school closures began) is 0.22 standard deviations in Spanish and 0.26 in math, while in 6th grade it is 0.25 standard deviations in Spanish and 0.32 in math. This corresponds to 0.65 and 0.86 equivalent years of schooling for 5th grade students in Spanish and math, respectively, and 0.75 and 1.05 for 6th graders. These losses are above those found in most high-income countries, but in line with findings from middle-income countries ([Betthäuser](#page-7-1) [et al.,](#page-7-1) [2023\)](#page-7-1).[13](#page-4-1) This is consistent with low- and middle-income countries having more prolonged school closures during the pandemic [\(UNESCO](#page-8-3), [2021b\)](#page-8-3). By June of 2023, 39 months after school closures began and 21 months after schools reopened, learning losses persist, although with signs of recovery. Compared to the learning levels before the pandemic, 5th grade students scored 0.08 standard deviations below in Spanish and 0.11 standard deviations below in math. The loss is equivalent to 0.23 and 0.36 years of schooling in Spanish and math, respectively. Thus, after 19 months (between November 2021 and June 2023) students were able to make up for 63% of the learning loss in Spanish and 57% of the loss in math. Our finding that learning losses were greater in math than in language in Guanajuato is consistent with what has been reported elsewhere [\(Betthäuser et al.,](#page-7-1) [2023\)](#page-7-1) — including in India [\(Guariso &](#page-7-3) [Björkman Nyqvist](#page-7-3), [2023;](#page-7-3) [Singh et al.](#page-8-7), [2022\)](#page-8-7) and the U.S. [Goldhaber](#page-7-11) [et al.](#page-7-11) ([2022\)](#page-7-11), [Halloran, Jack, Okun, and Oster](#page-7-12) [\(2021](#page-7-12)) and [Kuhfeld and](#page-7-13) [Lewis](#page-7-13) ([2022\)](#page-7-13). Likewise, our findings that recovery is faster in Language than in math echoes findings from Brazil [\(Lichand & Alberto Doria](#page-8-8), [2022\)](#page-8-8) where recovery in Portuguese has been faster than in math — but contrasts findings from India where recovery in math has been faster than in Hindi [\(Singh et al.,](#page-8-7) [2022](#page-8-7)). There is some evidence of heterogeneity, especially in terms of recovery. Girls generally have higher test scores than boys, but suffered from larger learning losses during the pandemic and recovery has been slower (Appendix Table A.5). While we are unable to study heterogeneity by students' socioeconomic status – no data on student characteristics was collected in 2020 – we can study heterogeneity 13 While our learning loss estimates are not as large as those found by [Hevia](#page-7-4) [et al.](#page-7-4) [\(2022](#page-7-4)) in Mexico, our estimates are not directly comparable. Campeche and Yucatan are much poorer states than Guanajuato. In addition, their sample includes all children aged 10–15, while we focus on Grades 5 and 6 (roughly 11- and 12-year-olds). Finally, given the differences in the test instruments mentioned above, the effects (in standard deviations) are not directly comparable (see [Singh](#page-8-15) ([2015\)](#page-8-15)). **Fig. 2.** Distribution of test scores in the Spanish and math tests, before schools close (March 2020), shortly after they reopened (November 2021), and over a year and a half after they reopened (June 2023). Note: These figures present the distribution of IRT scores across grades and time. [Fig.](#page-5-1) [2\(a\)](#page-5-1) compares the distribution of IRT scores for Grade 5 students in the Spanish test in 2020, 2021, and 2023, while [Fig.](#page-5-2) [2\(d\)](#page-5-2) does the same for the math test. [Fig.](#page-5-2) [2\(b\)](#page-5-2) compares the distribution of IRT scores for Grade 6 students in the Spanish test in 2020 and 2021, while [Fig.](#page-5-2) [2\(e\)](#page-5-2) does the same for the math test. [Figs.](#page-5-2) [2\(c\)](#page-5-2) and [2\(f\)](#page-5-2) compare the distribution of IRT scores for Spanish and math of Grade 5 students in 2020 to those enrolled in Grade 6 in 2021. Vertical lines indicate the median score for each group. See [Table](#page-4-0) [1](#page-4-0) for more statistics on the average performance of students in different grades at different times. by school characteristics that proxy the average socioeconomic status of students. For example, there are no differences in learning losses shortly after schools reopened between multigrade schools – which teach students from different grades in the same classroom and tend to be located in poorer and more remote regions – and regular schools. However, by 2023 regular schools show signs of recovery, while multigrade schools exhibit no signs of recovery (Appendix Table A.6). In terms of school management, private and public schools have similar learning losses in 2021 (Appendix Table A.7), but students in private schools appear to recover faster (albeit the difference is not statistically significant). Similarly, there were no differences in learning losses in 2021 between rural and urban schools, but recovery, although not statistically significant, appears to be faster in urban areas (Appendix Table A.8). Our results are robust to several choices of how to analyze the data. In particular, the results are robust to including school fixed effects (i.e., to estimating differences in performance across years within schools) — see Columns 2 and 4 in [Table](#page-6-0) [2](#page-6-0). The results are also similar if we use the percentage of correct answers (as opposed to the latent ability trait from the IRT model) to estimate learning levels (see Table A.3). The results do not vary either if we estimate the IRT model only over the common questions across all grades and rounds (see Table A.4). The results are fairly robust to different assumptions of how much students learn in the months between the exams (see Figures A.3 and A.4). Even under the most extreme scenarios (e.g., over 80% of learning in the academic year takes place in the four months between November and March), the learning loss estimates by the time schools re-opened are negative and statistically significant. Thus, it is unlikely that the difference in the timing of the exams explains the large losses we find shortly after schools re-opened. The effects by June 2023 are more sensitive – but still robust – to the pace of learning at the end of the school year. If students learn more towards the beginning of the school year (and thus, less than 10% of learning takes place between March and June), then we cannot reject full recovery of learning levels at 5% significance level. If, on the other hand, students learn more towards the end of the school year (and thus, more than 20% between March and June), then learning losses are more persistent than we estimate in our main specification in [Table](#page-6-0) [2.](#page-6-0) Finally, our results are also robust to potential differential attrition across rounds. [Table](#page-6-1) [3](#page-6-1) reports the results from the bounding exercise, as in [Lee](#page-8-6) [\(2009](#page-8-6)). Even in the unlikely scenario where all the students absent on the day of the assessment came from the top of the ability distribution, there are substantial learning losses (see Panel A). This lower-bound scenario corresponds to learning losses in 5th grade of 0.14 standard deviations in Spanish and 0.18 in math shortly after schools re-opened (March of 2021). By June of 2023, this lower bound also suggests no recovery, with learning losses of 0.19 and 0.22 standard deviations in Spanish and math, respectively. While the number of enrolled students does not change over time in these grades (see [Table](#page-4-0) [1](#page-4-0)), 5th and 6th grade students who attended school in 2019/20 and re-enrolled in grades 7 and 8 in schools after they reopened in 2021/22 had higher pre-pandemic test scores than those who dropped out (Appendix Table A.9), which suggests that dropout and attendance are related to student performance. Thus, the upper-bound scenario in Panel B seems more likely. In this scenario, 5th grade learning losses in 2021, shortly after schools reopened, are 0.29 and 0.34 standard deviations in Spanish and math, respectively. By 2023, this upper Table 2 Learning loss in Spanish and mathematics. | | Spanish | | Math | | |-------------------------------------------------------------------------------|---------|--------|--------|--------| | | (1) | (2) | (3) | (4) | | Grade 6 ( α 1 ) | .34*** | .33*** | .3*** | .3*** | | | (.031) | (.03) | (.036) | (.036) | | 2021 $(\alpha_2)$ | 35*** | 36*** | 38*** | 39*** | | | (.033) | (.032) | (.036) | (.037) | | Grade 6 $\times$ 2021 ( $\alpha_3$ ) | 033 | 032 | 056 | 057 | | | (.038) | (.038) | (.047) | (.047) | | 2023 | 0095 | 019 | 049 | 057 | | | (.033) | (.032) | (.038) | (.038) | | Grade 5: Average in 2020 ( $\alpha_0$ ) | -0.00 | -0.00 | -0.02 | -0.02 | | Grade 5: Learning loss 2021 $(\alpha_2 + 0.4\alpha_1)$ | -0.22 | -0.23 | -0.26 | -0.27 | | p-value( $H_0$ :Learning loss grade 5 (2021) = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | Grade 5: Learning loss 2023 ( $\alpha_4 - 0.2\alpha_1$ ) | -0.08 | -0.08 | -0.11 | -0.12 | | p-value( $H_0$ :Learning loss grade 5 (2023) = 0) | 0.01 | 0.00 | 0.00 | 0.00 | | Grade 6: Learning loss 2021 $(\alpha_2 + \alpha_3 + 0.4\alpha_1)$ | -0.25 | -0.26 | -0.32 | -0.33 | | p-value( $H_0$ :Learning loss grade 6 (2021) = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | EYOS Grade 5 in 2021 $(\frac{a_2+0.4a_1}{a_1})$ | -0.65 | -0.69 | -0.86 | -0.90 | | p-value( $H_0$ :EYOS Grade 5 in 2021 = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | EYOS Grade 5 in 2023 ( $\frac{\alpha_4 - 0.2\alpha_1}{\alpha}$ ) | -0.23 | -0.26 | -0.36 | -0.39 | | p-value( $H_0$ :EYOS Grade 5 in 2023 = 0) | 0.02 | 0.01 | 0.01 | 0.00 | | EYOS Grade 6 in 2021 ( $\frac{\alpha_2 + \alpha_3 + 0.4\alpha_1}{\alpha_1}$ ) | -0.75 | -0.78 | -1.05 | -1.09 | | p-value( $H_0$ :EYOS Grade 6 in 2021 = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | N. of obs. | 52,813 | 52,811 | 52,813 | 52,811 | | School fixed effects | No | Yes | No | Yes | Notes: This table presents the estimates from Eq. (1) using data from the six municipalities tested in 2020, 2021, and 2023. Learning levels are estimated using a single IRT model across grades and years. The learning loss measures are computed as the difference between the actual learning level and the expected level, taking into account that the exam was administered at different points in the school year. The "equivalent years of schooling" (EYOS) indicates the ratio of the learning loss to the normal progression in a typical school year. Standard errors are clustered at the school level. Statistical significance at the 1, 5, 10% levels is indicated by \*\*\*, \*\*, and \*. Table 3 Bounds for learning loss. | | Spanish | | Math | | |-----------------------------------------------------------------------------|---------|---------|--------|--------| | | (1) | (2) | (3) | (4) | | Panel A: Lower bound (right-tail trimming) | | | | | | Grade 6 ( $\alpha_1$ ) | .33*** | .33*** | .3*** | .3*** | | | (.028) | (.028) | (.032) | (.032) | | 2021 (α 2 ) | 27*** | 29*** | 3*** | 32*** | | | (.032) | (.032) | (.034) | (.034) | | Grade 6 $\times$ 2021 ( $\alpha_3$ ) | 03 | 027 | 053 | 053 | | | (.038) | (.037) | (.044) | (.044) | | 2023 | 12*** | 12*** | 16*** | 16*** | | | (.029) | (.028) | (.03) | (.03) | | Grade 5: Average in 2020 ( $\alpha_0$ ) | -0.14 | -0.14 | -0.16 | -0.16 | | Grade 5: Learning loss 2021 ( $\alpha_2 + 0.4\alpha_1$ ) | -0.14 | -0.16 | -0.18 | -0.20 | | p-value( $H_0$ :Learning loss grade 5 (2021) = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | Grade 5: Learning loss 2023 ( $\alpha_4 - 0.2\alpha_1$ ) | -0.19 | -0.18 | -0.22 | -0.22 | | p-value( $H_0$ :Learning loss grade 5 (2023) = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | Grade 6: Learning loss 2021 ( $\alpha_2 + \alpha_3 + 0.4\alpha_1$ ) | -0.17 | -0.19 | -0.23 | -0.25 | | p-value( $H_0$ :Learning loss grade 6 (2021) = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | EYOS Grade 5 in 2021 $\left(\frac{\alpha_2+0.4\alpha_1}{\alpha_1}\right)$ | -0.41 | -0.49 | -0.60 | -0.67 | | p-value( $H_0$ :EYOS Grade 5 in 2021 = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | EYOS Grade 5 in 2023 $\left(\frac{\alpha_4 - 0.2\alpha_1}{\alpha_1}\right)$ | -0.56 | -0.56 | -0.72 | -0.72 | | p-value( $H_0$ :EYOS Grade 5 in 2023 = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | EYOS Grade 6 in 2021 $(\frac{\alpha_2 + \alpha_3 + 0.4\alpha_1}{\alpha_1})$ | -0.50 | -0.57 | -0.78 | -0.85 | | p-value( $H_0$ :EYOS Grade 6 in 2021 = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | N. of obs. | 50,877 | 50,875 | 50,877 | 50,875 | | School fixed effects | No | Yes | No | Yes | | Panel B: Upper bound (left-tail trimming) | | | | | | Grade 6 ( $\alpha_1$ ) | .34*** | .34*** | .31*** | .31*** | | 5.1440 5 (M 1 ) | (.029) | (.029) | (.035) | (.035) | | 2021 $(\alpha_2)$ | 43*** | 43*** | 46*** | 46*** | | 2021 (42) | (.031) | (.031) | (.035) | (.036) | | Grade $6 \times 2021 \ (\alpha_3)$ | 042 | 041 | 065 | 064 | | 5.1440 5 × 2021 (43) | (.037) | (.037) | (.046) | (.046) | | 2023 | .1*** | .082*** | .058* | .041 | | 2020 | (.029) | (.029) | (.035) | (.035) | | Grade 5: Average in 2020 ( $\alpha_0$ ) | 0.13 | 0.13 | 0.11 | 0.11 | | Grade 5: Learning loss 2021 $(\alpha_0)$ | -0.29 | -0.29 | -0.34 | -0.34 | (continued on next page) Table 3 (continued) | | Spanish | | Math | | |-----------------------------------------------------------------------------|---------|--------|--------|--------| | | (1) | (2) | (3) | (4) | | p-value( $H_0$ :Learning loss grade 5 (2021) = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | Grade 5: Learning loss 2023 ( $\alpha_4 - 0.2\alpha_1$ ) | 0.03 | 0.01 | -0.00 | -0.02 | | p-value( $H_0$ :Learning loss grade 5 (2023) = 0) | 0.21 | 0.59 | 0.88 | 0.51 | | Grade 6: Learning loss 2021 $(\alpha_2 + \alpha_3 + 0.4\alpha_1)$ | -0.33 | -0.33 | -0.40 | -0.40 | | p-value( $H_0$ :Learning loss grade 6 (2021) = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | EYOS Grade 5 in 2021 $\left(\frac{\alpha_2+0.4\alpha_1}{\alpha_2}\right)$ | -0.85 | -0.86 | -1.08 | -1.10 | | p-value( $H_0$ :EYOS Grade 5 in 2021 = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | EYOS Grade 5 in 2023 ( $\frac{\alpha_4 - 0.2\alpha_1}{\alpha_1}$ ) | 0.10 | 0.04 | -0.02 | -0.07 | | p-value( $H_0$ :EYOS Grade 5 in 2023 = 0) | 0.20 | 0.59 | 0.88 | 0.52 | | EYOS Grade 6 in 2021 $(\frac{\alpha_2 + \alpha_3 + 0.4\alpha_1}{\alpha_1})$ | -0.97 | -0.98 | -1.29 | -1.30 | | p-value( $H_0$ :EYOS Grade 6 in 2021 = 0) | 0.00 | 0.00 | 0.00 | 0.00 | | N. of obs. | 50,880 | 50,878 | 50,880 | 50,878 | | School fixed effects | No | Yes | No | Yes | Notes: This table presents the estimates from Eq. (1) using data from the six municipalities tested in 2020, 2021, and 2023 using the trimming proposed in Lee (2009). Panel A trims the upper tails of the 2020 and 2023 data by the differential testing rate between years (see Table 1). Panel B trims the lower tails of the 2020 and 2023 data by the differential testing rates. The learning loss measures are computed as the difference between the actual learning level and the expected level, taking into account that the exam was administered at different points in the school year. The "equivalent years of schooling" (EYOS) indicates the ratio of the learning loss to the normal progression in a typical school year. Standard errors are clustered at the school level. Statistical significance at the 1, 5, 10% levels is indicated by \*\*\*, \*\*, and \*. bound suggests complete recovery, with no significant differences in performance compared to pre-pandemic levels. The bounding results are qualitatively similar if we trim the data within each school, rather than for the sample as a whole (see Table A.10). #### 5. Conclusions In this paper, we compare test score data for 5th and 6th grade students in the Mexican state of Guanajuato before and after the COVID-19 pandemic school closures. We show that learning losses shortly after schools re-opened (in March of 2021) were large, especially in math. Our baseline estimates suggest that learning outcomes in math declined by 0.26–0.33 standard deviations and by 0.22–0.29 standard deviations in Spanish. We also find evidence of persistence in learning losses: over 18 months after schools re-opened (by June of 2023), students still score below pre-pandemic levels, suggesting a recovery of $\sim$ 60% of the losses built up during school closures. These results are robust to alternative ways of standardizing test scores (for example, as the proportion of correct answers or using IRT), to including school fixed effects, and to bounding to adjust for differences in the proportion of missing data before and after the school closures. Our paper is one of the first to carefully and credibly study learning losses in a developing country. Like many countries, Mexico attempted to substitute in-person learning with various forms of remote instruction, including online educational platforms, TV education and homework delivered by teachers to students' homes. Our results show, however, that remote learning did not prevent learning losses. This finding echoes others from the U.S. literature, where online instruction appears to have been a poor substitute for in-person classes (Halloran et al., 2021). Compared with other developing countries, Mexico closed its schools during the pandemic longer than most countries. Therefore, Mexican students likely have more ground to make up than those in countries that reopened schools sooner. Recent work on the pandemic's effects on human capital in developing countries discusses policy options to recover learning losses, including lengthening the school day or year, simplifying the curriculum, grouping children by achievement rather than age, and remedial tutoring (Schady et al., 2023). The government lengthened the 2021/2022 school year and adopted other measures to cope with learning losses. Indeed, we find evidence of recovery but students are still lagging behind compared to the learning levels observed before the pandemic. ## Data availability The authors do not have permission to share data. #### Appendix A. Supplementary data Supplementary material related to this article can be found online at https://doi.org/10.1016/j.econedurev.2023.102492. #### References Ardington, C., Wills, G., & Kotze, J. (2021). COVID-19 learning losses: Early grade reading in South Africa. *International Journal of Educational Development*, 86, Article 102480. http://dx.doi.org/10.1016/j.ijedudev.2021.102480, URL https://www.sciencedirect.com/science/article/pii/S0738059321001334. Baird, M. D., & Pane, J. F. (2019). Translating standardized effects of education programs into more interpretable metrics. *Educational Researcher*, 48(4), 217–228. Betthäuser, B. A., Bach-Mortensen, A. M., & Engzell, P. (2023). A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic. *Nature Human Behaviour*, 7(3), 375–385. Boruchowicz, C., Parker, S. W., & Robbins, L. (2022). Time use of youth during a pandemic: Evidence from Mexico. *World Development*, 149, Article 105687. http://dx.doi.org/10.1016/j.worlddev.2021.105687, URL https://www.sciencedirect.com/science/article/pii/S0305750X21003028. Cárdenas, S., Lomelí, D., & Ruelas, I. (2022). COVID-19 and post-pandemic educational policies in Mexico. What is at stake? In F. M. Reimers (Ed.), Primary and secondary education during covid-19: disruptions to educational opportunity during a pandemic (pp. 153–175). Cham: Springer International Publishing, http://dx.doi.org/10.1007/978-3-030-81500-4\_6. Chalmers, R. P. (2012). mirt: A multidimensional Item Response Theory package for the R environment. *Journal of Statistical Software*, 48(6), 1–29. http://dx.doi.org/ 10.18637/jss.y048.i06. Evans, D., & Yuan, F. (2019). Equivalent years of schooling: A metric to communicate learning gains in concrete terms. World Bank Policy Research Working Paper. (8752). Goldhaber, D., Kane, T. J., McEachin, A., Morton, E., Patterson, T., & Staiger, D. O. (2022). The consequences of remote and hybrid instruction during the pandemic: Working Paper No. 30010, National Bureau of Economic Research, http://dx.doi.org/10.3386/w30010. URL http://www.nber.org/papers/w30010. Guariso, A., & Björkman Nyqvist, M. (2023). The impact of the COVID-19 pandemic on children's learning and wellbeing: evidence from India. *Journal of Development Economics*, 164, 103133. http://dx.doi.org/10.1016/j.jdeveco.2023.103133, URL https://www.sciencedirect.com/science/article/pii/S0304387823000883. Halloran, C., Jack, R., Okun, J. C., & Oster, E. (2021). Pandemic schooling mode and student test scores: evidence from US states: Working Paper No. 29497, National Bureau of Economic Research, http://dx.doi.org/10.3386/w29497, URL http://www.nber. org/papers/w29497. Hanushek, E. A., & Woessmann, L. (2008). The role of cognitive skills in economic development. *Journal of Economic Literature*, 46(3), 607–668. http://dx.doi.org/10. 1257/jel.46.3.607, URL https://www.aeaweb.org/articles?id=10.1257/jel.46.3.607. Hevia, F. J., Vergara-Lope, S., Velásquez-Durán, A., & Calderón, D. (2022). Estimation of the fundamental learning loss and learning poverty related to COVID-19 pandemic in Mexico. *International Journal of Educational Development*, 88, Article 102515. Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253. Kuhfeld, M., & Lewis, K. (2022). Student achievement in 2021-22: Cause for hope and continued urgency. URL https://www.nwea.org/research/publication/student-achievement-in-2021-22-cause-for-hope-and-continued-urgency/. - [Lee, D. S. \(2009\). Training, wages, and sample selection: Estimating sharp bounds on](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb15) treatment effects. *[Review of Economic Studies](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb15)*, *76*(3), 1071–1102. - [Lichand, G., & Alberto Doria, C. \(2022\). The lasting impacts of remote learning in the](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb16) [absence of remedial policies: Evidence from Brazil. Available at SSRN 4209299.](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb16) - [Lichand, G., Doria, C. A., Leal-Neto, O., & Fernandes, J. P. C. \(2022\). The impacts](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb17) [of remote learning in secondary education during the pandemic in Brazil.](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb17) *Nature [Human Behaviour](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb17)*, *6*(8), 1079–1086. - OECD (2019). *[PISA 2018 results \(volume I\): What students know and can do](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb18)*: *Tech. rep.*, [OECD Publishing.](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb18) - OECD (2020). School education during COVID-19: Were teachers and students ready? Mexico country note. URL [https://www.oecd.org/education/Mexico-coronavirus](https://www.oecd.org/education/Mexico-coronavirus-education-country-note.pdf)[education-country-note.pdf](https://www.oecd.org/education/Mexico-coronavirus-education-country-note.pdf). - Padilla-Romo, M. (2022). Full-time schools, policy-induced school switching, and academic performance. *Journal of Economic Behaviour and Organization*, *196*, 79–103. [http://dx.doi.org/10.1016/j.jebo.2022.01.025,](http://dx.doi.org/10.1016/j.jebo.2022.01.025) URL [https://www.](https://www.sciencedirect.com/science/article/pii/S0167268122000385) [sciencedirect.com/science/article/pii/S0167268122000385.](https://www.sciencedirect.com/science/article/pii/S0167268122000385) - [Schady, N., Holla, A., Sabarwal, S., Silva, J., & Yi Chang, A. \(2023\).](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb21) *Collapse and [recovery: How the COVID-19 pandemic eroded human capital and what to do about it](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb21)*: *Tech. rep.*[, World Bank.](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb21) - [Secretaría de Educación de Guanajuato \(2021\).](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb22) *Panorama educativo de guanajuato. [Edición 2021](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb22)*: *Tech. rep.*. - [Secretaría de Educación de Guanajuato \(2022\).](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb23) *RIMA recopilación de información para la [mejora de los aprendizajes. informe de resultados evaluación de matemáticas y español](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb23) [RIMA 2020 y 2021](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb23)*: *Tech. rep.*. - Singh, A. (2015). How standard is a standard deviation? A cautionary note on using SDs to compare across impact evaluations in education. URL [http://blogs.worldbank.org/impactevaluations/how-standard-standard-deviation](http://blogs.worldbank.org/impactevaluations/how-standard-standard-deviation-cautionary-note-using-sds-compare-across-impact-evaluations)[cautionary-note-using-sds-compare-across-impact-evaluations.](http://blogs.worldbank.org/impactevaluations/how-standard-standard-deviation-cautionary-note-using-sds-compare-across-impact-evaluations) - Singh, A., Romero, M., & Muralidharan, K. (2022). *COVID-19 learning loss and recovery: panel data evidence from India*: *Working Paper No. 30552*, National Bureau of Economic Research, [http://dx.doi.org/10.3386/w30552,](http://dx.doi.org/10.3386/w30552) URL [http://www.nber.](http://www.nber.org/papers/w30552) [org/papers/w30552.](http://www.nber.org/papers/w30552) - [Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D.,](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb26) [Viswanathan, M., et al. \(2016\). ROBINS-I: a tool for assessing risk of bias in](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb26) [non-randomised studies of interventions.](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb26) *BMJ*, *355*. - UNESCO (2021a). *[Resultados de logros de aprendizaje y factores asociados del estudio](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb27) [regional comparativo y explicativo \(ERCE 2019\)](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb27)*: *Tech. rep.*, UNESCO. - UNESCO (2021b). *UNESCO global dataset on the duration of school closures.*: *Tech. rep.*, URL [https://webarchive.unesco.org/web/20220315080724/https:/en.unesco.](https://webarchive.unesco.org/web/20220315080724/https:/en.unesco.org/covid19/educationresponse#durationschoolclosures) [org/covid19/educationresponse#durationschoolclosures](https://webarchive.unesco.org/web/20220315080724/https:/en.unesco.org/covid19/educationresponse#durationschoolclosures). - [Wang, H., Paulson, K. R., Pease, S. A., Watson, S., Comfort, H., Zheng, P., et al. \(2022\).](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb29) [Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb29) [of COVID-19-related mortality, 2020–21.](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb29) *The Lancet*. - World Bank (2022). *[Poverty and shared prosperity 2022: correcting course](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb30)*. The World [Bank.](http://refhub.elsevier.com/S0272-7757(23)00139-5/sb30)