---
title: "The Effect of School Grants on Test Scores: Experimental Evidence from Mexico"
authors_and_venue: "with Juan Bedoya, Rafael de Hoyos, Marcela Silveyra, and Monica Yanez-Pagans - Economica 91(363), 2024"
venue: "Economica"
year: 2024
doi: "10.1111/ecca.12523"
abstract: "We use a randomized experiment (across 200 public primary schools in Puebla, Mexico) to study the impact of providing schools with cash grants on student test scores. Treated schools received on average 16 USD per student each year for two years, an increase of 20% in public spending per child, after teacher salaries. Overall, the grants had no impact on student test scores. Lack of a treatment effect does not seem to be driven by poor implementation or a substitution away from other inputs (e.g. household expenditure)."
pdf_url: "https://mauricio-romero.com/pdfs/papers/Grants_Test_Scores_Mexico.pdf"
canonical_url: "https://mauricio-romero.com/pdfs/papers/Grants_Test_Scores_Mexico.pdf.md"
source: research.qmd
note: >-
  Machine-readable Markdown version of the paper, generated for LLMs.
---

## **ORIGINAL ARTICLE**

# **The effect of school grants on test scores: experimental evidence from Mexico**

**Mauricio Romero<sup>1</sup> Juan Bedoya<sup>2</sup> Monica Yanez-Pagans<sup>3</sup> Marcela Silveyra<sup>3</sup> Rafael de Hoyos<sup>4</sup>**

#### **Correspondence**

Mauricio Romero, Department of Economics, ITAM, Av. Camino, a Santa Teresa #930, Col. Héroes de Padierna, Mexico City 10700, Mexico Email: mtromero@itam.mx

#### **Abstract**

We use a randomized experiment (across 200 public primary schools in Puebla, Mexico) to study the impact of providing schools with cash grants on student test scores. Treated schools received on average ∼16 USD per student each year for two years, an increase of ∼20% in public spending per child, after teacher salaries. Overall, the grants had no impact on student test scores. Lack of a treatment effect does not seem to be driven by poor implementation or a substitution away from other inputs (e.g. household expenditure).

## **1 INTRODUCTION**

Despite significant investments in the education sector—government expenditure in education amounts to about 4.4% of GDP in low- and middle-income countries (World Bank [2017a\)](#page-15-0)—many children do not have access to quality education and do not acquire basic numeracy and literacy skills after many years of formal schooling (World Bank [2017d\)](#page-15-1). Still, many stakeholders claim that investing more in education will solve this 'learning crisis'[.1](#page-12-0) In the case of Mexico, the fact that expenditure per student is among the lowest in the OECD is usually highlighted as one of the reasons behind the poor learning outcomes (World Bank [2005;](#page-15-2) OECD [2018,](#page-15-3) [2019\)](#page-15-4). In this paper, we leverage a randomized controlled trial to study the effects of increasing the funds available to schools in the state of Puebla in Mexico by providing them with cash grants.

We study the implementation of a large-scale strategy, called *Escuela al Centro*, designed by the Government of Mexico to strengthen school autonomy and improve school principals' managerial capacity. The government implemented the strategy nationwide for three consecutive school years: 2015–16, 2016–17 and 2017–18. A core component of *Escuela al Centro* was grants to schools of, on average, ∼16 USD per student per year for two years (2015–16 and 2016–17), conditional on developing a school improvement plan approved by the school council. This amounts to an increase of ∼1.5% in public spending per child. However, since most of the budget (93%) is spent on teacher salaries, the grants amount to an increase of ∼20% in spending after excluding teacher salaries.[2](#page-12-1) The grant's size is larger than most other grant treatments

© 2024 London School of Economics and Political Science.

<sup>1</sup>ITAM

<sup>2</sup>Universidad de Cantabria

<sup>3</sup>World Bank

<sup>4</sup>ITAM and World Bank

in the literature (see Figure [3\(a\)](#page-10-0) below). However, the increase is similar as a proportion of the education budget (see Subsection II). Public schools in Mexico cover running costs with voluntary contributions from parents, transfers from state governments (subject to budget availability), or school grants from the Federal Government. Schools use grants mainly to purchase basic supplies (e.g. chalk, toiletries), classroom projectors and computers, conduct small infrastructure repairs, and pay for cleaning and maintenance. We randomly assigned 200 eligible public primary schools in Puebla to receive the grant (treatment, *n* = 101) or not (control, *n* = 99).

The treatment effect of the grants on student learning is negative (albeit, statistically insignificant). A year after schools received the last transfer (in 2018), students in sixth grade (at the end of primary) in treatment schools scored 0*.*08 (*p*-value 0.27) *lower* than those in control schools in a nationwide standardized test (PLANEA)[.3](#page-12-2) This result is robust to several studentand school-level controls. After including several controls, we can rule out an effect greater than 0*.*01 at the 95% confidence level[.4](#page-12-3) We find no evidence of heterogeneity in treatment effects by student or school characteristics.

Our findings add to the evidence from low- and middle-income countries that increasing schools' resources does not impact student learning outcomes on its own (McEwan [2015\)](#page-15-5)[.5](#page-12-4) However, as Glewwe and Muralidharan [\(2016\)](#page-14-0) point out, it is critical to understand whether this lack of a treatment effect comes from poor implementation or a substitution away from other inputs (e.g. household expenditure), as well as whether there are other binding constraints, and whether complementary inputs/reforms are necessary for the grants to be effective (as in Mbiti *et al.* [2019\)](#page-15-6). We can rule out poor programme implementation from state education authorities, as the administrative records show that the grants were indeed transferred to and received by the schools by the middle of the school year (around December). We do not find evidence that parents changed their inputs in response to the treatment[.6](#page-12-5) Further, there is no evidence that other government programmes were more or less likely to take place in treatment schools.

We cannot confirm experimentally whether other binding constraints exist or if complementary reforms are necessary. Still, Mexico's education system is known for a series of shortcomings, including low accountability, discretionary hiring and promotion of teachers and school directors, and lack of school autonomy restricting their possibility of adapting national- or state-level programmes to the needs of their students (World Bank [2005;](#page-15-2) OECD [2018,](#page-15-3) [2019\)](#page-15-4). While evidence from the USA has shown that increasing schools' resources improves student learning (Jackson and Mackevicius [2021;](#page-14-1) Jackson *et al.* [2016,](#page-14-2) [2021;](#page-14-3) Jackson [2018\)](#page-14-4), a likely explanation for the lack of positive effects from increasing school resources in Mexico and other developing countries' settings are the additional constraints faced by education systems.

Another possibility is that spending may be more effective if directed elsewhere. For example, the largest spending category—with a third of expenditure—was information and communication technologies (i.e. projectors, computers and televisions). However, this type of investment has been shown elsewhere to have negative (e.g. Malamud and Pop-Eleches [2011;](#page-14-5) Vigdor *et al.* [2014\)](#page-15-7) or no impact on academic achievement (e.g. Barrera-Osorio and Linden [2009;](#page-13-0) Fairlie and Robinson [2013;](#page-14-6) Beuermann *et al.* [2015;](#page-13-1) Cristia *et al.* [2017\)](#page-13-2) when they are not accompanied by software or other investments (Muralidharan *et al.* [2019\)](#page-15-8). Yet the programme's goal was to give schools autonomy in how to spend the funds. This was motivated by the idea that schools know their needs better—echoing decentralization reforms elsewhere (Bardhan [2002,](#page-13-3) [2016;](#page-13-4) World Bank [2003;](#page-15-9) Bardhan and Mookherjee, [2006\)](#page-13-5). Indeed, decentralization has been associated with better education outcomes (Barankay and Lockwood [2007;](#page-13-6) Galiani *et al.* [2008;](#page-14-7) Faguet and Sánchez [2008;](#page-14-8) Falch and Fischer [2012;](#page-14-9) Elacqua *et al.* [2021\)](#page-14-10). However, decentralization reforms often involved several aspects, not just budgetary autonomy, which may be why the grants that we study had no effect.

# **2 CONTEXT AND INTERVENTION**

# **Context**

In line with other middle-income countries, Mexico spends 4.6% of its GDP on education (World Bank [2017b\)](#page-15-10). While almost all children graduate from primary school (World Bank [2017c\)](#page-15-11), fewer than half achieve a basic level of proficiency in Mathematics and Spanish according to the 2018 national standardized tests (Instituto Nacional para la Evaluación de la Educación [2018\)](#page-14-11). Poor learning outcomes are even starker in schools located in marginalized areas, where only one in three students achieves a basic level of proficiency upon graduating from primary school.

Puebla, the state where the experiment takes place, has a population of just over 6 million (∼5.2% of the country's population) (Instituto Nacional de Estadistica y Geografia [2016b\)](#page-14-12). Located in the eastern central area of Mexico, Puebla is one of the poorest states, with a per capita GDP significantly below the national average in 2019 (91,000 versus 139,000 pesos per year) (Instituto Nacional de Estadistica y Geografia, [2016a\)](#page-14-13), and a poverty rate of 59.4% versus 35.9% at the national level (Consejo Nacional de Evaluación de la Política de Desarrollo Social [2018\)](#page-13-7). The average Poblano has fewer years of schooling than the average Mexican (8.5 versus 9.2) (Instituto Nacional de Estadistica y Geografia [2016a\)](#page-14-13). In 2015, the state performed below the national average on the 6th grade nationwide standardized test (PLANEA) in Language, but above the national average in Mathematics (Instituto Nacional para la Evaluación de la Educación [2016\)](#page-14-14).

# **The** *Escuela al Centro* **strategy**

The government implemented *Escuela al Centro* nationwide for three consecutive school years: 2015–16, 2016–17 and 2017–18. The strategy had two main components: school principals' managerial training and school grants.[7](#page-12-6)

The school principals' managerial training component focused on learning to use two tools: (i) a student assessment to monitor foundational students' skills (*Sistema de Alerta Temprana en Escuelas de Educación Básica*, SisAT), and (ii) a Stallings classroom observation tool to provide feedback to teachers on how to improve their instructional and pedagogical practices. As *Escuela al Centro* was designed to have national coverage, the government established a 'train the trainer' cascade model, under which state-level education authorities selected 10% of all primary school supervisors to receive training on the SisAT and Stallings classroom observation tool from a professional team of trainers. The trained supervisors then trained the rest of the supervisors in their states. After all supervisors in a state were trained (either directly by the team of professional trainers or by their peers), they were responsible for training the school principals in their jurisdictions. All schools in our sample received management training through the cascade model.

The school grant component consisted of a cash grant to schools provided annually, conditional on a school improvement plan approved by the school council. The main goal of the grants was to provide schools with more budgetary autonomy. As with many other decentralization initiatives, the premise was that schools know their needs better than central government officials (Bardhan [2002,](#page-13-3) [2016;](#page-13-4) World Bank [2003;](#page-15-9) Bardhan and Mookherjee [2006\)](#page-13-5). The grants were 5–40 USD per student per year, with the median (and the average) school receiving ∼16 USD per year (see the full distribution in Figure [1\)](#page-3-0)[.8](#page-12-7) The size of the grant was determined by three components: (1) the number of students in each school; (2) the level of 'marginalization' of the locality where the school was located; and (3) school performance measured by year-to-year changes in retention and promotion rates[.9](#page-12-8) The grants were disbursed between May

<span id="page-3-0"></span>**FIGURE 1** Distribution of school grants per student. *Notes*: This figure shows the distribution of the grant amount (per student per year) that schools received for two academic years (2015–16 and 2016–17).

and July each year, for two years (2015 and 2016). On average, ∼35% of expenditure went to information and communication technologies (i.e. projectors, computers and televisions), ∼16% went to basic classroom furniture and supplies (e.g. desks, chairs, blackboards and chalk), ∼13% went to conduct small infrastructure repairs and pay for cleaning and maintenance, ∼10% went to buy teaching materials (e.g. maps and charts) and other school supplies (e.g. paper, pens and pencils), ∼7.5% went to educational software, and ∼6% went to furniture and supplies for common areas (e.g. desks and chairs for the teachers' lounge and principal's office, and bathroom supplies). Less than ∼7% went to pay for training or other capacity-building activities.[10](#page-12-9)

To put the grant size in context, the government spent roughly 1090 USD per student in primary schools in 2016. Excluding teacher salaries (which take up 93% of the budget), the expenditure per child was ∼76 USD (OECD [2016\)](#page-15-12). Thus the grants increase ∼20% public spending per child, after teacher salaries. In absolute terms, the grant's size is larger than most other grant treatments in the literature (see Figure [3\(a\)](#page-10-0) below). However, the increase is similar as a proportion of the education budget[.11](#page-12-10)

The government was interested in testing the effectiveness of these grants. Therefore, subject to the over-subscription of eligible schools, it agreed to provide grants randomly in a subset of public schools. A randomized experiment that took place between 2007 and 2010 in Mexico—in a different sample and evaluating a different programme—found that an increase of 7.5 USD (equivalent to roughly 8.78 in 2016 USD) per student had a (statistically insignificant) effect of 0*.*08 after one year on student test scores (Barrera-Osorio *et al.* [2020\)](#page-13-8). The size of the grant that we consider is almost twice as large. The experiment is designed to be able to detect a treatment effect of at least 0*.*16 with 80% power and 5% size—that is, the expected treatment effect under a (strong) linearity assumption based on previous findings[.12](#page-12-11)

## **3 RESEARCH DESIGN AND DATA**

# **Sampling and randomization**

Local education authorities invited all public primary schools to apply for the school grant component of *Escuela al Centro*. Given budget constraints, we randomly assigned about half of the 200 eligible schools that applied for a grant to receive it (treatment, *n* = 101), and some to the control group (*n* = 99)[.13](#page-12-12)

<span id="page-4-0"></span>**FIGURE 2** Distribution of treatment and control schools (and other schools not in the experiment). *Notes*: Geographical information on the administrative areas of Mexico comes from Instituto Nacional de Estadistica y Geografia [\(2018\)](#page-14-15).

Our sampling frame included public primary schools with more than 60 students, and excluded multi-grade schools (i.e. those with at least one classroom that includes students from different grades)[.14](#page-12-13) Therefore public primary schools included in the experiment have more students and teachers than the average public primary school in Puebla (see Table A.1 of the Online Appendix).

Schools were stratified based on their locality's poverty level, and whether they were urban or rural. They were then randomized into treatment or control[.15](#page-12-14) Control schools were notified of the outcome without a promise to enter the programme in the future. Figure [2](#page-4-0) displays the geographical distribution of treatment and control schools, as well as every other public primary school in Puebla.

## **Data**

The data used in this study come mainly from administrative records provided by the government. Student learning outcomes are based on a nationwide standardized test known as PLANEA for its acronym in Spanish (*Plan Nacional para la Evaluación de los Aprendizajes*). It was a multiple-choice standardized test measuring Mathematics and Language learning levels in 6th, 9th and 12th grades. The goal of the test was to provide information to local authorities, supervisory teams and school directors on schools' learning levels at the end of each educational level (i.e. primary, lower- and upper-secondary). By linking the test items with the national curriculum, PLANEA identified content that each school should strengthen. PLANEA was applied to all schools in the country, including all students (in tested grade) in schools with 40 students or fewer. The test was applied to a random sample of 40 students in larger schools.

PLANEA was administered to 6th graders in June 2015 (baseline) and June 2018 (follow-up). The exam was designed by the former National Institution for Evaluation of Education (*Instituto Nacional para la Evaluación de la Educación*, or INEE) and was applied by the Secretary of Public Education in coordination with state educational authorities. For this study, the government provided access to anonymized student-level data for both years for all schools participating in the evaluation. The cohort tested in 2018 would have been in Grade 5 during the 2016–17 school year, and Grade 4 during the 2015–16 school year, and thus exposed to the programme for its full length. As part of participating in PLANEA, students also need to answer a survey before the standardized test. We use these surveys to measure whether parents invest more or less in their children's education in response to the grants in treatment schools.

In addition to test scores, PLANEA collects information on the location of each school. We use this information to determine each school's marginalization index based on its locality. The marginalization index, estimated by the *Consejo Nacional de Población* (CONAPO), considers localities' deficiencies in terms of education, housing, population and household income.

We also use administrative school census data collected by the federal and state-level education authorities known as '*Formato 911*'. Since 1998, *Formato 911* has been collected at the beginning and end of each school year. It gathers basic information on the number of students, the number of teachers and their qualifications, the school principal's characteristics, the number of classrooms, and its geographic location. Using a unique school identifier (*Clave de Centro de Trabajo*), these school census data can be matched with the PLANEA test scores.

To study teacher turnover, we use publicly available data from teacher payrolls paid by the federal government.[16](#page-12-15) These data record the schools in which teachers work at different points in time.

Finally, these schools were part of a larger school sample in which information on schools' managerial practices was collected in 2015 using the Development World Management Survey (DWMS) (Bloom and Van Reenen [2007;](#page-13-9) Bloom *et al.* [2013;](#page-13-10) Lemos and Scur [2016;](#page-14-16) Romero *et al.* [2022\)](#page-15-13). The DWMS is an adaptation of the World Management Survey for developing countries in which management managerial practices are measured along four dimensions: operations management, people management, target setting and monitoring. We use the DWMS to study heterogeneity in treatment effects by the quality of managerial practices in the school. We construct the management index following the DWMS methodology, and normalize it to have mean 0 and standard deviation 1 in our experimental sample.

## **Balance and attrition**

All student and school characteristics are balanced between the treatment and control groups at baseline (see Table [1\)](#page-6-0). The average school in our sample has 288 students, 9.2 teachers, and pupil–teacher ratio 30. In addition, 20% of schools are in urban areas, and 70% are in areas categorized as poor or very poor by the government.

Table [1](#page-6-0) shows the fraction of schools for which we have PLANEA data at endline (in 2018), which is ∼99%—we have PLANEA data for almost all the sample, except one control school. The proportion of schools with PLANEA data is balanced across groups, as is the number of students sitting for the PLANEA exam in each school. Thus while results on test scores are missing data for one control school, differential attrition is not a central concern in this setting. Student characteristics (e.g. age, gender and parental education) are balanced in 2018 (see Table A.2 of the Online Appendix), suggesting that students did not sort across schools in response to the treatment.

**TABLE 1** Balance across treatment and control groups.

<span id="page-6-0"></span>

|                                    | Mean     |           | Difference  |  |
|------------------------------------|----------|-----------|-------------|--|
|                                    | Control  | Treatment | ((2) − (1)) |  |
|                                    | (1)      | (2)       | (3)         |  |
| Panel A: Student level             |          |           |             |  |
| Mathematics score (2015)           | −0.02    | 0.02      | 0.04        |  |
|                                    | (0.99)   | (1.01)    | (0.09)      |  |
| Language score (2015)              | −0.02    | 0.02      | 0.04        |  |
|                                    | (0.99)   | (1.01)    | (0.05)      |  |
| % Male (2015)                      | 48.73    | 51.18     | 2.45        |  |
|                                    | (49.99)  | (49.99)   | (1.79)      |  |
| Panel B: School level              |          |           |             |  |
| Marginalization                    | 0.71     | 0.69      | −0.01       |  |
|                                    | (0.46)   | (0.46)    | (0.07)      |  |
| Urban                              | 0.21     | 0.18      | −0.03       |  |
|                                    | (0.41)   | (0.38)    | (0.06)      |  |
| Number of students (2015)          | 285.06   | 291.15    | 6.09        |  |
|                                    | (178.59) | (186.53)  | (25.82)     |  |
| Number of teachers (2015)          | 9.08     | 9.30      | 0.22        |  |
|                                    | (4.52)   | (4.67)    | (0.65)      |  |
| Student–teacher ratio (2015)       | 30.06    | 29.89     | −0.17       |  |
|                                    | (6.55)   | (6.63)    | (0.93)      |  |
| Baseline DWMS                      | −0.07    | 0.06      | 0.13        |  |
|                                    | (1.08)   | (0.94)    | (0.20)      |  |
| PLANEA endline missing             | 0.01     | 0.00      | −0.01       |  |
|                                    | (0.10)   | (0.00)    | (0.01)      |  |
| Students with PLANEA scores (2018) | 32.95    | 33.77     | 0.82        |  |
|                                    | (18.29)  | (17.77)   | (2.55)      |  |
| Number of observations             | 99       | 101       | 200         |  |

*Notes:* This table presents the means and standard deviations (in parentheses) for control schools (column (1)) and treatment schools (column (2)). The differences, taking into account the randomization design (i.e. including strata fixed effects), between groups is in column (3), with standard errors (in parentheses) are clustered at the school level. *Marginalization* is a variable coded 1 for areas with 'high' or 'very high' marginalization according to CONAPO, and 0 otherwise. The number of students and teachers is taken from the 'Formato 911' data for the 2015–16 academic year.

# **4 RESULTS**

## **Experimental results**

Our main estimating equation for student-level outcomes is

<span id="page-6-1"></span>
$$Y_{isg} = \alpha_g + \beta \ Treatment_s + \varepsilon_{isg}, \tag{1}$$

where *Yisg* is the outcome of interest for student *i* in school *s* in group *g* (denoting the stratification group used to assign treatment), *<sup>g</sup>* are strata fixed effects, *Treatments* is an indicator variable for a school *s* receiving grants, and *isg* is an error term. We use a similar specification without the *i* subscript to examine school-level outcomes. To test the robustness of our estimates and to increase the precision of the estimates, we also estimate versions where we control for (1) lagged test scores (i.e. the average school score in Mathematics and Language in the 2015 PLANEA exam), (2) student characteristics (age and gender), and (3) school characteristics (the level of marginalization, an indicator for urban location, an infrastructure index, and whether it is an indigenous school). We estimate these models using ordinary least squares, clustering the

<sup>\*, \*\*, \*\*\*</sup> indicate *p <* 0*.*10, *p <* 0*.*05, *p <* 0*.*01, respectively.

**TABLE 2** Effects on learning outcomes.

<span id="page-7-0"></span>

|                        | Mathematics<br>(1) | Language<br>(2) | Average<br>(3) | PCA<br>(4) |
|------------------------|--------------------|-----------------|----------------|------------|
| Treatment              | −0.0764            | −0.0745         | −0.0821        | −0.0818    |
|                        | (0.0735)           | (0.0686)        | (0.0741)       | (0.0739)   |
| Number of observations | 6799               | 6798            | 6673           | 6673       |

*Notes:* This table presents the treatment effects on learning outcomes (measured using PLANEA scores). The outcomes are Mathematics test scores (column (1)), Language test scores (column (2)), the average across subjects (column (3)), and a composite index across subjects (column (4)). All regressions take into account the randomization design (i.e. include strata fixed effects). Standard errors (in parentheses) are clustered at the school level.

standard errors at the school level. The coefficient of interest reflects the effect of receiving school grants on outcome *Yisg*.

PLANEA measures students' competencies in Mathematics and Language. We create two aggregate measures of students' ability: one by averaging students' scores across Mathematics and Language, and another using a principal component analysis (PCA). We focus on the PCA index for concreteness (and to avoid issues related to multiple hypothesis testing). However, our results are robust to using only Mathematics test scores, only Language test scores, or the average of these two.

Students in treatment schools scored 0*.*08 lower than those in control schools (*p*-value 0.27). We can rule out a positive effect greater than 0*.*06 at the 95% level (see Table [2\)](#page-7-0). This result is robust to student- and school-level controls (see Table [3\)](#page-8-0). If anything, there is suggestive evidence that the treatment effect is negative (−0*.*10, *p*-value 0.08). After including controls, we can rule out an effect greater than 0*.*01 at the 95% level. The distribution of test scores is statistically indistinguishable between treatment and control—although the control distribution stochastically dominates the treatment distribution (see Figure A.1 of the Online Appendix). Further, there is no evidence that the school grants affected other outcomes such as grade repetition or enrolment rates (see Table A.3 of the Online Appendix)—albeit we are underpowered to detect meaningful changes in these estimates.

One concern about the results from 2018 is that they reflect the 'persistent' effects of the grants a year after the last grant disbursement (which happened during the 2016–17 school year). We also have school-level aggregate data for the 2016 PLANEA scores, which measure learning outcomes while the programme is in effect[.17](#page-12-16) In line with the effects on the 2018 exam, we find a negative (but statistically insignificant) treatment effect on the likelihood students score in the top levels of the exam in 2016 (see Table [4\)](#page-8-1).

We estimate the effect of the per-student transfer on test scores by instrumenting the average transfer per student that each school received over the two years with the treatment assignment. The effects are still negative and small (see Table A.4 of the Online Appendix). An increase of 1 USD in the per student per year transfer lowered test scores in treatment schools (compared to control schools) by −0*.*007 (*p*-value 0.09). The results are similar if we do not instrument, but rather estimate directly the effect of the grant size (per pupil) on achievement.

# **Why do test scores not increase in response to the school grants?**

In this subsection, we try to shed light on why the grants do not improve test scores. We follow Glewwe and Muralidharan [\(2016\)](#page-14-0), and explore three possible reasons behind this lack of a treatment effect: (1) poor programme implementation; (2) a substitution away from other inputs (e.g. household expenditure); and (3) whether there are other binding constraints or whether complementary inputs/reforms are necessary for the grants to be effective (as in Mbiti *et al.* [2019\)](#page-15-6).

<sup>\*, \*\*, \*\*\*</sup> indicate *p <* 0*.*10, *p <* 0*.*05, *p <* 0*.*01, respectively.

**TABLE 3** Robustness of effects on learning outcomes.

<span id="page-8-0"></span>

|                        | PLANEA score |          |          |          |  |
|------------------------|--------------|----------|----------|----------|--|
|                        | (1)          | (2)      | (3)      | (4)      |  |
| Panel A: Mathematics   |              |          |          |          |  |
| Treatment              | −0.0764      | −0.0922  | −0.0952  | −0.0955  |  |
|                        | (0.0735)     | (0.0657) | (0.0655) | (0.0627) |  |
| Number of observations | 6799         | 6799     | 6799     | 6799     |  |
| Panel B: Language      |              |          |          |          |  |
| Treatment              | −0.0745      | −0.0953* | −0.0961* | −0.0934* |  |
|                        | (0.0686)     | (0.0549) | (0.0545) | (0.0511) |  |
| Number of observations | 6798         | 6798     | 6798     | 6798     |  |
| Panel C: Average       |              |          |          |          |  |
| Treatment              | −0.0821      | −0.103   | −0.105*  | −0.104*  |  |
|                        | (0.0741)     | (0.0625) | (0.0623) | (0.0585) |  |
| Number of observations | 6673         | 6673     | 6673     | 6673     |  |
| Panel D: PCA           |              |          |          |          |  |
| Treatment              | −0.0818      | −0.102   | −0.104*  | −0.104*  |  |
|                        | (0.0739)     | (0.0621) | (0.0619) | (0.0581) |  |
| Number of observations | 6673         | 6673     | 6673     | 6673     |  |
| Lagged scores          | No           | Yes      | Yes      | Yes      |  |
| Student controls       | No           | No       | Yes      | Yes      |  |
| School controls        | No           | No       | No       | Yes      |  |

*Notes:* This table presents the treatment effects on learning outcomes (measured using PLANEA scores). The outcome is a composite index across subjects. All regressions take into account the randomization design (i.e. include strata fixed effects). 'Lagged scores' indicates whether school average test scores from 2015 are included as controls. 'Student controls' indicates whether age and gender are included as controls. 'School controls' indicates whether the following controls are included: whether the school has a day shift, whether it serves an indigenous population, the school's age, whether the school is located in an urban area, and the marginalization index of the school's municipality. Standard errors (in parentheses) are clustered at the school level.

**TABLE 4** Effects on 2016 test scores.

<span id="page-8-1"></span>

|                        | Language |         |         |         | Mathematics |         |  |
|------------------------|----------|---------|---------|---------|-------------|---------|--|
|                        | (1)      | (2)     | (3)     | (4)     | (5)         | (6)     |  |
| Treatment              | −3.969   | −3.714  | −3.568  | −2.399  | −2.172      | −2.062  |  |
|                        | (3.527)  | (3.536) | (3.551) | (3.843) | (3.809)     | (3.782) |  |
| Number of observations | 200      | 200     | 200     | 200     | 200         | 200     |  |
| Control mean           | 30       | 30      | 30      | 41      | 41          | 41      |  |
| Lagged scores          | No       | Yes     | Yes     | No      | Yes         | Yes     |  |
| School controls        | No       | No      | Yes     | No      | No          | Yes     |  |

*Notes:* This table presents the treatment effects on the percentage of students in the two top achievement levels (out of 4). All regressions take into account the randomization design (i.e. include strata fixed effects). 'Lagged scores' indicates whether school average test scores from 2015 are included as controls. 'School controls' indicates whether the following controls are included: whether the school has a day shift, whether it serves an indigenous population, the school's age, whether the school is located in an urban area, and the marginalization index of the school's municipality. Standard errors (in parentheses) are clustered at the school level.

<sup>\*, \*\*, \*\*\*</sup> indicate *p <* 0*.*10, *p <* 0*.*05, *p <* 0*.*01, respectively.

<sup>\*, \*\*, \*\*\*</sup> indicate *p <* 0*.*10, *p <* 0*.*05, *p <* 0*.*01, respectively.

First, we can rule out poor programme implementation at the state level, as the administrative records show that the grants were indeed transferred to and received by the schools by the middle of the school year (around December). The due diligence performed by the World Bank (which partly funded the programme) did not find any evidence that funds were diverted elsewhere.

Next, we explore whether there is any substitution. Data from the surveys that students answered as part of PLANEA, suggest that this is not the case (see Table A.5 of the Online Appendix). Students in treatment schools are not more likely to report working (either in the family business or outside), and they are not less likely to report having adequate educational supplies and resources (e.g. textbooks, a computer and a desk) or having additional tutoring classes. The level of parental engagement (e.g. help with homework, and interest in school activities) is not different across treatment and control schools. In short, it does not seem as if households respond to the treatment by lowering their inputs.

Using data from other official government records, we explore whether treated schools are more or less likely to benefit from other programmes. In particular, we have data from the largest programmes at the time: Escuelas al Cien (Auditoria Superior de la Federeacion [2018\)](#page-13-11), Programa Fortalecimiento de la Calidad Educativa (Secretaría de Educación Pública [2017\)](#page-15-14), and more importantly, the Programa de la Reforma Educativa—the largest government programme at the time, which the then-president Enrique Peña Nieto championed (del Campo [2016\)](#page-14-17). We do not find evidence that government programmes favoured or discriminated against treatment schools (see Table A.6 of the Online Appendix).

Another possibility is that either students or teachers sort in response to the treatment. As mentioned above, the sample of students in 2018 looks similar in terms of observable characteristics across treatment and control schools (as does the number of students tested)—see Table [1,](#page-6-0) and Table A.2 of the Online Appendix. We also find no effect on teacher turnover (Table A.7) or on teacher characteristics (Table A.8), suggesting that the composition of teachers does not change in response to the treatment[.18](#page-12-17) Relatedly, we find no evidence that teachers adjust their teaching practices in response to the treatment (Table A.9)

Finally, we explore heterogeneous treatment effects by students' and schools' baseline characteristics. The goal is to provide insights into whether there are complementarities between the grants and other inputs in the education production function.[19](#page-13-12) We study heterogeneity by the student's gender, the school's baseline management quality, the school's marginalization index, and the school principal's gender and tenure.

<span id="page-9-0"></span>

|                        | Student<br>gender | Management<br>2015 | Principal<br>gender | Principal<br>tenure | Marginalization |
|------------------------|-------------------|--------------------|---------------------|---------------------|-----------------|
| Treatment              | −0.075            | −0.11              | −0.21**             | −0.060              | −0.079          |
|                        | (0.082)           | (0.092)            | (0.090)             | (0.092)             | (0.19)          |
| Treatment × Covariate  | −0.0096           | −0.023             | 0.28*               | −0.0043             | −0.0050         |
|                        | (0.065)           | (0.100)            | (0.15)              | (0.012)             | (0.21)          |
| Covariate              | 0.26***           | 0.092              | −0.098              | 0.013               | −0.31*          |
|                        | (0.047)           | (0.061)            | (0.11)              | (0.0082)            | (0.17)          |
| Number of observations | 6673              | 3860               | 6673                | 6649                | 6673            |
| Control mean           | 0.00              | 0.03               | 0.00                | −0.00               | 0.00            |

**TABLE 5** Heterogeneous effects on learning.

*Notes:* This table shows the results from estimating the equation in note 19 when the outcome variable is the PCA index from Mathematics and Language 2018 PLANEA scores. Student gender is equal to 1 if the student is a female (and 0 if they are male). 'Management 2015' refers to the index calculated with baseline information, 'Principal gender' takes value 1 for female principals, and 0 for males, 'Principal tenure' refers to the number of years as principal, and 'Marginalization' takes value 1 for schools located in areas with high or very high levels of marginalization. All regressions take into account the randomization design—i.e. include strata fixed effects. Standard errors (in parentheses) are clustered at the school level.

<sup>\*, \*\*, \*\*\*</sup> indicate *p <* 0*.*10, *p <* 0*.*05, *p <* 0*.*01, respectively.

<span id="page-10-0"></span>**FIGURE 3** Expenditure per student and treatment effects. *Notes*: These plots show the expenditure per student for different educational programmes (in 2016 USD) and the treatment effect that they had on learning outcomes (measured in standard deviations). The red dot in both plots corresponds to the programme that we study in this paper. (a) Randomized trials studying school grants around the world (see Table A.13 of the Online Appendix for details on each study). (b) Educational programmes in Mexico with

rigorous evaluations (see Table A.14 of the Online Appendix for details on each study).

#### (b)

We do not find evidence of heterogeneity by how well managed the school is or by the tenure of the principal. This suggests that there are no complementarities between management quality and school resources in this context. Finally, we do not find evidence that schools located in poorer areas benefit more from the grants.

There is some evidence that school grants have a negative treatment effect when the principal is male, and a null effect when the principal is female (see Table [5\)](#page-9-0). However, since the principal's gender may be correlated with other school attributes, we leave it to future work to test explicitly whether the principal's gender is an important factor in school grants' effectiveness.

The heterogeneity effects analysis reveals that schools in poorer areas do not benefit more from the treatment. This provides some suggestive evidence that the problem is not that the grant is too small to matter for some schools; however, other covariates correlated with poverty may explain this lack of effect. Further, reviewing previous findings using randomized controlled trials to study the effect of school grants in low- and middle-income countries, we find that despite a larger per-student grant amount (in 2016 USD), our treatment effects are among the smallest (see Figure [3\(a\)\)](#page-10-0). Comparing the cost and treatment effect with other education programmes that have been evaluated rigorously in Mexico (see Figure [3\(b\)\)](#page-10-0), there are many programmes—for example, diagnostic feedback (de Hoyos *et al.* [2017\)](#page-14-18), information on the returns to schooling (Avitabile and de Hoyos [2018\)](#page-13-13), extending the school year (Agüero and Beleche [2013\)](#page-13-14) or teacher incentives (Behrman *et al.* [2015\)](#page-13-15)—with lower costs and larger treatment effects; albeit, there are also more expensive programmes with treatment effects close to zero, such as need-based scholarships (de Hoyos *et al.* [2021\)](#page-14-19) or student incentives (Behrman *et al.* [2015\)](#page-13-15). It is unclear what makes a programme cost-effective in this setting, but overall, providing money directly to schools or to students does not seem to improve scores.

Overall, the results in this section suggest neither poor implementation nor a response by parents that could explain the lack of effect of the grants. Further, the heterogeneity analysis suggests no complementarities between management quality and resources in this context. The grants being 'too small' does not appear to be the problem either.

## **5 CONCLUSIONS**

We leverage the random assignment of school grants to study their impact on student learning in Puebla, Mexico. Overall, the grants did not improve learning; if anything, there is some evidence that they might have worsened student learning outcomes. Students in treatment schools scored 0*.*08 (*p*-value 0.27) *lower* in a nationwide standardized test (PLANEA) after three years. This result is robust to various student- and school-level controls (after which we can rule out an effect greater than 0*.*01 at the 95% confidence level), and there is no impact on other outcomes such as grade repetition or the number of students enrolled per grade.

We can rule out poor programme implementation at the state level as administrative records show that the grants were indeed transferred to and received by the schools. There is also no evidence of parents changing their behaviour in response to the treatment, or other programmes discriminating in favour of or against treatment schools.

There are at least a few other explanations that we cannot rule out. In general, we cannot rule out that there are other limiting factors or binding constraints in this setting, and that unless they are addressed, additional resources will prove ineffective. These factors or constraints may include low levels of accountability, low parental participation, and discretionary (as opposed to meritocratic) hiring of teachers and school principals. Further, providing schools with some budgetary autonomy was the main motivation behind the grants—under the idea that schools know their needs better than central government officials. However, this was not accompanied by other decentralization initiatives (e.g. related to curriculum or personnel decisions). This may explain why the results do not echo findings from education decentralization reforms elsewhere, which are positive in general (Barankay and Lockwood [2007;](#page-13-6) Galiani *et al.* [2008;](#page-14-7) Faguet and Sánchez [2008;](#page-14-8) Falch and Fischer [2012;](#page-14-9) Elacqua *et al.* [2021\)](#page-14-10).

#### **ACKNOWLEDGMENTS**

This study was possible thanks to the support of the Secteraría de Educación Pública (SEP) of Mexico. We are especially indebted to Pedro Velasco, Griselda Olmos, Lorenzo Baladrón, Germán Cervantes, Javier Treviño, and all the staff at the SEP Directorate of Education Management. Some of the results in this paper first appeared in the working paper 'School management, grants, and test scores: experimental evidence from Mexico' [\(http://hdl.handle.net/10986/35108,](http://hdl.handle.net/10986/35108) accessed 25 March 2024). We are especially grateful to Raissa Ebner, Renata Lemos and Daniela Scur for their collaboration in this project's early stages and subsequent discussions. Karina

Gómez provided excellent research assistance. The views expressed here are those of the authors alone and do not necessarily reflect the World Bank's opinions. Romero gratefully acknowledges financial support from the Asociación Mexicana de Cultura, A.C. All errors are our own.

## **ENDNOTES**

- <span id="page-12-0"></span><sup>1</sup> For example, the International Monetary Fund suggests that there is a fiscal gap that needs to be filled to achieve the sustainable development goals (SDG4) related to education (Gaspar *et al.* [2019;](#page-14-20) Lagarde [2018\)](#page-14-21). Likewise, the Global Partnership for Education and the Education Commission suggests that spending needs to double to achieve quality learning opportunities for all children (International Commission on Financing Global Education Opportunity [2016;](#page-14-22) Global Partnership for Education [2019\)](#page-14-23).
- <span id="page-12-1"></span><sup>2</sup> The school grants were initially part of a programme called *Programa Escuelas de Calidad*, which later became a component of *Escuela al Centro*. School councils (known as *Consejos Escolares de Participación Social*) are composed of parents, teachers, the school principal and the school supervisor.
- <span id="page-12-2"></span><sup>3</sup> *Plan Nacional para la Evaluación de los Aprendizajes* (PLANEA) is a nationwide standardized test that measures Mathematics and Spanish learning outcomes in grades 6, 9 and 12. See Subsection III for more details.
- <span id="page-12-3"></span><sup>4</sup> The treatment effect, after including student- and school-level controls, is −0*.*10 (*p*-value 0.08).
- <span id="page-12-4"></span><sup>5</sup> For example, Glewwe *et al.* [\(2009\)](#page-14-24) in Kenya, Blimpo *et al.* [\(2015\)](#page-13-16) in The Gambia, Das *et al.* [\(2013\)](#page-13-17) in India, Pradhan *et al.* [\(2014\)](#page-15-15) in Indonesia, Sabarwal *et al.* [\(2014\)](#page-15-16) in Sierra Leone, Beasley and Huillery [\(2016\)](#page-13-18) in Niger, Mbiti *et al.* [\(2019\)](#page-15-6) in Tanzania, and Carneiro *et al.* [\(2020\)](#page-13-19) in Senegal. Two closely related studies with experimental evidence from Mexico show that school grants did not impact student learning (Garcia Moreno *et al.* [2019;](#page-14-25) Barrera-Osorio *et al.* [2020\)](#page-13-8).
- <span id="page-12-5"></span><sup>6</sup> This contrasts with findings from Tanzania (Mbiti *et al.* [2019\)](#page-15-6) and India (Das *et al.* [2013\)](#page-13-17), where households lowered their expenditures (on education) in response to the school grants.
- <span id="page-12-6"></span><sup>7</sup> The description of the *Escuela al Centro* strategy is available online at [http://www.dof.gob.mx/nota\\_detalle\\_popup](http://www.dof.gob.mx/nota&uscore;detalle&uscore;popup.php?codigo=5488338) [.php?codigo=5488338,](http://www.dof.gob.mx/nota&uscore;detalle&uscore;popup.php?codigo=5488338) and the operating rules are available at [http://www.dof.gob.mx/nota\\_detalle.php?codigo](http://www.dof.gob.mx/nota&uscore;detalle.php?codigo=5509544&fecha=29/12/2017) [=5509544&fecha=29/12/2017](http://www.dof.gob.mx/nota&uscore;detalle.php?codigo=5509544&fecha=29/12/2017) (both accessed 24 March 2024).
- <span id="page-12-7"></span><sup>8</sup> The median school received ∼240 MXN per student in 2015, and ∼288 MXN per student in 2016. The average exchange rate was 15.88 MXN/USD in 2015, and 18.69 MXN/USD in 2016.
- <span id="page-12-8"></span><sup>9</sup> The criteria were part of the operating rules of the programme. See the legislation [\(https://www.dof.gob.mx/nota](https://www.dof.gob.mx/nota&uscore;detalle.php?codigo=5377404&fecha=26/12/2014) [\\_detalle.php?codigo=5377404&fecha=26/12/2014,](https://www.dof.gob.mx/nota&uscore;detalle.php?codigo=5377404&fecha=26/12/2014) accessed 24 March 2024) for further details.
- <span id="page-12-9"></span><sup>10</sup> These numbers are for the 2015–16 school year. We could not find reliable information on how schools spent their grant money in the 2016–17 school year.
- <span id="page-12-10"></span><sup>11</sup> For example, in 2016 USD, Glewwe *et al.* [\(2009\)](#page-14-24) study an increase of 3.72 USD per student, Das *et al.* [\(2013\)](#page-13-17) 3.73 USD per student, Blimpo *et al.* [\(2015\)](#page-13-16) 1.35 USD per student, Pradhan *et al.* [\(2014\)](#page-15-15) 5.95 USD per student, Beasley and Huillery [\(2016\)](#page-13-18) 2.14 USD per student, Carneiro *et al.* [\(2020\)](#page-13-19) 10.05 USD per student, and Barrera-Osorio *et al.* [\(2020\)](#page-13-8) 8.78 USD per student. In relative terms, the increase studied by Blimpo *et al.* [\(2015\)](#page-13-16) corresponds to 5% of the total school budget. The increase studied in Pradhan *et al.* [\(2014\)](#page-15-15) corresponds to 3.9% of the total school budget, but 14% after excluding teacher salaries. Finally, the increase in Carneiro *et al.* [\(2020\)](#page-13-19) corresponds to 7% of the total school budget, but 70% after excluding teacher salaries.
- <span id="page-12-11"></span><sup>12</sup> Based on historical data, we estimated an intracluster correlation of 0.23 on student outcomes, about 30 students per school, and we could explain 35% of the variation in the outcome variable using student and school characteristics. Our *ex post* minimum detectable effect is 0*.*16 (McKenzie and Ozier [2019\)](#page-15-17), suggesting that our *ex ante* power calculations
- <span id="page-12-12"></span>were correct. <sup>13</sup> A third treatment arm involved receiving management training from professional trainers (as well as the grant). That treatment is studied in a companion paper (Romero *et al.* [2022\)](#page-15-13), which documents that while management practices improve by 0*.*13, there were no meaningful improvements in test scores. Thus the total sample included 300 primary schools selected to participate in the experiment.
- <span id="page-12-13"></span><sup>14</sup> Multi-grade schools were excluded as the managerial intervention was focused on training school principals to coach teachers. In multi-grade schools, there are fewer classrooms (thus there are often multiple grades in the same classroom with the same teacher at the same time), and school principals also teach. Therefore they call for different school management models.
- <span id="page-12-14"></span><sup>15</sup> We ranked schools within each group based on their enrolment. Next, we assigned schools in a repeating sequence to 'no grants' (the control group in this study) or 'grants' (the treatment group); the order in which the sequence began
- <span id="page-12-15"></span>was randomized. <sup>16</sup> The data are available at [https://sep.gob.mx/es/sep1/Articulo\\_73\\_de\\_la\\_Ley\\_General\\_de\\_Contabilidad](https://sep.gob.mx/es/sep1/Articulo&uscore;73&uscore;de&uscore;la&uscore;Ley&uscore;General&uscore;de&uscore;Contabilidad&uscore;Gubernamental) [\\_Gubernamental.](https://sep.gob.mx/es/sep1/Articulo&uscore;73&uscore;de&uscore;la&uscore;Ley&uscore;General&uscore;de&uscore;Contabilidad&uscore;Gubernamental) Some teachers are paid by the state government. The data for teachers paid by the state are
- <span id="page-12-16"></span>unavailable. <sup>17</sup> The 2017 PLANEA exam did not include primary students.
- <span id="page-12-17"></span><sup>18</sup> A related concern is that some treatment and control schools are near each other (see Figure [2\)](#page-4-0), which could lead to spillovers to control schools. While we do not find evidence that students or teachers sort in response to the treatment, spillovers could happen through other channels. To address this concern, we estimate the treatment effects restricting

- the set of schools to those that are relatively isolated (see Table A.10 of the Online Appendix) and find similar effects. Following Bobba and Gignoux [\(2019\)](#page-13-20), we also estimate the treatment effects controlling for the number of schools that are treated nearby (Table A.11) or for the fraction of schools nearby that are treated (Table A.12), and find that the treatment effect is more negative. Overall, the lack of a positive treatment effect of the grants does not seem to be driven by spillovers.
- <span id="page-13-12"></span><sup>19</sup> Specifically, we estimate the equation *Yisg* = *<sup>g</sup>* + <sup>1</sup> *Treatments* + <sup>2</sup> *Treatments* × *cs* + 3*cs* + *isg*, where *cs* denotes the school or student characteristics that we use to measure heterogeneity, and <sup>2</sup> allows us to test whether there is any differential treatment effect. Everything else is the same as in equation [\(1\)](#page-6-1).

## **REFERENCES**

- Abadie, A. (2002). Bootstrap tests for distributional treatment effects in instrumental variable models. *Journal of the American Statistical Association*, **97**(457), 284–92.
- Abdulkadiroglu, A., Pathak, P. A. and Walters, C. R. (2018). Free to choose: can school choice reduce student *̆* achievement? *American Economic Journal: Applied Economics*, **10**(1), 175–206.
- <span id="page-13-14"></span>Agüero, J. M. and Beleche, T. (2013). Test-Mex: estimating the effects of school year length on student performance in Mexico. *Journal of Development Economics*, **103**, 353–61.
- <span id="page-13-11"></span>Auditoria Superior de la Federeacion (2018). Auditoría especial del gasto federalizado: programa escuelas al cien; available online at [https://www.asf.gob.mx/Trans/Informes/IR2016ii/Documentos/Informes\\_Especiales/Informes](https://www.asf.gob.mx/Trans/Informes/IR2016ii/Documentos/Informes_Especiales/Informes_Especiales_05.pdf) [\\_Especiales\\_05.pdf](https://www.asf.gob.mx/Trans/Informes/IR2016ii/Documentos/Informes_Especiales/Informes_Especiales_05.pdf) (accessed 25 March 2024).
- <span id="page-13-13"></span>Avitabile, C. and de Hoyos, R. (2018). The heterogeneous effect of information on student performance: evidence from a randomized control trial in Mexico. *Journal of Development Economics*, **135**, 318–48.
- <span id="page-13-6"></span>Barankay, I. and Lockwood, B. (2007). Decentralization and the productive efficiency of government: evidence from Swiss cantons. *Journal of Public Economics*, **91**(5), 1197–218.
- <span id="page-13-3"></span>Bardhan, P. (2002). Decentralization of governance and development. *Journal of Economic Perspectives*, **16**(4), 185–205. (2016). State and development: the need for a reappraisal of the current literature. *Journal of Economic Literature*, **54**(3), 862–92.
- <span id="page-13-5"></span><span id="page-13-4"></span>and Mookherjee, D. (2006). Decentralisation and accountability in infrastructure delivery in developing countries. *Economic Journal*, **116**(508), 101–27.
- <span id="page-13-8"></span>Barrera-Osorio, F., Gertler, P., Nakajima, N. and Patrinos, H. (2020). Promoting parental involvement in schools: evidence from two randomized experiments. NBER Working Paper no. 28040.
- <span id="page-13-0"></span>and Linden, L. L. (2009). The use and misuse of computers in education: evidence from a randomized experiment in Colombia. World Bank Policy Research Working Paper no. 4836.
- <span id="page-13-18"></span>Beasley, E. and Huillery, E. (2016). Willing but unable? Short-term experimental evidence on parent empowerment and school quality. *World Bank Economic Review*, **31**(2), 531–52.
- <span id="page-13-15"></span>Behrman, J. R., Parker, S. W., Todd, P. E. and Wolpin, K. I. (2015). Aligning learning incentives of students and teachers: results from a social experiment in Mexican high schools. *Journal of Political Economy*, **123**(2), 325–64.
- , , and Zhang, W. (2021). *Prospering through Prospera: CCT impacts on educational attainment and achievement in Mexico*. Technical Report, Faculty of Economics, University of Cambridge.
- <span id="page-13-1"></span>Beuermann, D. W., Cristia, J., Cueto, S., Malamud, O. and Cruz-Aguayo, Y. (2015). One laptop per child at home: short-term impacts from a randomized experiment in Peru. *American Economic Journal: Applied Economics*, **7**(2), 53–80.
- <span id="page-13-16"></span>Blimpo, M. P., Evans, D. K. and Lahire, N. (2015). Parental human capital and effective school management: evidence from The Gambia. World Bank Policy Research Working Paper no. 7238.
- <span id="page-13-10"></span>Bloom, N., Eifert, B., Mahajan, A., McKenzie, D. and Roberts, J. (2013). Does management matter? Evidence from India. *Quarterly Journal of Economics*, **128**(1), 1–51.
- <span id="page-13-9"></span>and Van Reenen, J. (2007). Measuring and explaining management practices across firms and countries. *Quarterly Journal of Economics*, **122**(4), 1351–408.
- <span id="page-13-20"></span>Bobba, M. and Gignoux, J. (2019). Neighborhood effects in integrated social policies. *World Bank Economic Review*, **33**(1), 116–39.
- <span id="page-13-19"></span>Carneiro, P., Koussihouede, O., Lahire, N., Meghir, C. and Mommaerts, C. (2020). School grants and education quality: experimental evidence from Senegal. *Economica*, **87**(345), 28–51.
- <span id="page-13-7"></span>Consejo Nacional de Evaluación de la Política de Desarrollo Social (2018). *Informe de Pobreza y Evaluación 2018—Puebla*; available online at [https://www.coneval.org.mx/coordinacion/entidades/Documents/Informes](https://www.coneval.org.mx/coordinacion/entidades/Documents/Informes_de_pobreza_y_evaluacion_2018_Documentos/Informe_Puebla_2018.pdf) [\\_de\\_pobreza\\_y\\_evaluacion\\_2018\\_Documentos/Informe\\_Puebla\\_2018.pdf](https://www.coneval.org.mx/coordinacion/entidades/Documents/Informes_de_pobreza_y_evaluacion_2018_Documentos/Informe_Puebla_2018.pdf) (accessed 30 March 2024).
- <span id="page-13-2"></span>Cristia, J., Ibarraran, P., Cueto, S., Santiago, A. and Severin, E. (2017). Technology and child development: evidence from the one laptop per child program. *American Economic Journal: Applied Economics*, **9**(3), 295–320.
- <span id="page-13-17"></span>Das, J., Dercon, S., Habyarimana, J., Krishnan, P., Muralidharan, K. and Sundararaman, V. (2013). School inputs, household substitution, and test scores. *American Economic Journal: Applied Economics*, **5**(2), 29–57.

- de Hoyos, R., Attanasio, O. and Meghir, C. (2019). Targeting high school scholarships to the poor: the impact of a program in Mexico. NBER Working Paper no. 26023.
- <span id="page-14-19"></span>, Ganimian, A. J. and Holland, P. A. (2021). Teaching with the test: experimental evidence on diagnostic feedback and capacity building for public schools in Argentina. *World Bank Economic Review*, **35**(2), 499–520.
- <span id="page-14-18"></span>, García-Moreno, V. and Patrinos, H. A. (2017). The impact of an accountability intervention with diagnostic feedback: evidence from Mexico. *Economics of Education Review*, **58**, 123–40.
- <span id="page-14-17"></span>del Campo, A. F. M. (2016). Programa de la reforma educativa: dimensión y análisis de sus lineamientos. *Revista Latinoamericana de Estudios Educativos (México)*, **46**(1), 79–94.
- <span id="page-14-10"></span>Elacqua, G., Munevar, I., Sanchez, F. and Santos, H. (2021). The impact of decentralized decision-making on student outcomes and teacher quality: evidence from Colombia. *World Development*, **141**, 105378.
- <span id="page-14-8"></span>Faguet, J.-P. and Sánchez, F. (2008). Decentralization's effects on educational outcomes in Bolivia and Colombia. *World Development*, **36**(7), 1294–316.
- <span id="page-14-6"></span>Fairlie, R. W. and Robinson, J. (2013). Experimental evidence on the effects of home computers on academic achievement among schoolchildren. *American Economic Journal: Applied Economics*, **5**(3), 211–40.
- <span id="page-14-9"></span>Falch, T. and Fischer, J. A. (2012). Public sector decentralization and school performance: international evidence. *Economics Letters*, **114**(3), 276–9.
- <span id="page-14-7"></span>Galiani, S., Gertler, P. and Schargrodsky, E. (2008). School decentralization: helping the good get better, but leaving the poor behind. *Journal of Public Economics*, **92**(10–11), 2106–20.
- <span id="page-14-25"></span>Garcia Moreno, V. A., Gertler, P. J. and Patrinos, H. (2019). School-based management and learning outcomes: experimental evidence from Colima, Mexico. World Bank Policy Research Working Paper no. 8874.
- <span id="page-14-20"></span>Gaspar, V., Amaglobeli, D., Garcia-Escribano, M., Prady, D. and Soto, M. (2019). Fiscal policy and development: human, social, and physical investments for the SDGs. International Monetary Fund Staff Discussion Note 19/03.
- <span id="page-14-24"></span>Glewwe, P., Kremer, M. and Moulin, S. (2009). Many children left behind? Textbooks and test scores in Kenya. *American Economic Journal: Applied Economics*, **1**(1), 112–35.
- <span id="page-14-0"></span>and Muralidharan, K. (2016). Improving education outcomes in developing countries: evidence, knowledge gaps, and policy implications. In S. M. E. A. Hanushek and L. Woessmann (eds), *Handbook of Economics of Education*, Vol. **5**. Amsterdam: Elsevier, pp. 653–743.
- <span id="page-14-23"></span>Global Partnership for Education (2019). *Guidelines for the monitoring of national education budgets*. Technical Report, Global Partnership for Education, Washington, DC.
- <span id="page-14-13"></span>Instituto Nacional de Estadistica y Geografia (2016a). Estructura económica de Puebla en síntesis; available online at [https://www.inegi.org.mx/contenidos/productos/prod\\_serv/contenidos/espanol/bvinegi/productos/nueva\\_estruc](https://www.inegi.org.mx/contenidos/productos/prod_serv/contenidos/espanol/bvinegi/productos/nueva_estruc/702825086213.pdf) [/702825086213.pdf](https://www.inegi.org.mx/contenidos/productos/prod_serv/contenidos/espanol/bvinegi/productos/nueva_estruc/702825086213.pdf) (accessed 30 March 2024).
- <span id="page-14-12"></span>(2016b). Panorama sociodemogrífico de Puebla 2015; available online at [https://www.inegi.org.mx/contenido](https://www.inegi.org.mx/contenido/productos/prod_serv/contenidos/espanol/bvinegi/productos/nueva_estruc/inter_censal/panorama/702825082314.pdf) [/productos/prod\\_serv/contenidos/espanol/bvinegi/productos/nueva\\_estruc/inter\\_censal/panorama/702825082314](https://www.inegi.org.mx/contenido/productos/prod_serv/contenidos/espanol/bvinegi/productos/nueva_estruc/inter_censal/panorama/702825082314.pdf) [.pdf](https://www.inegi.org.mx/contenido/productos/prod_serv/contenidos/espanol/bvinegi/productos/nueva_estruc/inter_censal/panorama/702825082314.pdf) (accessed 30 March 2024).
- <span id="page-14-15"></span>(2018). Marco geoestadístico de Mexico; available online at <https://www.inegi.org.mx/temas/mg/default.html> (accessed 29 March 2024).
- <span id="page-14-14"></span>Instituto Nacional para la Evaluación de la Educación (2016). Plan nacional para la evaluación de los aprendizajes (PLANEA): resultados nacionales 2015 available online at [https://www.inee.edu.mx/wp-content/uploads/2019/02](https://www.inee.edu.mx/wp-content/uploads/2019/02/resultadosPlanea-3011-2015.pdf) [/resultadosPlanea-3011-2015.pdf](https://www.inee.edu.mx/wp-content/uploads/2019/02/resultadosPlanea-3011-2015.pdf) (accessed 30 March 2024).
- <span id="page-14-11"></span>(2018). Resultados de PLANEA; available online at [https://www.inee.edu.mx/evaluaciones/planea/resultados](https://www.inee.edu.mx/evaluaciones/planea/resultados-planea) [-planea](https://www.inee.edu.mx/evaluaciones/planea/resultados-planea) (accessed 29 March 2024).
- <span id="page-14-22"></span>International Commission on Financing Global Education Opportunity (2016). *The Learning Generation: Investing in Education for a Changing World*. New York: International Commission on Financing Global Education Opportunity.
- <span id="page-14-4"></span>Jackson, C. K. (2018). Does school spending matter? The new literature on an old question. NBER Working Paper no. 25368.
- <span id="page-14-2"></span>, Johnson, R. C. and Persico, C. (2016). The effects of school spending on educational and economic outcomes: evidence from school finance reforms. *Quarterly Journal of Economics*, **131**(1), 157–218.
- and Mackevicius, C. (2021). The distribution of school spending impacts. NBER Working Paper no. 28517.
- <span id="page-14-3"></span><span id="page-14-1"></span>, Wigger, C. and Xiong, H. (2021). Do school spending cuts matter? Evidence from the Great Recession. *American Economic Journal: Economic Policy*, **13**(2), 304–35.
- <span id="page-14-21"></span>Lagarde, C. (2018). Keynote address at Secretary-General's high-level meeting on financing the 2030 agenda for sustainable development. IMF, 24 September; available online at [https://www.imf.org/en/News/Articles/2018/09/24](https://www.imf.org/en/News/Articles/2018/09/24/sp092418-keynote-address-at-secretary-general-s-high-level-meeting) [/sp092418-keynote-address-at-secretary-general-s-high-level-meeting](https://www.imf.org/en/News/Articles/2018/09/24/sp092418-keynote-address-at-secretary-general-s-high-level-meeting) (accessed 29 March 2024).
- <span id="page-14-16"></span>Lemos, R. and Scur, D. (2016). Developing management: an expanded evaluation tool for developing countries. RISE Working Paper no. 16(007).
- <span id="page-14-5"></span>Malamud, O. and Pop-Eleches, C. (2011). Home computer use and the development of human capital. *Quarterly Journal of Economics*, **126**(2), 987–1027.

- <span id="page-15-6"></span>Mbiti, I., Muralidharan, K., Romero, M., Schipper, Y., Manda, C. and Rajani, R. (2019). Inputs, incentives, and complementarities in education: experimental evidence from Tanzania. *Quarterly Journal of Economics*, **134**(3), 1627–73.
- <span id="page-15-5"></span>McEwan, P. J. (2015). Improving learning in primary schools of developing countries: a meta-analysis of randomized experiments. *Review of Educational Research*, **85**(3), 353–94.
- <span id="page-15-17"></span>McKenzie, D. and Ozier, O. (2019). Why ex-post power using estimated effect sizes is bad, but an ex-post MDE is not. World Bank Blogs, 16 May; available online at [https://blogs.worldbank.org/impactevaluations/why-ex-post-power](https://blogs.worldbank.org/impactevaluations/why-ex-post-power-using-estimated-effect-sizes-bad-ex-post-mde-not) [-using-estimated-effect-sizes-bad-ex-post-mde-not](https://blogs.worldbank.org/impactevaluations/why-ex-post-power-using-estimated-effect-sizes-bad-ex-post-mde-not) (accessed 29 March 2024).
- <span id="page-15-8"></span>Muralidharan, K., Singh, A. and Ganimian, A. J. (2019, April). Disrupting education? Experimental evidence on technology-aided instruction in India. *American Economic Review*, **109**(4), 1426–60.
- <span id="page-15-12"></span>OECD (2016). Education at a glance 2016; available online at [https://www.oecd-ilibrary.org/content/publication/eag](https://www.oecd-ilibrary.org/content/publication/eag-2016-en) [-2016-en](https://www.oecd-ilibrary.org/content/publication/eag-2016-en) (accessed 29 March 2024).
- <span id="page-15-3"></span>(2018). Education policy outlook 2018: putting student learning at the centre—Mexico; available online at <https://www.oecd-ilibrary.org/content/component/9789264301528-27-en> (accessed 29 March 2024).
- <span id="page-15-4"></span>(2019). Education at a glance 2019—Mexico; available online at [https://www.oecd-ilibrary.org/content](https://www.oecd-ilibrary.org/content/component/f12055c8-en) [/component/f12055c8-en](https://www.oecd-ilibrary.org/content/component/f12055c8-en) (accessed 29 March 2024).
- <span id="page-15-15"></span>Pradhan, M., Suryadarma, D., Beatty, A., Wong, M., Gaduh, A., Alisjahbana, A. and Artha, R. P. (2014). Improving educational quality through enhancing community participation: results from a randomized field experiment in Indonesia. *American Economic Journal: Applied Economics*, **6**(2), 105–26.
- <span id="page-15-13"></span>Romero, M., Bedoya, J., Yanez-Pagans, M., Silveyra, M. and de Hoyos, R. (2022). Direct vs indirect management training: experimental evidence from schools in Mexico. *Journal of Development Economics*, **154**, 102779.
- <span id="page-15-16"></span>Sabarwal, S., Evans, D. K. and Marshak, A. (2014). The permanent input hypothesis: the case of textbooks and (no) student learning in Sierra Leone. World Bank Policy Research Working Paper no. 7021.
- <span id="page-15-14"></span>Secretaría de Educación Pública (2017). Acuerdo número 20/12/17, por el que se emiten las reglas de operación del programa fortalecimiento de la calidad educativa para el ejercicio fiscal 2018. *Diario Oficial de la Federación.*
- <span id="page-15-7"></span>Vigdor, J. L., Ladd, H. F. and Martinez, E. (2014). Scaling the digital divide: home computer technology and student achievement. *Economic Inquiry*, **52**(3), 1103–19.
- <span id="page-15-9"></span>World Bank (2003). *World Development Report 2004*. Washington, DC: World Bank; available online at [https://elibrary](https://elibrary.worldbank.org/doi/abs/10.1596/0-8213-5468-X) [.worldbank.org/doi/abs/10.1596/0-8213-5468-X](https://elibrary.worldbank.org/doi/abs/10.1596/0-8213-5468-X) (accessed 29 March 2024).
- <span id="page-15-2"></span>(2005). Mexico: determinants of learning policy note. Report no. 31842-MX; available online at [http://hdl.handle](http://hdl.handle.net/10986/8284) [.net/10986/8284](http://hdl.handle.net/10986/8284) (accessed 29 March 2024).
- (2017a). Government expenditure on education, total (% of GDP)—low and middle income; available online at <https://data.worldbank.org/indicator/SE.XPD.TOTL.GD.ZS?locations=XO> (accessed 29 March 2024).
- <span id="page-15-10"></span><span id="page-15-0"></span>(2017b). Government expenditure on education, total (% of GDP)—Mexico; available online at [https://data](https://data.worldbank.org/indicator/SE.XPD.TOTL.GD.ZS?locations=MX) [.worldbank.org/indicator/SE.XPD.TOTL.GD.ZS?locations=MX](https://data.worldbank.org/indicator/SE.XPD.TOTL.GD.ZS?locations=MX) (accessed 29 March 2024).
- <span id="page-15-11"></span>(2017c). Primary completion rate, total (% of relevant age group); available online at [https://data.worldbank.org](https://data.worldbank.org/indicator/SE.PRM.CMPT.ZS?locations=MX) [/indicator/SE.PRM.CMPT.ZS?locations=MX](https://data.worldbank.org/indicator/SE.PRM.CMPT.ZS?locations=MX) (accessed 29 March 2024).
- <span id="page-15-1"></span>(2017d). *World Development Report 2018: Learning to Realize Education's Promise*. Washington, DC: World Bank; available online at <https://elibrary.worldbank.org/doi/abs/10.1596/978-1-4648-1096-1> (accessed 29 March 2024).

## **SUPPORTING INFORMATION**

Additional supporting information can be found online in the Supporting Information section at the end of this article.

**How to cite this article:** Romero, M., Bedoya, J., Yanez-Pagans, M., Silveyra, M. and de Hoyos, R. (2024). The effect of school grants on test scores: experimental evidence from Mexico. *Economica*, 1–16. <https://doi.org/10.1111/ecca.12523>