1. INTRODUCTION

A fact widely recognized in economic literature is that investment in human capital plays an important role in determining the economic growth of a country. The literature which analyzes public-policy programs such as scholarships is normally based on a theoretical framework of training human capital.

Likewise, there is also empirical evidence which constitutes a frame of reference to analyze these kinds of policies and which suggests that an investment in the quantity and quality of education which individuals receive depends on the returns as perceived by the parents (^{Woessmann and Hanushek, 2012}; ^{Glewwe, 2002}). Therefore, the challenge in developing countries is applying public policies which make a family’s investment in education more profitable, providing them with incentives for girls and boys to stay in school for longer.

In this light, one of the strategies is employing cash transfer programs which are conditional upon a greater scholastic participation. These public policies could have positive impacts in the profitability as seen by the families, as they not only generate an immediate increase in the income of underprivileged households, but also bring about a change in these households’ perception regarding the quantity and quality of the human capital receiving the scholarship. The result could be an increase in expectations regarding returns.

In 2001, the Scholarship Program for Socially Vulnerable Girls and Boys in Mexico City¹ was implemented (to be referred to as the Scholarship Program from here on), whose objective is to increase the participation and permanence in school of girls and boys who are socially vulnerable and of an age to attend basic education in Mexico.² The program consists of bestowing a monthly cash transfer of $800 MXN to participating families upon the condition that the children are enrolled in the current school year.

The logic behind this scholarship is to contribute to the scholastic permanence of those receiving the scholarship, thereby influencing their scholastic performance. The former is very possible. As the resources available to the families are thereby increased so that they can invest in education, health, and food, there is a greater motivation on behalf of the parents or guardians for their children to stay in school.

The primary objective of this paper is to determine the impact of the Scholarship Program in Mexico City on how parents perceive the academic performance of their children. For this, we used a new database resultant from a representative survey covering all the boroughs of Mexico City, carried out specifically to evaluate the impact of the program.

With the aim of estimating said impact, we used a theoretical model of the determinants of scholastic grades in the home, the impact of which contemplates the influence of a public policy program. We used a multiple regression and an ordered probit regression model, the latter of which allowed taking into consideration the categorical nature of the dependent variable. Based on said ordered probit model, we estimated the marginal effects of the different response categories.

The empirical results from the model suggest that there is a positive and significant impact had by Scholarship Program in Mexico City on the families’ perception of their children's grades. Indeed, the parents of those participating in the program perceive the grades that their children get in school as being higher.

Specifically, the probability that the parents perceive high grades (between 9 and 10)³ increases by 18.1% for the beneficiaries, when compared to the parents in the control group. At the same time, the probability that the parents perceive a lower grade (than 9) is significantly reduced in the treatment group (the perception of grades between 8 and 9 is reduced by 11.3% and grades between 6 and 8 by 6.7%) when compared to the control group.

This paper is made up of six sections, including this introduction. The second explains the workings of the Scholarship Program and summarizes the precedents in evaluating cash transfers programs and their impact on scholastic achievements by way of comparing various countries. The third section presents the descriptive statistics of the variables collected by the survey. The fourth details the theoretical framework and the estimation strategy. The fifth section analyzes the results of the quantitative impact upon perception. Finally, we briefly discuss the conclusions in the sixth section.

2. THE SCHOLARSHIP PROGRAM AND EMPIRICAL EVIDENCE

The Scholarship Program in Mexico City came into effect for the first time in 2001, with the aim of contributing to socially vulnerable boys and girls⁴ staying in school longer and for them to finish their basic education. The organism in charge of the operation is the National System for Integral Family Development in Mexico City (DIF-CDMX)⁵ . According to numbers from the National Council for the Evaluation of Social Development Policy (CONEVAL)⁶ , in 2015 there were 246,473 girls and boys who were vulnerable due to social inequalities (^{Gaceta Oficial Distrito Federal, 2015} - ²⁰¹⁸).

In 2015, the program served 25,500 students between 6 and 14 years of age, then enrolled in public schools at the primary and junior high level, to whom they bestowed a financial support conditional on staying in school in the form of $800 MXN per month, for a maximum period of three years. The budget allotted to the program at that time reached $24,800,000 MXN. Another pillar of the operation is the participation of beneficiaries in cultural and recreational activities (^{Gaceta Oficial Distrito Federal, 2015} - ²⁰¹⁸).

The cash transfer programs which are conditional upon children’s scholastic enrolment have been implemented in different developing countries in recent decades (^{Fiszbein et al., 2009}; ^{Inter-American Development Bank, 2017}; ^{Stampini and Tornarolli, 2012}). There is evidence that this kind of program has had significant growth in the Caribbean and Latin American region; likewise, a general consensus has come about on the presence of a causal relationship between these cash transfer programs and indicators of children staying in school, such as enrolment, attendance, and grades completed (^{Glewwe and Kassouf, 2012}; ^{Brauw, et al., 2015}; ^{Behrman, et al., 2009}).

Nevertheless, for this kind of program to have a more relevant impact there needs to be greater benefits in the actual acquisition of skills and knowledge needed for future work (^{Woessmann and Hanushek, 2012}). A traditional method for evaluating the acquisition of skills by students is through their continuing in school and the grades which they get. Some of the cash transfer programs created in developing countries show a positive impact on the results of indices of scholastic performance. This paper focuses on the analysis of the perception parents have of their children's grade average, as well as how it varies by participating in the public Scholarship Program in Mexico City.

In this regard, ^{Stampini et al., (2018)} study the impacts of the Programme of Advancement Through Health and Education (PATH) which is a cash transfer program conditional on attending school throughout basic education in Jamaica. This study generated results on two types of scholastic achievements: a) school grades; and b) placement in secondary schools. The findings suggest that there is an increase in the grades of male students which can be attributed to receiving cash transfers. Likewise, placement in higher ranking secondary schools also had a positive effect in the case of boys who were beneficiaries in the program compared to those who were not. Nevertheless, the authors found no evidence of any significant effect in the case of girls and their results.

Another cash transfer program conditional on staying in school which is given to poor families in Brazil is evaluated by ^{Simões and Sabates (2014)}. The findings of the investigation, based on a cross-section model as well as a fixed effects panel model attributes a positive effect on grades in standardized tests taken by students receiving this benefit which furthermore is reliant on the length of time they have been in the program.

^{Garcia and Hill (2010)} studied the effects of the Colombian program of conditional cash transfers ("Families in Action"⁷ ) and the school grades of children and teenagers. Via a propensity score matching technique, the authors find the program has a positive effect on scores in scholastic exams. Though this relationship is true for students between six and 12 in rural areas, in urban areas there is no significant effect. Therefore, the effect is negative in teenagers between 13 and 16 years of age.

Meanwhile there are other observable as well as non-observable factors which determine the results of children's learning. Some studies show that personal traits play an important role. Evidence specifically points out the function which some families traditionally assigned girls as continuing to be a determinant in the results of students’ grades.

3. DATA

The data comes from the survey given to households specifically designed to evaluate the impact of the Scholarship Program for socially vulnerable girls and boys in Mexico City. This survey is the first to be carried out to analyze the effect of the program and represents a baseline as such. The surveys were carried out in the month of January 2016 and their design follows a comparative approach between a treatment group and the control group sampled from all the boroughs of Mexico City. Said surveys provide information on the students in primary and junior high for the 2014-2015 school year, as well as a wide set of variables. It is important to highlight that the program has had no significant changes in its design since its creation in 2001 until the evaluation carried out in 2016 (^{Gaceta Oficial Distrito Federal, 2016}).

All the students in public primaries or junior highs in Mexico City were considered eligible for the study if they met at least one of the following criteria: a) were beneficiaries of the scholarship program in the 2014-2015 school year, or b) were on the waitlist to be accepted into the program for the same school year. As such, the group studied was 25,500 beneficiaries (treatment group) and 1516 students on the waitlist (control group). From those groups, we chose a random sampling with representatives from each borough for a total of 201 girls and boys where 104 (51.7%) students belonged to the treatment group and 97 (48.3%) the control group.

The information on the students’ status and participation in the Scholarship Program was gotten directly from the program's administrative registries. Nevertheless, to corroborate their status we implemented a filter at the beginning of the questionnaire to thereby ensure the correct assignation of the students in each group, be it as a beneficiary or on the waitlist. With this procedure, we discarded those who did not meet the requirements of being in one group or the other during the school year under evaluation, in spite of them being on the register.

The survey collected information on the perception had by mothers, fathers, and guardians⁸ regarding the children’s overall grades during the 2014-2015 school year. With this we obtained the indicator of scholastic results.⁹ This indicator had three response categories: the first of these was with regards to the perception of grades between 9 and 10, the second for those between 8 and 9, and the third is for those between 7 and 8. The students were considered for the study only if the guardians provided information on the overall grade average.¹⁰ As such, the indicator presented values for a total of 182 of 201 selected students.¹¹

An initial look at the data (see Figure 1) shows that the opinions of mothers, fathers and guardians of both students benefiting as well as those who are not, have very similar percentages in regards to grade averages between 9 and 10 (44.2% and 44.8% for each group, respectively), which can be categorized as the best grade averages. Meanwhile, the percentage of students with intermediate averages between 8 and 9 is higher in the control group (42.7%) than in the treatment group (36.1%). On the other hand, the percentage of students with a lower range (between 7 and 8) is greater among the group of beneficiaries (19.8%) compared to that of those on the waitlist (12.5%).

Source: created by the authors with data from the survey evaluating the impact of the Scholarship Program in Mexico City (2016).

Figure 1 Perception of grade averages for the 2014-2015 school year for the treatment and control groups 

If acceptance into the program were carried out in a random manner, this data could signal a precise estimation between participating in the program and the results of how grades are perceived. However, due to acceptance not working in this fashion, the results of simple differences cannot yet be interpreted in causal terms as they do not take into consideration the impact of other observable as well as non-observable characteristics on the indicator’s result.

In order to control all the characteristics which could impact the final results of the perception of grades, we incorporated the groups of regressors which, according to empirical evidence (^{Glewwe and Lambert, 2010}; ^{Glewwe and Kremer, 2006}), make up a production function of scholastic achievements. Table 1 shows a statistical summary of the primary variables of each group: personal, family, and locality traits, further controlled by participation in the public policy program.

The first, second and third column of each table are for the total sample, the treatment group, and control group respectively and represent the average values for each variable, as well as its standard deviation. The fourth column shows the results of simple average differences for each trait between the treatment and control groups. In the fifth column, we carry out the same test but also taking into consideration the effects of geographical location.

The data from Table 1 points out that the sampling of students is made up of 50.7% girls and 49.3% boys. The distribution of said students is 29.4% in junior high and 70.6% in primary¹² according to their level of enrolment; 83.8% of guardians considered their children to have a good relationship with their classmates and 91.9% stated that they were well behaved in school. On average, students dedicate 1.2 hours to studying in addition to the time needed to do their homework; 90.0% of students were the children of the household’s primary provider.

The family background is presented in Table 2. The data shows that 91.5% of guardians are female and in 81.9% of the cases, the mother is directly responsible for the student’s scholastic activities; 15.4% of guardians only finished elementary school while 8.7% were junior high dropouts. 64.7% of the time, the guardians reported as being employed. With regards to scholastic support, 86.6% of guardians mentioned reviewing the children's homework very frequently. While the rest did it infrequently or never.

Table 3 presents the traits of the homes and communities. Here we see that 61.7% of households have a phone in the home; on the other hand, only 40.8% have Internet in the household. The average monthly expenses which vulnerable households allot to education is $318.3 MXN. On the other hand, the expenses allotted to health are only $118.4 MXN per month while expenses dedicated to the household are $468.8 MXN. Regarding the conditions of the community, a factor which could be relevant is the time spent commuting by students to get to and from school. The participants take on average 31.5 minutes on this commute.

The testing of differences in averages of the variables, both in their simple form as well as controlling the fixed effects of geographical location, does not show any statistically significant discrepancies in either of the groups. This could indicate that both groups are highly comparable.

4. FRAME OF REFERENCE

The process of students accruing scholastic achievements can be analyzed using a production function approach. With this aim, following the theoretical framework developed by ^{Glewwe and Lambert (2010)} and ^{Glewwe and Kremer (2006)}, we employed this methodology in order to evaluate the impact of this public policy which is geared towards aiding socially vulnerable girls and boys in Mexico City in their scholastic achievements. Overall, the accruement of scholastic achievements can be determined by variables associated with the student, their household traits and characteristics of the schools, as can be seen in equation 1.

In this first equation, L is the children's scholastic achievements. AE is the years of schooling. N is the students’ traits and also includes variables which are in not observable, such as innate ability. H represents the family background which impacts scholastic achievements. I represents the investment made by the families in the children's education, where said investments can either be monetary or in terms of time helping the children. Finally, E refers to all the factors related to the quality of the schools and teachers which could generate changes in scholastic results.

The impact of each trait can be interpreted by maintaining the rest of the factors constant. Nevertheless, the form in which some variables affect scholastic achievements is not only direct, as is represented in equation 1; rather, changes in the variable of school and teacher quality (E) could give way to modifying the decisions taken by the households; i.e. increasing or decreasing the investments which families make in education, as well as the years of study which they choose for their children.

As such, the effect of the variation in scholastic conditions could not be captured completely. In order to get the total impact of said changes, one can propose a reduced form of the production function of scholastic abilities. For that purpose, we first express the years of schooling and the investment which families make in education in function of the changes in E and in other variables. This relation is represented by equations 2 and 3.

In equation 2, as well as 3, P represents the prices which impact educational decision in households. Therefore, substituting equations 2 and 3 into 1, we obtain the following reduced formula of the function of scholastic achievements:

If equation 4 takes into consideration the indirect effects generated by changes in variables in the results of scholastic achievements, the impact of public policies still need to be evluated. The policies which the government put into play in order to support students’ educational achievements can influence the school quality variables as well as the prices of education perceived by the households. These are therefore presented in equations 5 and 6 which represent this relationship.

Equations 5 and 6 contemplate the interaction between public policies and community traits where the students reside as well as how they influence the quality of schools and the price of education. Therefore, we can substitute 5 and 6 in 2 and 4 in order to get a reduced form of the equation for scholastic achievements which wholly takes into consideration the causal impact of different factors on the indicator of scholastic achievements.

Equation 8 is of utmost interest when evaluating the effect of a public-policy scholarship program on the perception of students’ school grades in Mexico City. Finally, this equation can be expressed in terms of a regression as:

In equation 9, L is the result of the perception of school grades and controls all the variables in each field of study where: N refers to variables which are individual in nature, HR is the co-variables of the household, C is community traits, PP the public policies and u captures the random error.

5. RESULTS

Table 4 presents the estimations of the impact had by this Scholarship Program on the perception that guardians have regarding the overall grades of their children in the 2014-2015 school year.

There are two types of estimated models: OLS (column 1) and ordered probit (column 2). Both models are estimated based on the production function of scholastic achievements presented by the frame of reference. As such, we control four types of regressors in the models: the student's traits, the family background, the fixed effects of the communities and, obviously, the shock caused by public policy and its scholarship program whose impact we wish to evaluate.

The OLS and ordered probit coefficients show that, controlled by other traits, participating in the scholarship program increases guardians’ perceptions of their children and better grades.

According to the OLS model, being a beneficiary could cause a rise of 24.9% in said perception. If the ordered probate model ends up with better results, due to the dependent variable’s multiple categories, the interpretation of said coefficient is not direct like in the OLS model, though the latter confirms that the relationship between the two variables is positive and statistically significant. For a better interpretation of the results, we acquired the marginal effects of said specification which are presented at the end of this section.

On the other hand, the empirical results confirm the relationship between the prevention function of scholastic achievements established in the theoretical framework, which follows the methodology proposed by ^{Glewwe and Lambert (2010)} and ^{Glewwe and Kremer (2006)} for developing countries. In this regard, aside from the shock caused by public policy allotting scholarships, which is a positive impact on the guardians’ perception of their children’s school grades, the indicator of scholastic results is determined by three other traits: each student’s personal straits, their family background and the specificities of the community in which they live.

To be specific, we found the following variables of students to be relevant: level of schooling, relationship with classmates, behavior in school, and hours spent studying.¹³ The mothers and fathers of students who had a good relationship with their classmates perceived a greater academic performance. This increase was 25.5%, even though the relationship was not statistically significant. The perception of good grades increased in a significant manner when children were well behaved at school (58.2%), and when they spent more time studying (20.0%).

What corresponds to the group of covariables of the guardian’s background, the impact of the level of schooling was as expected. Guardians with a lower level of schooling have a lesser perception of high grades. When they were junior high dropouts or had only finished primary school, their perspective of good grades reduced significantly, by 58.9% and 2.1%, respectively. This could be related to the fact that parents with less education lack the necessary tools to help their children with their homework compared with parents who stayed in school longer (^{Glewwe, 2005}).

The minors who were the household’s main provider’s children reduce the guardian’s perception of high grades (-66.1%). When the mother is responsible for the students' educational concerns, the perception of good grades increased by 29.6%. On the other hand, the fact that the guardian is a woman, or that the guardian in question is currently employed reduces the perception (60.0% and 24.6% respectively). This is according to some studies in developing countries (^{Haisman et al., 2010}; ^{Haisman and Smits, 2012}). The prior could be due to the fact that among the more vulnerable population mothers tend to work to secure additional income for the families rather than as a symbol of empowerment. This situation could imply a lack of time among these women to carry out other activities such as taking care of the minor’s education.

Another trait of these households is the resources which they choose to invest in the education of their children, and how much goes into purchasing other goods. The resources allotted to education can be economic in nature or a question of time spent helping the children. The economic resources are represented in our model by educational, health and household expenses and the availability of internet in the home as a learning tool. On the other hand, the time spent helping the child is measured by the frequency with which the guardians check their children's homework.

Educational expenses did not have a significant effect on the perception of greater school grades due to the fact that this category of expenses averages $200 MXN per month and barely represents 9.4% of the samplings’ total monthly expenses. On the other hand, the effect of health expenses proved to be null and insignificant on the perception of grades. This could also be related with the low amount which these families invest in health. This amount is $118 MXN per month and represents 3.5% of their monthly expenses. In spite of this, household expenses could be relevant in these families as they represent 13.8% of all their spending. As such, said expenses could have a greater impact on reducing the family's income, which then reduces the family’s spending on education and in turn the perception of good grades.

In turn, the availability of internet in households did prove to have a positively statistically significant effect on the guardians’ perception of better school grades for their children. It increases said perception 26.6%. The frequency with which tutors reviewed the students’ homework also increased the perception of better grades by 19.9%, even though this relationship is not significant.

When talking about the effects had by the community, we refer to the time invested taking the children to and from school, as it shows a negative relationship with the guardians’ perceptions of their children having better grades. Finally, we include in the estimations the fixed effects by borough, with the ultimate aim of capturing the spatial differences.

As we have seen, the coefficient in the probit equation does not have a direct interpretation, although it can be employed in calculating the probabilities of obtaining a specific grade and which were acquired by means of marginal effects of the variable representing the shock caused by this public policy. Table 5 presents these marginal changes. The data reveals that students who participate in the program increased 18.18% the probability that their tutors perceive the more favorable grades between 9 and 10. At the same time, this group reduces the probability of perceiving intermediate and lower grades (11.3% and 6.7% less, respectively). Said probabilities are statistically significant for all levels of the dependent variable.

6. CONCLUSIONS

A public policy strategy widely used in developing countries in order to increase its population’s investment in human capital with lesser resources is the implementation of cash transfer programs for education. In 2001 in Mexico City the Scholarship Program was implemented with the goal of incentivizing children from the most vulnerable homes to stay in school and successfully finish their basic education. The program provides students in public primary and junior high schools with a transfer of $800 MXN per month¹⁴ and the possibility of participating in cultural and recreational activities.

This type of program increases the available resources for households, thereby encouraging the children to stay in school. Nevertheless, they could also generate a change in the parents’ perception of their children's scholastic performance, which in turn modifies their expectations. Along these lines, their perception is relevant, as it is linked with the learning process, primarily through the parents’ encouragement. Said motivation correlates to staying in school longer and the students’ grades (^{Gutman and Akerman, 2008}; ^{Chiapa, et al., 2012}).

This paper constitutes a contribution to scientific literature by estimating for the first time the effects of participating in the Scholarship Program in Mexico City on the perception held by parents of their children's grades. This study compares, via a counterfactual approach, the results of both students benefiting and not benefiting from this program. The data employed was acquired from the first evaluation of the program's impact, which allowed us to have information on the perception of the grade average for the 2014-2015 school year, both for the groups of children receiving the scholarship as well as for the group which did not. The method for lifting information from the questionnaires consisted of a random sampling representing all the boroughs of Mexico City.

The estimations are based on a production function of scholastic performance as measured by the grades on students’ report cards. The determinants are four categories: individual, family, location, as well as the shock of the public policy. This last bit is represented in the model by the availability of the cash transfer program. The estimation strategy consisted in linear regression and ordered probit regression models, where the latter is the more appropriate model given the categorical condition of the dependent variable.

The findings indicate that once influenced by the rest of the factors, the scholarship program generates a positive impact upon the perception of guardians in regards to their children's higher grades. An improvement in the perception is important in order for the guardians to feel motivated for the children to stay in school longer, rather than leaving it for other activities. The OLS estimations, in particular, show a coefficient of 24.9% in an improved perception. The marginal effects obtained from the ordered probit regression model indicate that the parents of students receiving the scholarship increase their perception of high grades (between 9 and 10) by 18.1%. At the same time, they reduced their perception of intermediate grades (between 8 and 9) and lower averages (between 7 and 8) by 11.3% and 6.7% respectively.

The analysis of the effectivity of public policy programs, such as scholarships, turns out to be largely relevant in order to understand the effectiveness of the actions taken, in as much in objective indicators, as well as in collateral results and, based on this, whether to strengthen, or if necessary, reorient said strategies. This study represents a baseline for the program and opens the possibility for further follow-up research, allowing for the implementation of different strategies in estimating the effects over time.