Introduction

In modern societies knowledge is central for socio-economic development as well as improvement of the educational systems. To this respect, the Organization for the Economic Co-Operation and Development (OECD) in 2000 implemented the Programme for International Student Assessment (PISA) in order to evaluate the knowledge and skills acquired by 15-year-old students on the basis of their ability to extrapolate from what they have learned, and applied this knowledge to new situations and contexts (^{PISA 2009}), in other words, the adequate level of competency to meet social and employment requirements successfully or exercise the rights, liberties and responsibilities of active citizenship in the knowledge societies of the 21 century. In the 2006 report, it is noteworthy the low levels of proficiency of students of the Iberoamerican countries, specially Latin-America, in practically all the literacies evaluated by PISA: science, mathematics and reading (Iberoamerica in ^{PISA 2010}). For instance, in science the situation is extremely worrying since in Brazil, Colombia, Argentina and Mexico more than a half of students are below level 2, percentage that is far from OECD average 19 %. This level according to PISA is the reference to be prepared for the challenges of a globalized world and knowledge society. PISA offers an international comparative framework analyzing the quality of education and levels of social equity in relation to achievement. PISA also offers the guidelines to improve educational systems in a specific country or a region within it, based on the identification of the main factors that affect the educational processes. The principal factors identified are: 1) the Socio-Economic-Status (SES) linked to the development of the countries and the distribution of wealth, 2) the degree of urbanization connected to the access of information and services, 3) the Class Size, 4) attitudes and the degree of responsibility of students, teachers and families. To exemplified consider Finland and Mexico which spend nearly the same percentage of the Gross Domestic Product (GDP) to education 6.2 and 6.4, however the proficiency levels of each country are totally disparate, with Finland in the top three in practically all literacies evaluated and Mexico, together with other Latin-American countries, on the contrary at the end of the list of participating countries. It is interesting to note that even when Finland has 40 % of its population in rural zones, compared to 24 % of Mexico, achieves better proficiency levels than Mexico. Likewise, the per capita GDP and The Gini Index of Finland are three times and two times better than in Mexico, which means that the wealth and how it is distributed is better in Finland. The other factor that contrast between Mexico and Finland is the Class Size, with Mexico above the OECD mean. As a result of assessments like PISA Mexico has adopted standardized evaluations at elementary education (^{ENLACE 2014}) (ENLACE, spanish acronym) and admission processes at secondary and tertiary education (CENEVAL 2014) (CENEVAL: EXANI-I, EXANI-II and EXANI-III, spanish acronyms). With these initiatives Mexico start to assess its educational processes, that traditionally never happened. However, there is an evident discrepancy between these evaluations and the PISA results for Mexico. This in part due to that national evaluations assess rote learning, rather than deeper understanding and reasoning. In addition, the multiple pitfalls documented so far (^{Lebedeff 2009, 39}; ^{Riveros 2012, 497}), leave in doubt the objectivity and credibility of these evaluations. Even more important is that the educational reforms that have been adopted by the educational system rarely attend real necessities or particularities of the institutions of education in general terms. The 2009 PISA report shows encouraging results for Mexico due to the improvement in science, mathematics and reading literacies (^{PISA 2010}). However, Mexico is below the OECD average and far from the top five countries with better proficiency levels.

In science, particularly in physics at bachelors level the situation described above is not so different. The lack of systematic assessment of the teaching-learning processes bring with itself a number of endemic problems, that are so common or at least is the perception in the community, as: deficient enrollment, low graduation rate, and high backwardness and dropout rates. In fact, in practically all physics faculties nationwide the physics educational reforms obey to official demands and not to real necessities of the dynamical process that underlies in the teaching-learning processes, that at the end should be the guidelines of the change in the curriculum. So, even when we formally adapt the new educational tendencies, like the education focused in the students (competencies) with instructors only facilitating the educational processes actually no change is presented within the everyday teaching-learning activities within classroom. Moreover, we never do a systematic tracking and analysis of the best-known reforms in the physical-educational research community as: Tutorials In Introductory Physics (^{McDermott 2002}), Real-Time Physics (^{Wittmann 2014}), studio physics, SCALE-UP, Workshop Physics (^{Cummings 2009, S38}; ^{Belcher 2003, 8}), Interactive Lecture Demonstration (^{Cummings 2009, S38}), cooperative group solving (^{Cummings 2009, S38}), and Peer Instruction (^{Fagen 2012, 206}). All these reforms document the different ways in which it is possible to achieve gain in conceptual mastery. However, none of these is universal, but it is possible to identified the most relevant factors to implement a successful innovative reform. Between them are: small collaborative work and reduced lecture, the appropriateness of these new educational forms, instructors awareness and understanding as critical to successful innovative practice, full implementation of educational reforms to mention a few (^{Wittmann 2014}; ^{Cummings 2009, S38}; ^{Belcher 2003, 8}; ^{Fagen 2012, 206}; ^{Henderson 2005, 778}; ^{Rogers 2003}).

In the present paper we show the results of the first assessment of the physics reform that undertook the Academic Unit of Physics (UAF, spanish acronym) of the Autonomous University of Zacatecas (UAZ, spanish acronym) Mexico in 2002. This reform consists in the implementation of one-year bridging courses in physics and mathematics: General Physics I and II, Laboratory of General Physics I and II, Arithmetics, Algebra, Geometry, Trigonometry, Analytic Geometry, Differential and Integral Calculus. Together with the mentioned reform it was implemented an admission process in order to identify the students that truly need the one-year basic courses and those that have the necessary ground in physics and mathematics. Since its creation in 1987 the UAF offers a four-year program in physics that in 2002 suffered one of its major modifications not only for the incorporation of the one-year basic courses but also for other important changes in the curriculum that we will detail later. So, after 2002 the UAF has two programs: the five-year program and the four-year program which differ in the inclusion of the one-year bridging courses. We will refer to these two programs as the five-year propaedeutic program and the four-year traditional program as well as the program from 1987 to 2002 as the Traditional-1987 and from 2002 (this program combines both the five-year propaedeutic and the four-year traditional program) to present as Reform-2002. The present assessment shows that the student performance is practically the same for the five-year propaedeutic program and the four-year traditional program with an average score of 84 %. However, important indicators as the dropout, efficiency and graduation rates show that something is happening in the academic dynamic of the five-year propaedeutic program since the efficiency and graduation rates are 17 % and 31 % worsen in comparison with the four-year traditional program, meanwhile the dropout rate is 15 % better. Comparing the Traditional-1987 and the Reform-2002 programs rates like enrollment and dropout are improved by more than 100 % and 5 % after the 2002 reform, whereas the efficiency and graduation rates have fallen by 6 % and 15 %. In the case of the 2010 admission (selection) process we have analyzed the pre and post-course scores, the learning gains, the correlation between the post course scores with homework performance as well as the relation with the official admission tests, EXANI-II-CENEVAL, that the UAZ has implemented in the last years. All courses (Arithmetics, Algebra, Trigonometry) that form part of the selection process show a moderate shift to higher scores comparing the pre and the post course scores. This is reflected in the learning gains which shows a maximum of 0.7. However, after two weeks of courses 40 % of students score sixty percent, meanwhile 40 % and 20 % of them show scores above and below the mentioned 60 %. The correlation between the post-course score and the homework performance is 0.7, indicating that students that do well in homework get better post-course scores. Last but not least, the correlation between the post-course scores and the EXANI-II-CENEVAL scores shows a important data dispersion and only the five top scores of both tests coincides. Additionally, taking into account the students that were selected for the four-year traditional program we easily find that the post-course score that determines what students go to the five-year propaedeutic program or to the four-year program is 70 %.

Results and discussion

The Academic Unit of Physics of the Autonomous University of Zacatecas started its activities in 1987 offering a four-year term bachelors physics program that we will called four-year traditional program from now on. At that time, the academic and infrastructure conditions were not the best, resulting in a reduced number of students and an academic staff with modest skillfulness. Fifteen years later, in 2002 to be specific, the Academic Unit of Physics implemented a reform of the bachelors physics program in order to relief some endemic problems as: the high dropout rate, the low efficiency of the program, the low graduation rate and the high backwardness rate. Unfortunately, there is not clear evidence of a systematic tracking of the mentioned rates and a serious evaluation of the teaching-learning process that indicates the need of a reform. The reform consists in the inclusion of one-year preparation courses as arithmetic, trigonometry, basic algebra, differential and integral calculus, general physics and laboratory of general physics at high school level. The candidates for the bachelors physics program, since 2002, must take a two-week or three-week courses in elementary mathematics and physics (selection process) in order to discern or select the students for the traditional and propaedeutic programs. Taking into account that both parts of the mentioned reform, the inclusion of one-year preparation courses and the two-week selection process, are important such that we present the assessment of them separately.

Propaedeutic reform

As already mentioned in the introduction it is well known that Mexico presents a low students performance in mathematics at elementary levels (^{PISA 2009}, Iberoamerica in ^{PISA 2010}) and it is not an absurd supposition to consider that this characteristic is also present at high school. Unfortunately, in Mexico as in many other countries there is not a systematic assess- ment of the educational processes, that be the basis for future educational reforms. To this respect, the academic reform analyzed and discussed here is not the exception and it is based on a perception, that we consider as the central hypothesis, that the high dropout, the low graduation and the high backwardness rates are caused by the lack of the basic concepts of mathematics and physics at high school level. So, with an appropriate selection of students (see next subsection) that clearly differentiate between candidates with well established mathematic-physics grounds and those without it, and then providing the mentioned grounds to the latter it is reasonable that academic performance of these two groups would be the same or nearly the same. Recently, Marina Milner-Bolotin and colleagues (^{Milner-Bolotin 2011, 020107}) report that students with substantially less educational backgrounds might be served in a separate classroom or by offering them a bridging course before attempting the introductory undergraduate physics course. This of some way supports our hypothesis despite the university and country-context differences between our findings and the Milner-Bolotin ones (^{Milner-Bolotin 2011, 020107}). Likewise, Eric Brewe and colleagues at the Florida International University point out that certainly educational reforms play a major role to establish supportive learning environments for under-represented students in physics by improving conceptual development, however it is important to take care of equity issues, as historically under-represented students arrive at university physics with different conceptual understanding (^{Brewe 2010, 010106}).

In order to compare the academic performance of those students subjected to the one-year preparation courses and those that go directly to the traditional program we have collected the data of four generations arranging and showing it by semester, Fig. 1a.

Figure 1 (a) Generation score per semester of the five-year propaedeutic program (dark color) and the four-year traditional program (light color) for: first (blue), second (red), third (green) and fourth (black) generation, respectively. (b) Students score per generation of the five-year propaedeutic program (dark color) and the four-year traditional program (light color) for: physics (blue), experimental (red), and mathematical (green) courses, respectively. 

The dark and light colors correspond to the scores of those students subjected to the one-year preparation courses and those that go directly to the traditional program, respectively. In an ascending order, the blue, red, green and black bars represent the first, second, third and fourth generation. It is evident that in most cases the students subjected to the one-year preparation courses show a better academic performance, except for the fourth generation for which in most semesters the students of the four-year traditional program have higher scores. On average, the students of the propaedeutic program perform 15, 10 and 10 % better than the students of the traditional program for the first, second and third generation, respectively. For the fourth generation the students of the traditional program perform 15 % better than the students of the propaedeutic program, see Table I.

Table I Students score average (%) per generation of the propaedeutic and traditional programs. 

To know how the student scores are distributed among the different courses we have distributed them according three categories, that are physics (blue), experimental (red) and mathematical (green), see Fig. 1b. In the case of the experimental and mathematical courses the students scores of most generations are better in the propaedeutic program (dark colors) than in the traditional one (light colors) with an opposite trend for the physic courses. Taking the average of the student scores over the different generations for the mentioned categories the students performance is 15 and 10 % higher in the propaedeutic program with respect to the traditional one for the experimental and mathematical categories and a 15 % in the physics courses but with an opposite trend, Table II. Additionally, the experimental courses present the higher scores irrespective of the generation.

Table II Students score average (%) per course category of the propaedeutic and traditional program. 

To end this subsection we compare the data of the enrollment, dropout, efficiency, and graduation rates of students prior to the propaedeutic reform (Traditional-1987) and after it (Reform-2002), see Table III.

Table III Comparison of the dropout, efficiency and graduation rates prior (Traditional, 1987) and after (Reform, 2002) the 2002 reform as well as the average of enroll students per generation for the same comparison. These rates have been averaged over 16 and 4 generations with the exception of enrollment that is averaged over 16 and 9 generations, prior and after the reform, respectively. 

In the case of the dropout, efficiency and graduation rates we have taken 16 and 4 generations of the Traditional-1987 and the Reform-2002 for the analysis. Meanwhile the enrollment rate is averaged over 16 and 9 generations of the Traditional-1987 and the Reform-2002, respectively. It is clearly seen that the enrollment rate has increased more than 100 % and the dropout rate diminished slightly, 4 %, since 2002. On the other hand, the efficiency and graduation rates are getting worse with a reduction of 6 % and 15 %. These results are quite interesting since according to our central hypothesis the reform should improve the performance as well as the enrollment rate, even more it reduces the dropout rate, however the efficiency and graduation rates are not better after the reform. This indicates that something is happening in the process, so we will analyze later (academic context and subtle coincidences subsection) the possible factors that influence the mentioned results. In Table IV, the dropout, efficiency and graduation rates of those students that after 2002 are selected to the five-year propaedeutic program and the four year traditional one are shown. One of the consequences that brings the propaedeutic program is a reduction of the dropout rate, rate that is always desirable to be as low as possible. However, the propaedeutic program has very reduced efficiency (4 %) and graduation (13 %) rates. On the contrary the four-year traditional program shows averages of 21 % and 44 % for the same rates, moreover these values are better than the corresponding ones to the Traditional-1987.

Table IV Dropout, efficiency and graduation rates after the 2002 reform. The five-year propaedeutic and the four year traditional programs are compared. 

Selection process

Following our central hypothesis, a good selection process is essential to identify those students that can go directly to the traditional program and those that need one-year of preparation courses. According to the results of the last subsection seems that the selection process since 2002 has been performed in a good way, but it has never been evaluated. So, in order to evaluate the mentioned processes we have collected the data of the 2010 two-week selection process and analyzed it accordingly, Figs. 2 and 3.

Figure 2 Pre (Black) and Post (Blue) students performance in the two-weeks propaedeutic 2010 courses of (a) Arithmetic, (b) Trigonometry, (c) Algebra and (d) the average of a, b and c. 

Figure 3 (a) Pre students score versus students gain. (b) Students course score versus students homework score. 

We have examined the students performance in Arithmetics, Trigonometry and Algebra prior to the two-week courses (Pre) and after it (Post), Fig. 2 (a), (b) and (c). In the case of Arithmetics the pre student scores are distributed almost equally between 40 to 100 %, with the number of students oscillating between 10 to 20 % of the total and the highest number of students (20 %) presenting at 80 % pre student score, black dots of Fig. 2a. The post student scores clearly shift to high scores with a maximum of 40 % of the number of students at a score of 80 %, blue dots Fig. 2a. In the case of Trigonometry comparing the pre and post student scores there is a shift to high scores, however a maximum is presented at a score of 50 % representing more that 30 % of the number of students, Fig. 2b. For Algebra the trend is totally opposite with respect to Arithmetics since most students show low pre scores, 0 to 30 %, and only a reduced number of them scores between 80 to 100 %, black dots Fig. 2c. After the two-week Algebra course the student scores clearly shift to high scores with a maximum of the number of students at a score of 40 and slight modifications in the high score range, blue dots Fig. 2. In Fig. 2d, we show the average of student scores taking over the three mentioned courses finding a shift to higher scores comparing the pre and post scores. There is a maximum in the number of students at a post score of 60 representing a shift of 20 % with respect to the pre student score maximum. Interesting to note is that in the high score range (90 and 100) the number of students is practically the same before and after the two-week courses, meanwhile in the score range of 70 to 80 there is a moderate increase in the number of students post two-week courses representing 35 % of the number of students. These scores are the reference that determines what students go to the propaedeutic or traditional program. In Fig. 2d we include two vertical lines, red and green, that fulfill these criteria. If the green vertical line is the reference, then the students that go to the traditional program are 20 %, whereas a 40 % is found if the red vertical line is chosen.

To know more about the students performance we have calculated the normalized learning gains as a function of the pre course scores and the correlation coefficient of post course scores to homework scores following (^{Finkelstein 2005, 010101}). In the case of learning gains it is found that the pre high scores have practically null gain and that the increase at reference scores comes from low gains with moderate pre-course scores and moderate or high gains with low pre-course scores, black dots inside the red circle in Fig. 3a. In the case of overall performance, post-course scores, to homework scores we find that the correlation coefficient is r = 0.65 indicating that students who performed well on homework did well in the courses as well as that even when the courses are short (two weeks) they are well integrated, Fig. 3b. Similar values for the correlation coefficient have been found in tutorial studies (^{Finkelstein 2005, 010101}).

To end this subsection we analyze the correlation of the scores of the 2010 official test EXANI-II-CENEVAL with respect to the post-course scores, Fig. 4. As it is clearly seen there is a perfect correlation between the top five scores, meanwhile the rest of scores show a considerable dispersion pointing out that if the official test is taken as the selection process only a reduced number of students could be selected for the traditional program. Besides, it is important to mention that the students selected to the traditional program correspond to those that are at or above the red vertical line of Fig. 2d indicating that the reference score of the 2010 selection process is 70 %.

Figure 4 Correlation of the scores of the 2010 official test EXANI-II-CENEVAL to the post-course scores of the 2010 selection process. 

The academic context and subtle coincidences of the propaedeutic reform

There are two main factors associated with the learning processes: 1) cognitive and 2) environmental structures. The former is well known that depends on individuals (^{di Sessa 1993, 105}; ^{Ruumethart 1980}), while the latter (^{Cole 2001}) is no less significant as was pointed out by some authors (^{Finkelstein 2005, 010101}). The environmental structures constitute the educational experience that is affected by the local context, norms and tools that serve to mediate student actions and therefore participate in the intellectual processes. We can see these structures as nested frames that are not independent each other, 40 on the contrary interact between them at different levels (^{Finkelstein 2005, 010101}). As we want to correlate our results with this constructivist perspective it is important to present the academic context of the Academic Unit of Physics of the Autonomous University of Zacatecas.

As we can see from Table V the program of physics has suffered four major changes since 1987. From 1987 to 2001 the number of courses in physics and mathematics remained practically unchanged, the most important difference in this period was the inclusion of five Lab courses in 1999, which largely obey the change in the infrastructure conditions of the Academic Unit of Physics. This Table also shows the main difference between the five-year propaedeutic program (2002-1) and the four-year traditional one (2002-2), being the number of mathematic courses the major difference between both programs. Another interesting feature since 2002 is the inclusion of seven optative courses in the program. These courses have the objective of approaching the students to current research activities of the academic staff as well as prepare them to their future bachelor’s dissertation. Likewise, the academic burden of students increased about 15 % with respect to the former 1999 program.

Table V Curricular evolution from 1987 (Starting Program) to the last reform (2002) according to the number of courses in the areas of: Physics, Mathematics, Complementary, Labs and Optative, respectively. 

In Fig. 5, we show the academic staff evolution per semester, where the black, blue, red, green and magenta curves represent the total of the staff, staff with Dr. degree, staff with Master degree, staff with Bachelor degree and others, respectively. As we can see, the academic staff with Dr. degree determines the staff evolution with a general increase since 1987. In 2010 there are 30 members and most of them have the highest academic habilitation. In the case of student’s enrollment the first ten years of the program was low, giving, as a result, a ratio of students to staff members of nearly 2, Fig. 6.

Figure 5 Academic Staff evolution per semester from 1987 to 2010. The black curve corresponds to the total staff members, meanwhile the blue, red, green and magenta represent the members according to the staff degree: Ph. D., Master, Bachelor and other, respectively. 

Figure 6 (Black) Enrollment evolution and (Blue) student to staff members ratio per year from 1987 to 2010. 

After that, the enrollment has increased more than 100 % reaching its maximum value after 19 years of the program (in 2006), moreover this increase coincides starkly with the reform of the program in 2002 changing the overall students to staff members ratio to 5 in 2010. Another important characteristic that we have to mention is the number of class groups that have been opened for the newcomers being 1 from 1987 to 2001 and 2 from 2002 to 2010, which turned out in an average class size not so far from the recommended for science (^{Finkelstein 2005, 010101}). Even more, if we taking into account the dropout and backwardness rates, turns out that the class size is closer to ideal.

In the case of graduation it is important to mention that the UAF-UAZ has had two basic forms to get the Bachelor’s degree: 1) Bachelor’s dissertation and 2) credits of master’s programs. The former represent roughly the 89 % of the graduation rate, meanwhile the latter obeyed a nationwide trend of the 90’s which in the case of UAF-UAZ remained until 2003. In Fig. 7 (a) we show the ratio of the graduated students to the UAF staff members per year from 1990 to 2010. As we can see the mentioned ratio present an irregular pattern with sharply changes in practically every year of the program. The maximum ratio achieved has been roughly 0.7 in 2001 and in most years between 1996 and 2010 the ratio oscillates between 0.2 and 0.5. These numbers are worrying if we consider that the enrollment of students shows a larger slope than staff members over the last years. In an ideal situation in which the dropout rate be zero and the efficiency rate be close to 100 % the graduated students to staff members ratio will be or at least should be 1.5-2.0. Even more, if we consider that historically the UAF has had a dropout rate of roughly 55 % the graduated students to staff members ratio have to be between 0.5 and 1.0 nor the 0.2-0.5 that we see in Fig. 7 (a). So, after the 2002-Reform the net effect over the graduation rate has been a fall down of roughly 70 % passing from 38 % to 23 %, see Table III. In order to explain what is happening in the graduation process or at least find a reasonable cause, we have analyzed the distribution of thesis guided in the history of the Academic Unit of Physics, Figs. 7 (b) and (c). Considering that 50 % of the members of the current academic staff have 15-20 years in the program and that the other 50 % has less than 10 years it is not understandable that only five advisors have more than six thesis guided, Fig. 7 (b). Interesting to note it is also the large number external advisors that guide mainly one thesis as well as it is also important to mention that most of these thesis come in co-advisory with some member of the academic staff of the UAF which turns out in collaborations of the UAF academic staff and researchers in other institutions in the national and international context, Fig. 7 (c).

Figure 7 (a) Ratio of the graduated students to the staff members per program’s year since 1990 to 2010. (b) Number of staff members versus the number of thesis guided, the gray and blue historgrams correspond to staff members of the UAF-UAZ and external to it, respectively. (c) Distribution of co-advisors internal (Gray) and external (Blue) to UAF-UAZ. 

According to the academic context described above we consider that:

Finally, it is important to mention that the assessment presented here is the first step to analyze the educational processes in Mexican science faculties like the Academic Unit of Physics of the Autonomous University of Zacatecas. Further studies about gender, Socio-Economic-Status and degree of urbanization are needed in order to obtain a whole picture of the educational process, and more important to implement educational reforms that attend real needs of specific faculties immersed in the local, national and international context. To this respect, science-faculty administrators and science-educational authorities have to be aware of the difference between students with different educational backgrounds and gender particularities in order to tailor their instruction accordingly such that the educational reforms be sustained (^{Milner-Bolotin 2011, 020107}; ^{Pollock 2007, 010107}; ^{Kost-Smith 2010, 020112}; ^{Pollcok 2008, 010110}; ^{Goertzen 2011, 020105}).

Conclusions

In summary, we have assessed a educational reform which consists of an inclusion of bridging courses of physics and mathematics in an undergraduate physics program. Particularly, we have compared four generations of the five-year propaedeutic program (with bridging courses) with the four-year traditional program (without bridging courses) of the Physics Faculty of the Autonomous University of Zacatecas Mexico. It is found that in most semesters the students performance is practically the same in both programs, even in some of them the five-year program presents better results, supporting the need of a propaedeutic year in order to overcome high school deficiencies. This reform additionally improves the enrollment and dropout rates, contrary the graduation and efficiency rates are worsen. We also evaluate the 2010 two-week selection process of the physic candidates finding that only the top scores of the official test (EXANI-II-CENEVAL) match perfectly with the candidates selected for the four-year traditional program. All this indicates that even when the five-year propaedeutic program and the two-week candidate courses are not implemented with prior assessment of the teaching-learning processes they partially fulfill the primary objectives of provide the necessary skills for scientific studies and to select the candidates for the five-year program and the four-year one, respectively. Additionally, we analyze the academic context of the mentioned processes finding that successful academic reforms have to be accompanied of administrative decisions that take into account the differences between students with different backgrounds and gender particularities in order to tailor their instruction accordingly.