Artículos

 

Teacher knowledge and teaching in Panama and Costa Rica: A comparative study in Primary and Secondary education

 

M. Alejandra Sorto*, Jeffery H. Marshall**, Thomas F. Luschei*** and Martin Carnoy****

 

* Texas State University – San Marcos, USA; sorto@txstate.edu

** Sapere Development Solutions, USA; jmarshall@sapere.org

*** Florida State University, USA; tluschei@lsi.fsu.edu

**** Stanford University School of Education, USA; carnoy@stanford.edu

 

Recepción: Septiembre 5, 2008
Aceptación: Abril 30, 2009

 

RESUMEN

Este estudio tiene como finalidad examinar la relación entre el conocimiento didáctico de las matemáticas, la preparación y las prácticas de los docentes de tercero y séptimo grado en Panamá y Costa Rica. El conocimiento matemático y su didáctica son medidos con un instrumento que incluye matemáticas a tres niveles: uno básico, otro más avanzado y uno más de aplicación en situaciones de enseñanza–aprendizaje; las prácticas docentes son grabadas por medio de video y su calidad se evalúa a través de una rúbrica común. En Panamá y Costa Rica las prácticas docentes y las respuestas a preguntas diseñadas para medir el conocimiento especializado muestran deficiencias en este dominio crítico. Asimismo, evaluamos en qué medida cada una de estas formas de conocimiento puede ser diagnosticada con base en las características de los docentes, y hallamos algunos resultados significativos.

PALABRAS CLAVE: Conocimiento matemático del profesor, conocimiento pedagógico del contenido, métodos multivariados, observaciones de aula, prácticas docentes en matemáticas.

 

ABSTRACT

This study examines the relationship between mathematical knowledge for teaching, the preparation of and classroom practices of third and seventh grade teachers in Panama and Costa Rica. Mathematical content and pedagogical knowledge are measured with an instrument that includes three levels: basic content, advanced content and content situated in the teaching learning context. The quality of teacher practice was evaluated via a rubric applied to videotaped lessons. In Panama and Costa Rica, responses to questions designed to measure specialized knowledge for teaching and evaluation of the teacher practice revealed deficiencies in this critical domain. In addition, some significant relationships were found between each of these aspects of teacher knowledge and characteristics of the teachers

KEY WORDS: Teacher mathematical knowledge, pedagogical content knowledge, multivariate methods, classroom observations, mathematical teaching practices.

 

RESUMO

Este estudo tem como finalidade analisar a relação entre o conhecimento didáctico da Matemática, a preparação e as práticas dos docentes dos terceiro e sétimo anos no Panamá e na Costa Rica. O conhecimento matemático e a sua didáctica sao avahados com um instrumento que incluí Matemática a três níveis: um básico, outro mais avançado e um mais relacionado com a aplicação em situações de ensino–aprendizagem; as práticas docentes sao gravadas em vídeo e a sua qualidade é avahada através de uma rúbrica comum. No Panamá e na Costa Rica, as práticas docentes e as respostas ás questões construidas para avahar o conhecimento especializado mostram deficiencias neste dominio crítico. Desta forma, avaliamos em que medida cada urna destas formas de conhecimento pode ser diagnosticada com base nas características dos docentes e salientamos alguns resultados significativos.

PALABRAS CHAVE: Conhecimento matemático, conhecimento pedagógico do conteúdo, métodos multivariados, observações de aula, práticas docentes em Matemática.

 

RÉSUMÉ

L'examen des relations entre, d'une part, la connaissance didactique des mathématiques et, d'autre part, la preparation et les pratiques des enseignants pour des écoliers ágés de 8 ans et de 12 ans au Panama et au Costa Rica constitue le sujet de cette étude. Les connaissances en mathématiques et la didactique en rapport sont évaluées gráce á un instrument de mesure qui inclut des mathématiques de trois niveaux : mathématiques de base, mathématiques á niveau plus avancé et mathématiques plus centrées sur les exercices d'application dans le cadre de situations mélant enseignement et apprentissage. Les pratiques enseignantes sont enregistrées sur bande video et la qualité est évaluée en commun. Au Panama et au Costa Rica, les pratiques enseignantes et les réponses apportées á des questions concues pour évaluer les connaissances spécialisées mettent en relief certaines déficiences dans ce domaine critique. Cette étude se pose aussi la question suivante : dans quelle mesure chacune de ees formes de connaissance peut étre évaluée á partir des caractéristiques des enseignants? On peut affirmer que certains des résultats auxquels nous parvenons sont significatifs.

MOTS CLÉS: Connaissance mathématique du professeur, connaissance pédagogique des contenus, méthodes multivariées, observation des classes, pratiques enseignantes pour les mathématiques.

 

1. INTRODUCTION

Almost all Central American countries have recently started the process of developing and implementing new educational standards, which include, for most of the countries, new content and higher expectations than before. As a consequence, governments in some of these countries are starting to make important decisions with respect to the preparation of teachers. These decisions are usually made based on non–empirical evidence due to the lack of resources to carry out educational studies; and to the fact that developing countries are often left out of international comparison studies like TIMSS (Trends in International Mathematics and Science Study)¹ and PTEDS (Preparatory Teacher Education Study). The study detailed here comes from a larger project that was initiated through a combination of private and public sector interest in education in Panama and Costa Rica (Carnoy et al. 2007a). The study's relevance cuts across several lines of research and policy. One, it helps inform policymakers in these countries about the quality of teachers with respect to their knowledge and teaching practice. Also, by exploiting variation in the two contexts it is possible to draw larger lessons about the role of policy in determining school quality. For example, teachers in Costa Rica are required to have bachelor's degrees, which makes it possible to compare teaching performance across systems with different levels of commitment to teacher quality. Finally, the results add to a literature on teacher quality that has little representation in the Central American region.

Both Panama and Costa Rica have successfully increased access to schooling for the mass majority of their school–age populations. Between 1991 and 2004, Panama expanded net secondary enrollment from 54% to 64% (World Bank World Development Indicators). About 40% of the tertiary age cohort attends post–secondary education. Of the labor force in Latin America, Costa Rica has one of the highest average levels of education—the average adult Costa Rican has 7.3 years of education. This expansion in the two countries will surely continue into the future. However, there is room for improvement in both educational systems in terms of the amount of learning that students are able to achieve in each year that they attend school.

The purpose of this study is to document qualitative differences in the Costa Rican and Panamanian education systems. We make this case empirically using data collected in primary and middle ("pre–media" in Panama, "III Ciclo" in Costa Rica) schools in both countries on the nature and quality of mathematics teaching. The emphasis is on two elements of this teaching. First, using extensive teacher questionnaires we are able to measure different forms of the teacher's mathematics knowledge, including basic and more advanced content knowledge (up to seventh grade mathematics) as well as specialized "pedagogical content knowledge" (Shulman, 1986). Then, based on classroom observations, we are able to break down math lessons in each country in order to compare instructional strategies and the quality of lesson development.

We hypothesize that Costa Rica's historical commitment to education results in a more qualified teaching corps and more effective classroom teaching, on average, than in Panama. The data we collected provide a unique opportunity to make teacher quality comparisons, as well as address questions that have received little attention previously in education research. The comparative framework also facilitates "working backwards" to explain how different teacher training and preparation regimes affect observable elements of quality. This emphasis on the role of public policy is of particular interest to Panamanian education authorities interested in upgrading the country's human capital base. But as evidenced by previous comparative studies in mathematics (e.g. Ma, 1999), the lessons that we learn from these comparisons are likely to resonate beyond the Central American context.

 

2. CONCEPTUAL AND ANALYTICAL FRAMEWORK

2.1 Conceptual Framework

Figure 1 provides a simple conceptual overview of different forms of teacher knowledge proposed by the authors. The focus here is on mathematics, although figure 1 is general enough to be applicable to all subjects. This conceptual framework is derived from existing theories about teacher preparation, teacher knowledge, and teaching practices (Shulman, 1986; Stein et al., 2000; Ball & Bass, 2000). The middle part consists of three circles intersecting each other depicting the knowledge teachers draw upon when teaching. The left hand circle represents general pedagogical knowledge. General pedagogical knowledge refers to knowledge of "how teachers manage their classrooms, organize activities, allocate time and turns, structure assignments, ascribe praise and blame, formulate the levels of their questions, plan lessons, and judge general students understanding." (Shulman, 1986, p. 8). Teachers accumulate these pedagogical skills in pre–service and in–service pedagogical training courses, through "experiential learning" that comes from trial and error in their own classroom, and through "mentor effects" that result from watching other teachers or working closely with other school personnel (teachers, directors, etc).

On the right hand side is the content knowledge circle divided into lower and higher elements. Lower refers to the level that is being taught (i.e. third grade), while higher is for more advanced grades or levels beyond the grade the teacher is responsible for. This is consistent with how higher level knowledge is defined in mathematics education circles in the United States, where "one level up" is a common reference for more advanced knowledge². Teachers obtain math knowledge primarily in formal pre–service training mathematics classes. There are some additional opportunities for learning mathematics content, such as on–going formal study where the teacher is exposed to higher levels of mathematics.

At the intersection of pedagogical and content knowledge lies a specialized form of knowledge prized especially by education researchers. This domain is commonly referred to as pedagogical content knowledge (PCK) (Shulman, 1986), and its evolution in mathematics reflects a growing emphasis on practiced–based metrics for analyzing teaching effectiveness in the classroom. Pedagogical content knowledge turns up in myriad ways in the classroom. It refers to the application of mathematical knowledge for teaching others, especially young children. Examples include the powerful explanations that teachers use to develop deep understanding of concepts that are part of the curriculum, the ways in which they draw linkages with other elements of mathematics, and the questions they pose to students. These kinds of skills, it is argued, can only be accumulated through practice or very specialized training activities (Hill & Ball, 2004; Ball, Hill & Bass, 2005)³. To illustrate the significance and distinction between what it means to be knowledgeable and proficient in mathematics, and to know mathematics in ways that enables teaching practice, consider the following example offered by Ball and Bass (2000).

Suppose you posed four numbers —7, 38, 63, and 90— to a class and asked the students to identify which of the numbers were even. And suppose, further, that you got [a] paper back from one of the students with none of the numbers circle. What would you make of this? Is this answer surprising or predictable? What might this student actually know? What number or numbers would you pose next to find out with more precision what the student thinks? Why would that selection be useful? (p. 1).

As Ball and Bass note any teacher will correctly identify which numbers in the above list are even, but understanding what there is to know about even numbers goes beyond being able to do this oneself. Knowing mathematics for teaching demands knowing and enacting mathematical and pedagogical reasoning practices that are deeply intertwined, as teachers consider the kinds of questions Ball and Bass raise about what and why students might do and think, how to follow up and get more information from particular students, or what might be a good next task.

In total there are four distinct knowledge areas depicted in figure 1: lower and higher content knowledge, pedagogical knowledge, and pedagogical content knowledge. Which is most important? By definition the PCK element is made up of critical strands of knowledge that most directly influence the teacher's ability to develop curriculum. This form of knowledge is also likely to have the closest link with the features of effective teaching, listed in the lower half of figure 1. This does not mean that PCK is the only teacher knowledge area that matters, or that the other knowledge elements affect teaching quality solely through their impact on PCK. A simple example is the teacher's ability to control the class, which is an element of general pedagogical skills rather than specific PCK. The role of higher–level knowledge is especially difficult to assess, and is the source of some disagreement in the field. Among academic mathematicians especially there is a strong belief that even primary school teachers should be comfortable with higher level (meaning university) mathematics content. This assumes to some extent that pedagogical skills are derived from studying higher level math—a position that is strongly contested by many mathematics education researchers. But the emphasis on content knowledge also recognizes the importance in mathematics of making linkages with advanced levels, and the potential consequences when teachers make mistakes that result from deficient content knowledge. As these kinds of errors accumulate the overall integrity of mathematics instruction is affected, which can in turn cause student frustration and the perception that mathematics is "too hard" or "poorly taught."

 

2.2. Analytical Framework

Data Collection and Sample

After a brief pilot data collection in several schools in the Panama City area, 3 6 schools and two teacher training institutes were visited during a two week period towards the end of the school year. The sample was drawn randomly from Ministry of Education data files, and included both primary and middle ("pre–media") schools from three provinces: Panama, Colón and Chiriqui. The provinces were intentionally chosen to account for different realities within the country. Panama and Colon represent the center of the Panama Canal and urban development zones in the country, although both provinces include rural schools. Chiriqui is located in the western part of the country and includes a high proportion of rural, isolated schools. Together, the three provinces include about 70 percent of Panama's student population. In some cases the primary and pre–media schools share facilities, so the actual number of school sites visited was closer to 25. The total number of teachers who completed questionnaires was 176.

We visited Costa Rican schools during the first two months of their school year using similar personnel from education supervisor offices as enumerators. A total of 56 primary and lower secondary schools were visited, as well as three teacher training faculties. The geographical coverage was similar to that in Panama, and focused on six provinces in the most populous center of the country. The enumerators were drawn from supervisors working in four districts in San Jose.

The schools were selected randomly from Ministry of Education files and divided between urban and rural locales. This resulted in a total of 209 teachers that filled out questionnaires.

In both countries questionnaires measuring teacher knowledge and background were applied in all grades in both the primary and pre–media schools. The emphasis, however, was on grades three and seven. Grade 3 was chosen because of the existence of comparable data from other Latin American countries at this grade level (Carnoy, et al. 2007). Seventh grade is a useful point to begin comparisons between primary and lower secondary levels. Finally, the pre–service teachers who were given the questionnaire were chosen based on students in attendance at teacher education programs on the day of visits, and there was little distinction by year of study, program, etc.

Table I summarizes the samples of teachers and teachers–in–training that completed the questionnaires. Overall, the sample breakdown by country is not very different between Panama and Costa Rica. Both samples are weighted towards primary school teachers, urban areas and public schools. For Costa Rica there is a slightly higher percentage of rural schools compared with Panama, although there are also more private schools in the sample (a number of these are private schools where teachers are paid by the Costa Rican government).

In sum, the samples appear to be sufficient for the general comparative framework we implement in this study. There is no guarantee that either sample is representative of the country, so the comparisons must be treated with some care. But the fact remains that the teachers were drawn randomly from similar cross–sections of schools covering similar geographic areas of each country; this strengthens the underlying validity of the comparisons made.

 

Variable Measurement and Methodology

Different types of variable measurements and methodologies were utilized for the three main components of our conceptual framework (figure 1). For teacher training, we conducted structured interviews with University and Normal School instructors, and with Ministry of Education officials. For teacher knowledge, we designed paper–and–pencil questionnaires aligned with national curricula guides. For teaching practices, we videotaped lessons of a subgroup of teachers that completed the questionnaire.

With respect to teacher knowledge, each type of teacher (primary, pre–media (lower secondary) and teacher–in–training) completed a slightly different version of a questionnaire⁴. For content knowledge all three versions include both primary level and middle/secondary level mathematics content questions. For third grade level content the items were in the form of multiple choice or fill in the blank, and covered the areas of basic operations, multiplication–division, fractions, geometry, measurement and statistics. The primary teacher questionnaire consisted of 19 items, while the middle school teacher questionnaire consisted of only 15 items, and teachers–in–training 11. Higher–level mathematics content knowledge was measured by a series of multiple–choice items drawn from the seventh grade curriculum. Primary and pre–media teachers questionnaires consisted of 10 items of this kind, while the teachers in training questionnaire consisted of 8 items⁵.

One of the most original features of our data is the information on the teacher's specialized knowledge of mathematics instruction, also known as pedagogical content knowledge (PCK). What makes the PCK construct different compared with content or pedagogy is that every question is about a "big" idea or concept embedded in a teaching situation. The purpose of this is to measure if the teacher can apply the content knowledge to the job of teaching. Because this is rooted in a deep understanding of the content, some think of it simply as conceptual understanding of the mathematics. Before this construct was considered in mathematics education research, it was believed that teachers simply learned mathematics and pedagogy separately before learning how to teach others by integrating these knowledge forms in the classroom.

The items for this study are drawn from two sources. One group was created and applied as part of similar studies in Guatemala and Honduras (UMCE, 2004; Marshall and Sorto, 2007), which is convenient for the Panama and Costa Rica contexts since they require little alteration and the content is aligned with both curricula. The second source comes from Deborah Ball and Heather Hill (Learning Mathematics for Teaching Project) at the University of Michigan, who have created closed items (multiple choice) for measuring a construct that includes pedagogical content knowledge called mathematical knowledge for teaching (Hill, Ball & Schilling, 2008). The more open–ended activities usually present teachers with a specific situation and ask them to comment, or to create a problem by themselves. The closed items also are drawn from real world teaching situations, but are simplified somewhat in order to fit in the multiple–choice format (see an example of an item that measures PCK below written in English here but given in Spanish to the teachers).

A series of classical and modern test theory techniques are incorporated to analyze the teacher answers on the various items. This was done for both the content and pedagogical knowledge questions. Based on these initial results a handful of problematic questions were removed from the analysis, mainly because large numbers of teachers did not answer them, or the items were not very correlated with the rest of the items. For the open ended questions where teachers could not simply choose an answer from a list of options, teacher's answers were graded using a rubric and then assessed based on their correlation with the rest of the exam items. This is a basic form of reliability test, although there were comparatively few items of this type on the various forms. Also, Item Response Theory (IRT) technique was incorporated to create a comparable measure of ability for each teacher, regardless of which form they completed. This kind of analysis is made possible by the existence of anchor items, which are questions that appear on all four teacher questionnaire forms. IRT uses the teacher responses on the anchor items to evaluate the difficulty of all of the items and then uses this information to create a single, comparable estimate of the teacher's overall knowledge.

Finally, multivariate techniques are incorporated to explain the variation in the various mathematics knowledge outcomes. This kind of "production function" analysis is common in quantitative studies in education, although usually the dependent variable is student achievement instead of teacher achievement. The purpose of this aspect of the analysis is to understand more about the underlying factors that may determine teacher knowledge levels, as well as carry out more precise comparisons between types of teachers and schools.

With respect to teaching practices, these were captured by hand–held video cameras, 20 lessons in Panama and 30 lessons in Costa Rica. This was done with the prior consent of the teacher, who was assured that the purpose of the video was purely investigative. The lessons were then analyzed considering three critical elements of teaching: mathematical proficiency of the lesson (as defined by National Research Council's (NCR, 2001) study of mathematics instruction), level of cognitive demand (as defined by Stein et al. (2000)), and the teacher's mathematical knowledge observed in the lesson (adapted from Learning Mathematics for Teaching Project's Quality of Mathematics Instruction video instrument). The reason to analyze the videos using the above rubric was to quantify the quality of mathematics in instruction and to assess the knowledge of the teachers when teaching. By quality of mathematics in instruction, we mean the extent of key mathematical characteristics in a lesson, including the richness and correctness of mathematical content, the opportunity the students have to engage in tasks with different levels of cognitive demand, and the ability of the teacher to teach effectively.

To capture the presence of the different elements, a coding system was used for each lesson. After observing a particular lesson the researcher adjudicated a code of "present" (P) or "not present" (NP) for each strand or component that defines the three elements of teaching mentioned above (see Section 4 for more details). A conservative judgment was used for the "present" code, that is, if the component was observed at least once during the lesson, a code of P was adjudicated. Other video studies (e.g. Learning Mathematics for Teaching Project) have broken the lessons into small segments of 5 or 10 minutes to account for the complexity of instruction; however when this method was applied to our lessons, we did not find significant differences between the codes considering the lesson as a whole and lessons broken into smaller segments. For logistical reasons we did not conduct any type of structured or semi–structured interviews with the teachers that were videotaped. This is a limitation because interviews would allow us not just to validate our code system but also to enrich our understanding of teaching practices in these countries.

 

3. RESULTS

We have some a priori reasons to expect Costa Rican teachers to do better on the various measures of quality. Costa Rica spends more on schooling overall, which translates into longer training for teachers and higher pay, particularly for primary teachers. Furthermore, in the recently released study of student achievement by UNESCO, Costa Rica performed much higher than Panama (UNESCO, 2008). In the following sections we test whether or not these differences turn up in observable differences in teacher mathematics knowledge. We only present the results for teacher training and teacher knowledge in this section. Section 4 is entirely dedicated to the results and analysis of teaching practices.

 

3.1. Teacher Education, Professional Development, and Salaries

In comparing teacher preparation between Panama and Costa Rica, several general points can be made. First, all Costa Rican primary teachers have university degrees, while in Panama, primary teachers can opt for a Normal School degree (grades 10–12, with an additional year of postsecondary training) or a university degree, which means that, on average, Costa Rican primary teachers have more training and slightly more preparation in mathematics than Panamanian teachers. However, primary teachers with university degrees have similar levels of preparation in mathematics: a primary teacher with a four–year degree (Licenciatura) from the University of Panama receives one mathematics content course, two statistics courses, and two math education courses. A primary teacher with a four–year degree (Bachillerato) from the University of Costa Rica receives two math content courses, one introduction to statistics for education majors course, and one math education course. Second, at the secondary level, math teachers in both countries receive considerable preparation in math content. However, teachers with Bachilleratos in secondary math education from the University of Costa Rica have three courses in math education, while teachers with Licenciaturas in math and Profesorados in secondary education from the University of Panama receive no math education courses. In this respect, Costa Rican secondary math teachers are likely to be better prepared to teach the mathematics content that they have learned at the university.

In both Panama and Costa Rica, there do not appear to be strong incentives for teachers participating in professional development. However, professional development opportunities for Costa Rican teachers appear to be much greater. For example, Costa Rica has a National Pedagogy Center (Centro Nacional de Didáctica, CENADI) in charge of the professional development of both primary and secondary teachers. While in Panama, training opportunities are concentrated during teachers' vacation time, Costa Rican teachers have opportunities to participate in extensive professional development during both vacations and the school year. Moreover, these opportunities take place at multiple levels and from various sources. Finally, the existence of CENADI and the requirement of regional offices to design and submit a yearly professional development plan make the Costa Rican approach appear somewhat better coordinated than the approach in Panama.

Costa Rican teachers earn on average somewhat higher salaries than Panamanian teachers. Although Costa Rica's household survey does not distinguish between primary and secondary teachers (all younger teachers have at least university degrees), Costa Rican teachers aged 25–44 in 1995 averaged 7% higher salaries than primary school teachers in Panama in 2000 and 21% higher in 2005 (Carnoy et al., 2007b). However, after converting salaries based on purchasing power parity (a method that uses a larger basket of goods and better reflects the amount of goods that salaries can actually purchase), Costa Rican teachers earn considerably more than Panamanian teachers, especially at the primary level. For example, a Costa Rican teacher in his or her mid 30s earned approximately 1,100 PPP dollars in 2000, compared to approximately 700 PPP dollars earned by a Panamanian primary teacher.

Of course, the ability to recruit well–educated young people into teaching depends largely on the salaries available in other professions. Relative salaries (as opposed to nominal salaries) give a much better picture of the tradeoffs that young people must make when deciding whether to go into teaching. Due to different occupational classification schemes, it is difficult to compare relative salaries between Panama and Costa Rica. However, comparing the salaries of male scientists to male teachers in the two countries, the differential is much greater in Panama in 2000 (124% more for scientists) than in Costa Rica in 2005 (68% more for scientists). Comparing scientists to profesores in Panama, scientists only earned 30% more in 2000. Similarly, male social scientists in Panama in 2000 earned 204% more than male maestros but only 76% more than male profesores. In Costa Rica in 2005, male social scientists earned 74% more than male teachers, a very similar differential to that between Panamanian social scientists and profesores. Consequently, differences in salaries between Panamanian and Costa Rican teachers, as well as between other Panamanian professionals and Panamanian teachers, appear to be largely driven by the low salaries earned by Panamanian maestros, or primary teachers. Given the large differential between salaries in math–oriented professions (for example, engineers and scientists) and salaries of maestros in Panama, it may be much more difficult in that country to recruit young people who have strong mathematics knowledge into primary teaching.

 

3.2. Teacher Content Knowledge

Third grade Content Knowledge

Table II summarizes teacher mathematical knowledge. With respect to content knowledge of third grade content, the first result that stands out is the significant advantage for Costa Rican teachers—at all levels. Their averages are roughly 90–95 percent in each category, which is consistent with a teacher preparation regime that requires university level education and more exposure to mathematics content. The Panama teachers, by contrast, answer about 83 percent of the third grade content items correctly. The discrepancy is much lower (but statistically significant) among pre–media (Panama) and "Ciclo III" (Costa Rica) teachers, as the Panamanian group has an average of 89.4 percent.

The largest difference is between the pre–service teachers. The Costa Rican pre–service teachers, who are all university students, score almost 30 percent higher than their Panamanian counterparts at teacher training institutes. There are some potential validity issues with these comparisons in terms of year of study, etc. For example, the Panamanian Normal School students who participated in the exam were in the final year of their training, which is the age equivalent of being a first–year university student. But overall the four comparison categories provide strong support for our guiding hypothesis related to Costa Rican superiority in teacher preparation.

Even though not shown here, the comparisons for gender, location of school and type of school reveal few differences. Male and female teachers do not have consistently different content knowledge at this level. The same is true for teachers working in rural and urban areas. These results are somewhat surprising, as gender differences in mathematics are not uncommon in Latin America, although usually not at this level of content (Marshall and Sorto, 2007). We could also expect teacher quality differences by location if rural areas are less desirable places to work. For the private–public comparison there is some evidence of a private school advantage in Panama. Given the low scores overall in public schools this is not surprising, since private schools are likely to recruit more educated teachers.

The overall averages in table II are useful for making general statements about teacher knowledge across countries, and comparing certain kinds of teachers within each context. One point to note is that the advantage for Costa Rica is the same when the comparisons are restricted to public school teachers only. The relatively low scores in Panama are therefore somewhat troubling, and at least raise some doubts about the overall quality of the teaching force. One way to further contextualize these results is to see actual examples of teacher mistakes. Box I provides an example item taken from primary level mathematics in the area of geometry. The example is useful because it shows how teacher response patterns to a specific item vary by country and level of training.

⁷

The results in box I help contextualize the global summaries of teacher primary level content knowledge presented in table II. Among the grade 3 teachers in Panama, 71.7 percent answered this item correctly, which makes it one of the more difficult items⁶. Eight of the 46 teachers chose answers a, b or c, while another five left the answer blank. This latter group is somewhat problematic, since we cannot be certain that they left it blank because they weren't sure of the answer (see discussion above). Nevertheless, the results for this item are consistent with the overall pattern for third grade teachers in this area of knowledge: most are comfortable with these items, but a significant minority show signs of basic content knowledge deficiencies. Costa Rican third grade teachers have fewer problems with this item. Roughly 91 percent answered it correctly. The same is true for pre–media and Ciclo III teachers in Panama and Costa Rica, which is not surprising since these are mathematics specialists.

What do the results in table II and box I mean for teacher training and preparation in Panama? In Panama primary teacher basic content knowledge is clearly not at an "ideal" level. It is probably unrealistic to expect all primary school teachers—in any context—to answer every primary level item correctly. There will always be errors, and these questionnaires were applied in many cases during the working day when teachers were giving classes. But the fact that upwards of one quarter of the primary school teachers did not get the correct answer to the example in box I highlights the need for better preparation for teachers. These questions are not very difficult, and for third grade teachers this is content that is taken directly from the official curriculum for that grade. This in turn also indicates potential deficiencies in the teacher's pedagogical content knowledge at this level, since their ability to frame useful explanations and activities is likely to be limited when they struggle (at times) with the same content.

 

Seventh Grade Content Knowledge

Table II also summarizes teacher proficiency in seventh grade mathematics. Once again, the main finding is the discrepancy between Costa Rica and Panama. For third grade teachers the gap is very large—almost thirty percent. This is further evidence of a more intensive pre–service preparation emphasizing content knowledge at different levels of the subject.

We should expect Panama middle school teachers to be better trained than their primary level counterparts, and the results largely confirm this. The fact that they score higher than the Panama primary and third grade teachers is not surprising. More importantly, they are nearer the Costa Rican average. This gap is still about 13 percent, which is significant. But the average is high enough for the Panama teachers to be reasonably comfortable with content at the level they are responsible for. There is also a large difference between the two groups of pre–service teachers. Again, we are not entirely certain about the validity of this comparison in terms of the year of study that is being compared. But the important point is that Costa Rican pre–service teachers are scoring on a par with the Costa Rican middle school teachers. This is far ahead of their Panamanian counterparts.

Box II presents an example item from the middle school level together with teacher responses by category. The idea is once again to expand the description of teacher knowledge using a real–life example. The question is not very difficult, although it tests a higher order skill than the geometry item in box I (ordering decimals). The pre–media (Panama) and Ciclo III (Costa Rica) teachers perform well, especially the Costa Ricans. The large gap in knowledge levels is most evident in third grade, where the Costa Rican third grade teachers score almost as high as the Panama middle school teachers. And the Panama third grade teachers demonstrate a serious lack of knowledge about this particular content area.

⁸

Two points stand out from table II and box II. First, in terms of all teachers' knowledge of seventh grade mathematics content, the Costa Rican advantage continues, especially among third grade teachers. These large differences in "higher" level content knowledge provide very clear evidence that the Costa Rican teachers receive a more demanding pre–service preparation. Nevertheless, we must be wary of drawing sweeping conclusions about their relative effectiveness in the classroom based on this particular content since we only expect it to be indirectly related to teaching in grade 3 (see Conceptual Framework).

The second main point is that the teacher quality "gap" between Panama and Costa Rica is less serious at the middle school level. The Costa Rican teachers still score higher than their Panamanian counterparts, and the gap is similar to that between third grade teachers for the third grade content (table II). But based on the third grade and seven content item results the Panama middle school teachers appear to have sufficient knowledge; this is not necessarily the case for third grade teachers in Panama.

 

3.2. Pedagogical Content Knowledge

If student achievement were based solely on the teacher's knowledge of the same content then we would expect students from both countries to score fairly high on exams. But there are other teacher skills in addition to "knowing what you teach." We have already shown why higher level knowledge is relevant, to some degree, and on this count the Costa Rican primary teachers are well prepared. But effective teachers also have a profound conceptual knowledge of the content that they are teaching, which is crucial for their work of passing along knowledge to others. This means not just knowing the content, or higher level mathematics; otherwise anyone who scores well on a math test could teach well.

One of the most original features of our data is the information on the teacher's specialized knowledge of mathematics instruction, also known as pedagogical content knowledge (PCK). The empirical basis for understanding PCK and its influence on teacher effectiveness and student achievement is very limited. Two studies have attempted to link variation in the teacher's specialized knowledge with student achievement differences (Hill, Rowan and Ball, 2005; Marshall and Sorto, 2007). But even in descriptive form there is little evidence from questionnaires that probe what teachers can do in terms of pedagogical knowledge. This also means there are potential validity issues as researchers continue to test these kinds of questions. We will address this in following sections when we compare teacher scores on different constructs.

The second part of table II presents a summary of the pedagogical content knowledge results for the primary level PCK activities, which refer to teaching situations encountered in third grade mathematics. With averages of 50 percent or lower the results for these items suggest that specialized teaching knowledge levels are less than ideal in both countries. For Costa Rica especially this result is significant because these teachers scored so high on both levels of mathematics content knowledge. The implication is that higher level content knowledge alone is not sufficient for providing teachers with the kinds of specialized teaching skills that come from combining content knowledge with pedagogical experiences.

This disconnect between higher–level content knowledge and PCK is further supported by the results for pre–media (or Ciclo III) teachers. Despite scoring nearly perfect on both lower and higher content knowledge, Ciclo III Costa Rican teachers demonstrate specialized teaching knowledge of third grade mathematics that is nearly identical to the specialized teaching knowledge of third grade teachers in Costa Rica. In fact, based solely on content knowledge comparisons, the apparent quality gap between middle school teachers in Costa Rica and primary teachers in Panama is enormous. But using a more applied knowledge framework based on the grade the teacher is responsible for, the difference is not nearly so pronounced.

These initial results support the mathematics education researcher position on effective teaching: effective teaching is largely a function of level–specific competency, and does not automatically result from having higher–level content knowledge (Hill and Ball, 2004). This is a tentative conclusion, however, because the PCK instrument only measures teacher ability on paper and may not reflect actual teaching behaviors. Using classroom observations, we also categorize teaching proficiency based on what teachers do in the classroom; this will provide some additional "triangulación" to check on this preliminary conclusion we have reached regarding teaching effectiveness at the two grade levels.

We still need to address what predicts PCK in third grade. At the bottom of table II some correlations are presented linking PCK with lower and higher content knowledge, and two kinds of classes. The results are very inconsistent across countries, and for third grade level content knowledge, the compression of scores (especially in Costa Rica) makes correlational analysis somewhat problematic. Nevertheless, two results do stand out. First, higher level content knowledge does somewhat predict higher levels of third grade PCK. This is not inconsistent with the PCK formulation; although it would predict that third grade content knowledge matters. Second, the number of pedagogy classes the teachers report having taken is more significantly related to PCK than is the number of content classes. There are some measurement issues with these class types, but the interesting implication is that PCK can probably be taught (e.g. Hill and Ball, 2004).

Once again the global averages are useful for comparisons, but tell us little about what these teachers can actually do. Examples in box III and IV help put the results into a more useable context. The activity presented in box III was an open–ended item graded with a common rubric. The item is a version of the well–known item used by Ma (1999) in her comparisons of teachers in the United States and China. Teachers were asked to create a real life problem for their students that required the use of division of fractions. The responses show that almost all of the teachers in Panama scored zero, in part because a significant portion did not even attempt the activity. This creates some interpretation problems, since these teachers may have felt like they did not have the time to complete this activity. But the more likely explanation, at least based on the results from other parts of the questionnaire, is that they were not very comfortable with the content being covered. In Costa Rica the teachers generally attempted the question, but also scored low. Most received only one point, which means that they created a problem that can be solved by dividing by 2 and not dividing by ½ as the problem asks.

⁹

¹⁰

In box IV we have a closed item measuring pedagogical content knowledge in the area of statistics or data analysis. This item was included in all of the questionnaires and therefore we have results across all levels.

First result that stands out is the higher degree of difficulty for both countries compared to the others items. One reason could be because the item covers statistical concepts that go beyond graph comprehension. Another result that stands out is that Costa Rican teachers are not leading on the performance of this item. In fact, more third grade Panamanian teachers correctly answered it than any other group of teachers followed by pre–service teachers from the University of Panama. Interestingly, 73% of those third grade teachers that answered correctly graduated from the University of Panama and not from the Normal School. We also note that the seventh grade teachers from both countries scored significantly lower than the other two levels (p < 0.01 in a logistic regression model). The implication is that secondary teacher preparation in statistics in both countries does not focus on graphical representations. The popular choice, especially among Costa Rican seventh grade teachers, that none of the two representations is appropriate for illustrating the center and spread of the distribution, suggests that secondary teachers do not recognize these as representations of data with the same features as a bar graph and a dotplot or histogram. It is worth noting that the pattern of performance in mathematical knowledge for teaching for this sample is somewhat different. Still both countries scored much lower than in the domain of mathematical content, but Costa Rican teachers were the leaders.

In addition to measuring pedagogical content knowledge at the primary school level we also created items for measuring the teacher's specialized knowledge at the pre–media level. These items were covered mainly in the pre–media teacher instrument, with a handful included in the pre–service questionnaire. Since the pre–service teachers in both Costa Rica and Panama are mainly preparing for primary school these comparisons by level of instruction are not particularly important.

The results are generally consistent with the earlier comparisons among third grade teachers. There is no significant difference between Panama and Costa Rica teachers on these activities. This is not that surprising given the relatively equal content knowledge and the more advanced training received by middle school mathematics teachers in Panama compared with their third grade counterparts. But the second result is also consistent with third grade: the overall averages are low enough to suggest some deficiencies in the teachers' specialized knowledge.

In sum, the main conclusion from this section on pedagogical content knowledge is that teachers in Panama and Costa Rica do not have a high level of specialized knowledge for teaching mathematics. The Costa Rican third grade teachers do score significantly higher than their Panamanian counterparts, but they are still struggling with some of these more demanding questions that test their teaching knowledge. This form of knowledge is important because we feel that a profound conceptual understanding of the topic one teaches is necessary for passing along knowledge to children. It is not enough to have a basic grasp of the content itself—most teachers have that—or even a basic grasp of how to teach it—most of these teachers appear to have that, too. Effective teachers have high levels of specialized knowledge, and based on these results we should expect some consequences in the classrooms of teachers who do not have such specialized knowledge.

What are the consequences in the classroom when the teacher does not have a high level of specialized teaching knowledge? We will return to this in some of the classroom videoanalysisactivitiesinthenext section.However, teacher'sproficiency on similar items in other contexts has been shown to be a significant predictor of student achievement (Hill, Rowan and Ball, 2005; Marshall and Sorto, 2007).

 

3.3. Multivariate Analysis

The correlational analysis presented at the bottom of table II provided some initial clues into the factors that determine the teacher's pedagogical content knowledge. In this section we continue this line of analysis using a more powerful analytical tool called multivariate regression analysis. Two questions are most pressing. First, can we identify concrete mechanisms for improving teacher content knowledge or PCK, such as taking more classes? Second, how are these different forms of knowledge related to each other, especially meaning how do third and seventh grade content knowledge predict PCK? This information is useful because we need to understand the extent to which each form of knowledge is "taught," as opposed to it being accumulated through experiences or natural ability.

The purpose of multivariate analysis is to replicate the conditions of an experiment where all things are equal except one "treatment" variable. Of course these data are not experimental, and there are potential interpretational problems with the results that follow. But at the very least the multivariate results provide some complementary evidence to help interpret the results and draw linkages— however tentative—with other results found in this study.

The results are detailed in table III. The dependent variables are measured in standard deviations, as are the independent variables (when feasible). For the determinants of the teacher's third grade content knowledge, the results do not add much to the discussion already presented. Most of the variables are insignificant, especially in Costa Rica where the teachers scored very high and there is not much variation overall. The control for teacher type are very significant in Panama, and confirm that pre–service teachers score much lower than primary teachers (1.36 standard deviations lower), and middle school teacher score higher than everyone else (about half a standard deviation). In the estimation that uses data from both countries, the control for Panama versus Costa Rica is also very negative and significant. This simply indicates that when all else is equal the Panama teachers have much lower (nearly one standard deviation) content knowledge. One interesting result in the Panama–only estimation is that the number of content classes taken by the teacher significantly predicts higher achievement. This makes sense—to some extent—but it remains to be seen how important these pre–service preparation classes are as predictors of teaching knowledge.

For seventh grade content knowledge (middle columns) the results are similar to the third grade content results. The significant difference by gender appears to be driven primarily by the lower scores for Panamanian females versus their male counterparts. In the Panama–only model the number of pedagogy classes is a significant predictor of seventh grade content knowledge. Each class is associated with about 0.10 standard deviations higher PCK. This is somewhat surprising, and does raise some issues about the validity of these variables that measure the teacher's (self–reported) coursework. Once again the controls for teacher type and Panama confirm the descriptive comparisons presented in earlier tables.

Our main interest is in understanding the factors that predict higher levels of teacher pedagogical content knowledge. These estimations for third grade PCK are presented in the right hand side columns of table III. One result that stands out is for the teacher's self–reported pedagogical preparation, which is significantly associated with PCK in Panama. For Costa Rica the point estimate is also positive but not significant at conventional statistical levels. This is a tentative linkage, and we are reminded of the potential problems with these measures of classes. But the suggestion is that PCK could be improved through teacher training.

The other variables of interest in the PCK estimations are the content knowledge controls. This linkage is an important one because pedagogical content knowledge has an obvious content knowledge component. In theory the content knowledge that matters should come from third grade rather than seventh grade. We have already argued that the low third grade PCK scores for Costa Rican and Panamanian middle school teachers shows how higher level knowledge has little to do with this domain of knowledge. The results in table III are mixed. In Panama the expected linkage does hold: the teacher's third grade content knowledge does predict their third grade PCK (standardized effect of 0.17). But in Costa Rica neither measure is significant. This is not that surprising given the lack of variation in the primary level knowledge measure.

 

4. QUALITY OF TEACHING MATHEMATICS

We present the results and analysis of the videotaped lessons. The lessons were analyzed by the presence of three different elements: mathematical proficiency, level of cognitive demand, and the observed teacher knowledge. We should note that one main difference about the lessons is the length of instruction per lesson.

In Panama students receive, officially, 45 minutes of mathematic instruction every day; although we observed lessons as short as 30 minutes. In Costa Rica the classes are organized to be much longer, with two "double shifts" of 80 minutes, one single shift of 40 minutes, and one single shift labeled as "remedial" weekly.

 

4.1. Mathematical Proficiency

The quality of the mathematics content being taught can also be assessed by observing the presence of five intertwined strands that form the mathematical proficiency variable as defined by the authors of Adding It Up (National Research Council (2001)). Mathematical proficiency, a term that encompasses expertise, knowledge, and facility in mathematics, captures what the authors believe to be necessary for anyone to learn (and by implication teach) mathematics. The authors identify five key strands of mathematical proficiency, as follows:

– Conceptual understanding. Comprehension of mathematical concepts, operations and relations.

– Procedural fluency. Skill in carrying out procedures flexibly, accurately, efficiently, and appropriately.

– Strategic competence. Ability to formulate, represent, and solve mathematical problems.

– Adaptive reasoning. Capacity for logical thought, reflection, explanation and justification.

– Productive disposition. Habitual inclination to see mathematics as sensible, useful, and worthwhile, coupled with a belief in diligence and one's own efficacy (p. 117).

These strands are not to be taken as individual goals but rather as an interdependent and interwoven definition of proficiency. If any one of the five elements is missing the learning process is not considered complete. We rate each of the lessons based on these five strands, identifying which strands are lacking for any given mathematics lesson. It is a lot to expect all strands to be present in individual lessons, especially in Panama where the classes are between 30 to 45 minutes long. Instead we are more concerned about the extent to which all strands turn up in the overall summary of multiple lessons. In other words, are there specific elements of proficiency that are largely absent from these classrooms?

Results for both grades and countries are shown in figure 2. At both grade levels, a high percentage of lessons in Costa Rica had as a goal to engage students in conceptual understanding. It was clear from the observations at the third grade level, that there is a culture among Costa Rican teachers to value conceptual understanding before students move to the manipulation of symbols or computation. However, as we will see later, not all teachers did this in an efficient way. The opposite is true for the procedural fluency strand. A lower percentage of classrooms in Costa Rica (compared with Panama) engage students in the skill of carrying out procedures. In Panama most of these procedures were related to basic operations, while in Costa Rica they included the construction of geometric shapes or use of measurement instruments. In both countries there were few instances where students had to show the ability to formulate, represent, and solve mathematical problems, which is also known as strategic competence. When this strand was observed the students were given mathematical problems applied to real world situations and asked to apply their knowledge of previous mathematics content learned to arrive at a solution. The students were either engaged in whole class or group discussions.

Again, a higher percentage of lessons in Costa Rica, although a low overall percentage, engaged students in logical thought, reflection, explanation, and justification ("Reasoning"). Rules, definitions and procedures were often presented without providing an opportunity for students to wonder why they were true. When students were involved in working on a problem or asked to give an answer, they were not expected to explain their reasoning or provide a valid justification. Many educators call this type of teaching as "answer–centered."

Finally, the last category refers to the level of students' ability to see mathematics as sensible, useful, and worthwhile ("Disposition"). This category was observed only during the lessons where students were either involved in the application or reasoning of mathematics, which in both countries was low. However, among those lessons students seemed to enjoy and value the logical thinking and problem solving activities.

Larger differences between countries are observed for seventh grade lessons (figure 2). It bears restating that these differences are based on fairly small samples of classrooms (7 for Panama and 19 for Costa Rica). All lessons in Panama were about computing or following a procedure with a few making linkages to the underlying concept. In Costa Rica a higher percentage of lessons attempted to relate the procedure with the concept (or concepts). Again, not all teachers were effective at it. In Panama, none of the lessons made linkages of the mathematics to problem solving and in Costa Rica very few did so—about the same percentage as third grade. The logical thinking and reasoning were also low in both countries. A better level of productive disposition was observed in Costa Rica lessons; however it is still low in absolute terms. Overall, the mathematics instruction in Costa Rica for seventh grade "looked" closer to the mathematical instruction in third grade. More pronounced differences between grades were observed in Panama.

 

4.2. Level of Cognitive Demand

Beyond the topic covered in the lesson is the kind and level of thinking required of students on a particular topic or mathematical task, which enriches and relates to our previous measurement of mathematical proficiency. We refer to this aspect as the level of cognitive demand.

For level of cognitive demand we derive a rubric from Stein et al.'s (2000) classification of higher and lower cognitive demand. These include:

– Memorization. Recollection of facts, formulae, or definitions.

– Procedures without connections. Performing algorithmictype of problems that have no connection to the underlying concept or meaning.

– Procedures with connections. Use of procedures with the purpose of developing deeper levels of understanding concepts or ideas.

– Doing Mathematics. Complex and nonalgorithmic thinking, students explore and investigate the nature of the concepts and relationships.

Even though the level of cognitive demand is an aspect more related to the learner, it is the teacher that controls and directs the required level for his or her students. In a similar study using videotapes from a large group of TIMSS participants, the researchers found that teachers implement lessons at a lower level than what the lessons are intended for (TIMSS 1999 Video Study). This was a characteristic especially of countries with lower student achievement overall.

Why compare Panama and Costa Rica on this aspect? What do we gain by comparing the required level of thinking of students? There are several reasons. First, the level of cognitive demand of the lesson, or the mathematical activities the students engage in, is closely associated with the deep understanding of concepts in mathematics. This is independent of whether students are put in groups or are given manipulatives, and we can also make inferences about the teacher's pedagogical knowledge, since he or she needs to be involved in the same kind and level of thinking.

The actual measurement is not easy. Lessons often have multiple stages, and students and teachers are involved in several mathematical activities that vary in their complexity, often driven by the main goal of the lesson. Therefore, one lesson can be characterized on a whole as a low–level lesson or a mixture of low and high levels. To provide a more systematic way to characterize the lessons, we used Stein et al.'s categories. These include memorization, procedures without connections, procedures with connections, and doing mathematics. The memorization and procedures without connections are related to the aspect of mathematical proficiency of procedural fluency, tasks that engage kids on procedures with connections often call for conceptual understanding and reasoning, and tasks that engage kids with "doing mathematics" are tasks that have the presence of all aspects of mathematical proficiency strand.

Results for both grade levels and countries are shown in figure 3. At the third grade level, in both countries a large percentage of the lessons required students to simply recall rules and definitions, or perform algorithms with no relation to the underlying concepts. Panama had a higher percentage of these types of lessons. Opposite patterns are observed for the higher–level cognitive demands. Both countries have a smaller percentage of lessons requiring students to understand the meaning of operations or underlying concepts behind the procedures, and a still smaller percentage require students to investigate or explore relationships between mathematical ideas. Panama had an especially low percentage of higher–level demand lessons, although Costa Rica overall is not much higher.

There are two important observations about the level of cognitive demand for lessons observed in both countries. One, the observed level was the one implemented by the teacher and not necessarily the level intended. For example, the tapes from Costa Rica show that about 60% of the teachers had intended to deliver a higher level lesson, guided by textbooks, pre–prepared activities, and concrete models. However, only 43% successfully implemented it. The Costa Rican teachers also tend to use more demanding questions, but the actual formulation and sequence of questions does not always make it possible to probe the students' conceptual understanding. These findings are consistent with results from the TIMSS 1999 Video Study and with findings by Stein et al. Mathematical tasks or problems with high level cognitive demands "are most difficult to implement well, frequently being transformed into less–demanding tasks during instruction" (2000, p. 4).

We were not able to measure the degree of intended levels in Panama's taped lessons. Most teachers in Panama did not use textbooks, concrete models, or other materials that could have been useful for assessing the goal of the lesson. However, based on an examination of the official third grade mathematics textbook, the levels of cognitive demand were more evenly distributed across topics than the observed distribution in the tapes.

The second main finding is that lesson length plays a key role in the implementation of high–level cognitive demands. Costa Rica's lessons were about twice as long as Panama's lessons. This means there were more opportunities for the students to engage in different activities in each of the different levels. This does not mean that Costa Rica students receive dramatically more mathematics during the week; the total time difference is about half an hour. But by using "blocks" of class time on different days the Costa Rican strategy allows for longer periods. This does present the challenge of holding the students' attention for long periods, and there are some concerns about student engagement in the Costa Rican classrooms as time goes on. But there may be some potential advantages to this strategy in terms of lesson development, especially when combined with better prepared teachers. Compared with other aspects of Costa Rican schools this is a potentially easy element to copy, but teachers in Panama would have to undergo considerable retraining to teach in a different format.

The overall pattern of levels of cognitive demand is similar in both countries for seventh grade (figure 3). Panama lessons engage students in tasks that are focused on memorizing definitions and/or procedures and only a few lessons made connections with other concepts. None require the students to either explore/understand the nature of mathematical concepts or analyze and actively examine multiple solutions.

The Costa Rican lessons had a more uniform distribution of levels, which is what is expected. Note that seventh grade has a more uniform distribution than third grade. A lower percentage of lessons were associated with low levels of cognitive demand in seventh grade compared with third grade. We should note that third grade involves students working on different tasks of high cognitive demand than seventh grade. For example, third grade tasks are for the most part hands–on activities that illustrate a mathematical concept and seventh grade tasks are interesting mathematical problems which require students to reason.

Another important observation about the implementation of the level of a specific task is that seventh grade teachers were more effective at it. That is, if a lesson was intended for a high level, the implementation was also at high level.

 

4.3 Mathematical Knowledge Observed

In the previous section (Section 3) we provided extensive measures of teacher knowledge based on their answers to items on a questionnaire. In this part of the analysis we turn to classroom observations to classify teacher knowledge. This is a novel approach with few antecedents, and a number of challenges to implement.

We characterize the observed teachers' knowledge in a lesson in a similar way as in the written instrument that measures their specialized knowledge with questions. This includes:

– Content knowledge. This refers only to teachers' knowledge of the mathematics being taught, grade 3 and 7 in our case.

– Pedagogical knowledge. This refers to knowledge of instructional techniques beyond lecture mode. Elements include how well the teacher has all of the students engaged, his/her use of proper classroom management techniques, and the quality of instructional materials.

– Pedagogical content knowledge. This refers to the appropriate integration of the instructional techniques with the mathematical concept being taught and its effectiveness on student learning.

Of course, what is observed in one lesson does not measure the entire body of knowledge a teacher has in mathematics, or any of the other kinds of knowledge. But the purpose of looking at the teacher's knowledge for these lessons is not to characterize the entire knowledge of a teacher; for this we would need a case study where we observe a teacher for a long period of time. The purpose is to measure how well the teacher uses these specific knowledge forms in a particular lesson.

Figure 4 shows the percentage of teachers that demonstrated knowledge in each of the kinds of knowledge described above. One important note is that the kind of knowledge demonstrated was connected with the overall goal of the lesson. In both countries, almost all teachers did not say or write anything incorrect with serious consequences for future learning. In terms of pedagogical knowledge, it was clear that Costa Rican teachers use better pedagogical techniques. In particular, the Costa Rican teachers used concrete models to illustrate concepts, and more frequently used hands–on activities such as measuring, cutting, coloring, and pasting. This measure is linked with the intended level of cognitive demand of the lesson analyzed above. The final element is the degree of effectiveness of the use of these techniques and how well they were connected with the mathematical concept being taught; this is measured by the last category. Costa Rica also showed a higher percentage of teachers with this quality, however the difference is not as large as the pedagogical knowledge category.

Patterns of teachers' knowledge for seventh grade are somewhat different than grade three (figure 4). First, every teacher observed in both countries spoke and/or wrote correct mathematical statements. A lower percentage of teachers used pedagogical techniques compared to third grade in both countries. The interesting result here is that seventh grade teachers in both countries showed more effectiveness in the use of those techniques than third grade teachers. Some of this effectiveness was clearly due to the pedagogical technique being implemented, like a well planned lesson with a rich task presented to students and good "flow" of the lesson. Others were effective because of the powerful explanations and skillful level of communication on the part of the teacher to bring the complex mathematical ideas to the level of the student (almost like in an artistic manner). The first kind of teacher becomes effective by formal education, while it is likely that the second is effective because of his or her talent and/or experience.

 

5. CONCLUSIONS AND RECOMMENDATIONS

5.1. Conclusions

The results of this study provide some useful insights into the relative effectiveness of teachers in each country, at least based on their ability to answer mathematics questions of different types and their performance in the classroom. The main results include:

– Teaching practices in Panama and Costa Rica are marked by considerable differences. The differences are more pronounced in third grade than in seventh grade. A typical Costa Rica mathematics classroom in third grade is characterized by a large portion of the time working with concrete materials, often constructed by the children themselves. Children in these classrooms are often busy cutting, coloring, pasting, counting, grouping and moving objects. They also have prepared activities or workbooks to use. The teacher then involves the children in a whole class discussion about the concept or topic being taught and the lesson ends with a summary or children recording their work in notebooks. What these classrooms in Costa Rica lack is the connection with the underlying mathematical concepts. The focus is on the construction and manipulation of the concrete materials as a physical activity and not as a pedagogical tool to help children learn the abstract underlying concept. Teachers in Costa Rica show a variety of ways to teach young children, hence showing good pedagogical skills but lacking knowledge of how to connect it to the mathematics. In this way the lessons are not as efficient as they could be, even though the teachers have very high content knowledge of the topic.

– By contrast, the typical lesson in Panama third grade is characterized by the teacher lecturing about a concept or a topic for a short time, doing an example of an exercise in the board, and the rest of the time children work on their notebooks doing similar problems. Children in Panama have less opportunity to manipulate concrete materials, to participate in activities that require motor skills, and to engage in whole class discussions. In part this is because their lessons are about half the time compared with lessons in Costa Rica, and there is also a resource effect in the form of a lack of instructional materials. One good practice observed in Panama is the ability of teachers to check student work and be able to correct mistakes as the learning is happening. This can be done because of the small number of children per classroom and the technique of sending children to the board to show their work. However, even this practice is not as efficient as it could be because the "checking" is answer–centered, and does not make it possible to get to the source ofmistakes or different ways ofthinking. This is another example of deficient pedagogical content knowledge.

– Seventh grade mathematic classrooms in both countries have more similarities than third grade classrooms. Teachers in both countries use lecturing as the main mode of instruction, followed by individual work on a list of exercises. The main difference is that Costa Rican teachers use more prepared worksheets than in Panama. Panamanian teachers use the blackboard to present the exercises. It is clear that these teachers are proficient in the content they are instructing. But when it comes to making connections and touching on elements of the critical knowledge that is required for teachers to be effective, once again they fall short in both countries.

– In terms of mathematical quality of the lessons, both countries focus more on knowing rules and procedures than in making connections, reasoning why rules and procedures work or solving problems. Costa Rican teachers are clearly more focused on developing these kinds of conceptual activities, but they are not always successful.

– In purely qualitative terms, our findings are consistent across the different measures used in this study. The teachers are generally proficient with content, especially in seventh grade and in Costa Rica in both grades, and this shows up in their pedagogy. The Costa Rican third grade teachers have higher levels of pedagogical content knowledge, and this turns up primarily in their classrooms in the form of more effective pedagogy and a better attempt—if still incomplete—to draw linkages with more advanced concepts. In sum, the Costa Rican third grade classes are superior, mainly because of the pedagogy employed. But in all four classroom types—third and seventh grades in both countries—there is a clear need to upgrade the teacher's profound teaching knowledge of mathematics. And there also appears to be a need to get teachers to apply more of their knowledge in the classroom.

 

5.2. Recommendations

These conclusions are based solely on teacher responses to questionnaires and on the contextualization of these findings considering teaching processes inside the classroom. Nevertheless, the results are important enough to warrant some recommendations for teacher training and preparation. These include:

– Restructure the time allocation for the mathematic classes in Panama. It is not clear that long periods (80 minutes) in Costa Rica are better than the shorter periods (30 to 45 minutes) used in Panama because there are many classroom in Costa Rica that do not utilize the double period efficiently. Lessons in Panama are too short to engage students with activities that require explorations, investigations, manipulation and creation of mathematical models associated with higher levels of mathematical proficiency and cognitive demand. These longer periods are more crucial in grade 3 than in grade 7.

– Panama can learn from the large repertoire of teaching techniques Costa Rica teachers show at the third grade level. In terms of pedagogy, Costa Rican teachers are more up to date with recent developments in the field, and their classes are similar to those found in industrialized contexts like the United States. One of the teachers even used the latest software that is being used in US classrooms to teach geometry. These are skills that can be easily attained by specialized teacher training or a course in teacher preparation.

– Costa Rica seems to benefit from the use of prepared materials and textbooks that are aligned with the curriculum and therefore "force" the teachers to teach at the level expected by the national standards. Panama can learn from Costa Rica how to make teachers use the textbooks. Availability is not the issue, but changing teacher behaviors is clearly critical.

– Panama needs to increase the mathematical knowledge required to become a primary teacher in both areas, content and pedagogical content. The combinations of these two are powerful skills that make teachers increase their teaching quality. This means more pre–service classes, which in Panama especially must have a content focus as well.

– Seventh grade teachers in both countries need to be aware of how mathematics is taught in the elementary school. This can be done by creating a special program for teachers of 7–9 grades or by taking the same pedagogical content classes as the elementary teachers. Secondary teachers also need to increase their knowledge of how to illustrate concepts of middle grades with multiple representations or models. The instruction of seventh grade was mostly represented with symbols (numbers or variables).

– Costa Rican teachers need to improve further their teaching of higher mathematical content. Our classroom observations in both third and seventh grade suggest that many teachers are not using the content knowledge plus PCK that they have to teach effectively. This may be a result of a curriculum that is not sufficiently demanding, or of a teacher training that does not instill enough confidence in teachers to use their mathematical knowledge in their teaching, or of a general culture of low expectations in schools that manifests itself in low demand on students by teachers. Another explanation could be the resistance of teachers to change their instruction (Ramos & Font, 2008).

– Other countries in Central America can take general lessons from this comparison of Costa Rica and Panama. Although both countries are economic leaders in the region, they have approached education very differently. While Costa Rica has required that all primary teachers earn a minimum of a bachelors degree, a large fraction of Panamanian primary teachers have less than a bachelors. Additionally, given a relatively low level of mathematics taught at Panama's secondary–level teachers college, Costa Rican primary teachers have on average much greater preparation in mathematics and teaching of mathematics. These differences may stem in part from Costa Rica's historical commitment to education, manifested by relatively high average educational attainment. On the other hand, Panama has recently demonstrated a strong commitment to empirical research and collection of data that inform this report and will inform future policy. We can make two tentative recommendations for other Central American countries seeking to improve educational quality: (1) increase investments in mathematics education of teachers at all level of the system and (2) evaluate these and other investments with sound empirical research.

 

ACKNOWLEDGEMENT

The authors wish to thank Emanuel Gonzalez–Revilla for providing the funds for this study. We also would like to thank Cristina Cardoze Kelly and Haydee Mendiola for their logistic and administrative assistance in Panama and Costa Rica respectively. Finally, we would like to thank the personnel of the Ministry of Education of both countries for their collaboration and assistance.

 

REFERENCES

Ball, D. L. & Bass, H. (2000). Interweaving content and pedagogy in teaching and learning to teach: Knowing and using mathematics. In J. Boaier (Ed.), Multiples perspectives on the teaching and learning of mathematics. Westport, CT: Ablex.        [ Links ]

Ball. D. L., Hill, H., & Bass, H. (2005). Knowing mathematics for teaching: Who knows mathematics well enough to teach third grade, and how can we decide? American Educator, Fall 2005, 14–22.        [ Links ]

Carnoy, M., with A. Gove & J.H. Marshall (2007). Cuba's academic advantage: Why students in Cuba do better in school. Stanford University Press.        [ Links ]

Carnoy, M., Luschei, T., Marshall, J., Naranjo, B., & Sorto, A. (2007a). Improving Panama's education system for the Twenty–first Century." Unpublished Manuscript, University of Pennsylvania.        [ Links ]

Carnoy, M., Luschei, T., Marshall, J., Naranjo, B., & Sorto, A. (2007b). Improving Panama and Costa Rica's education system for the Twenty–first Century economy and society: A comparative study. Unpublished Manuscript, University of Pennsylvania.        [ Links ]

Conference Board of the Mathematical Sciences. (2001). The mathematical education of teachers. Providence, R. I. and Washington, D. C: American Mathematical Society and Mathematical Association of America.        [ Links ]

Hill, H. & Ball, D.L. (2004). Learning mathematics for teaching: Results from California's mathematics professional development institutes. Journal for Research in Mathematics Education, 35 (5), 330–351.        [ Links ]

Hill, H., Rowan, B., & Ball, D.L. (2005). Effects of teachers' mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42, 371–406.        [ Links ]

Hill, H., Ball, D. L., & Schilling S. G. (2008). Unpacking pedagogical content knowledge: conceptualizing and measuring teachers' topic–specific knowledge of students. Journal for Research in Mathematics Education, 39 (4), 372–400.        [ Links ]

Kilpatrick, J., Swafford, J., & Findell, B. (Eds.). (2001). Adding it Up: Helping Children learn mathematics. Washington DC: National Academy Press.        [ Links ]

Learning Mathematics for Teaching (LMT) (2006). Measures of teachers' knowledge for teaching mathematics. Ann Arbor,         [ Links ]

MI: Author. Retrieved February 23, 2009, from University of Michigan, LMT Project Web site: www.sitemaker.umich.edu/lmt        [ Links ]

Ma., L. (1999). Knowing and teaching elementary mathematics: Teachers' understanding of fundamental mathematics in China and the United States. Mahwah, N.J.: Lawrence Erlbaum Associates.        [ Links ]

Marshall, J.H. and M.A. Sorto. Teaching What You Know or Knowing How to Teach? The Effects of Different Forms of Teacher Mathematics Knowledge on Student Achievement in Rural Guatemala. Harvard Educational Review. Unpublished manuscript        [ Links ]

Ramos, A. B. y Font, V. (2008). Criterios de idoneidad y valoración de cambios en el proceso de instrucción matemática. Revista Latinoamericana de Investigación en Matemática Educativa. 11 (2), 233–265.        [ Links ]

Shulman, L.S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15, 4–14.        [ Links ]

Stein, M.K., Smith, M.S., Henningsen, M.A. & Silver, E.A. (2000). Implementing Standards–Based Mathematics Instruction: A Casebook for Professional Development. New York: Teachers College Press.        [ Links ]

UMCE (2004). Factores asociados al rendimiento académico de los alumnos de la educación primaria en Honduras 2002. (Factors associated with primary school academic achievement in Honduras 2002). Published by UMCE project. Tegucigalpa, Honduras.        [ Links ]

UNESCO (2002). Estudio cualitativo de escuelas con resultados destacables in seite países latinoamericanos. Santiago, Chile: Author.        [ Links ]

UNESCO (2008). Segundo Estudio Regional Comparativo y Explicativo. Retrieved February 29, 2009, from the United Nations Educational Scientific and Cultural Organization Web site: http://unesdoc.unesco.org/images/0016/001606/160659S.pdf.        [ Links ]

 

NOTES

¹ A few months after this study was conducted, the results of the Second Regional Comparative and Explanatory Study by the Laboratorio Latinoamericano de Evaluación de la Calidad de la Educación (LLECE) indicated that Costa Rica had a median score significantly superior in both third and sixth grade; while Panama had a median score significantly inferior among the 17 Latin American countries that participated in the study. However, the LLECE report only provides school–related variables to explain differences among countries. Our study contributes to the LLECE report by providing the important variables of teachers' knowledge and teaching practices.

² In their 2001 report (The Mathematical Education of Teachers) the Conference Board of the Mathematical Science, American Mathematical Society, and Mathematical Association of America recommends "a thorough mastery of the mathematics in several grades beyond that which they expect to teach, as well as of the mathematics in earlier grades."

³ For example, Ball, Hill & Bass (2005) argue that a "mathematically literate" person would struggle to answer questions they created that measure specialized knowledge.

⁴ In Panama two versions of the primary questionnaire were applied. These were identical except the grade three teacher version included so–called "opportunity to learn" (OTL) questions about teacher coverage of specific components of the curriculum. These questions were not included on the other forms in Panama, and for the Costa Rica data collection it was decided not to include the grade three OTL questions.

⁵ Teachers were assured that these questions were for research purposes only, and were asked to complete items when they had left them blank. However, in some cases they chose not to respond to items. These blank items are marked as incorrect. This is justified by some preliminary statistical analysis that shows that teachers tended to leave harder items blank, and these same teachers tended to score lower on the remaining items that they did answer. However, if teachers were simply skipping some questions then this strategy will bias the overall averages downwards. This is not a serious problem, however, since a small percentage (less than 10 percent) of the content knowledge questions were left blank.

⁶ Among the 12 anchor items for lower level content knowledge only two other items are as difficult or more difficult (for Panama teachers). Four of the others have averages correct above 90 percent.

⁷ Original item was presented in Spanish language.

⁸ Original item was presented in Spanish language.

⁹ Original item was presented in Spanish language.

¹⁰ Original item was presented in Spanish language.