Difficulties of items related to energy and matter: Implications for learning progression in high school chemistry

La dificultad de preguntas sobre materia y energía: implicaciones para una progresión de aprendizaje para cursos de química en bachillerato

Xiufeng Liu¹

¹ Graduate School of Education, State University of New York at Buffalo. E-mail: Xliu5@buffalo.edu.

Abstract

Understanding the development of high school students' ideas and ways of reasoning about matter and energy is of central importance to foster meaningful learning in chemistry courses. Similarly, it is critical to investigate student understanding of models and modeling. In this work, we used computer model-based assessment questions to investigate student understanding of: a) matter and energy, and b) models and modeling, in the context of nine different chemistry topics. The comparison of item difficulty across topics allowed us to elicit a possible learning progression for high school chemistry. The results of our study can inform the development of secondary school chemistry curricula.

Keywords: Learning progressions, high school chemistry, matter, energy, computer models.

Resumen

La comprensión de cómo se desarrollan las ideas y las formas de razonamiento de los estudiantes sobre los conceptos de materia y energía es de central importancia para favorecer el aprendizaje significativo en cursos de química. Es también crítico investigar la comprensión de los estudiantes sobre modelos y modelaje. En este trabajo utilizamos preguntas de evaluación basadas en modelos computacionales para investigar la comprensión de los estudiantes sobre: a) materia y energía, y b) modelos y modelaje, en el contexto de nueve temas diferentes de química. La comparación del nivel de dificultad de preguntas sobre diferentes temas nos permitió hacer visible una posible progresión de aprendizaje sobre temas de química en el bachillerato. Los resultados de nuestro estudio pueden informar el desarrollo de currículos de química en ese nivel educativo.

Palabras clave: progresiones de aprendizaje, química de bachillerato, materia, energía, modelos computacionales.

Introduction

Recently, the concept of learning progression has been promoted as a useful tool for organizing science curricula and conducting science instruction (NRC, 2007), and it has informed the development of the next generation science education standards (NRC, 2012; Achieve, 2013). While a number of tentative learning progressions have been developed in different areas (e.g., Alonzo & Steedle, 2009; Duncan et at., 2009; Jin & Anderson, 2012; Songer et at., 2009), little empirical research is available on a learning progression for high school chemistry. When examining the sequence of chemistry topics taught in secondary schools, we may find various orders, which suggests that there is not a commonly agreed upon curricular sequence. Experienced chemistry teachers using the same textbook may decide to follow a different sequence of topics for various valid reasons, and often these reasons are not explicitly articulated to new and more inexperienced teachers. Consequently, there is a need in chemical education research to articulate a learning progression for high school chemistry.

In order to develop such a learning progression, we first need a rationale or framework for the progression of core concepts and ideas. Fostering the development of chemical reasoning can be a powerful way to approach this task. We can conceptualize chemical reasoning as consisting of two main dimensions: understanding of matter, energy, and their interactions, and understanding of models. Matter and energy are foundational for chemistry. Chemistry is the study of the structure, composition, and changes of matter, where the concept of energy serves as a mathematical account for stability and transformation. A combined matter and energy approach to studying chemistry provides a systematic view of the material world; it represents how experts approach questions related to the physical world. This systematic way of thinking was better described by Feynman in one of his well-known lectures, Atoms in Motion (Feynman, 1963). Using a water drop as an example, Feynman explained how various chemistry concepts and ideas, including kinetic molecular theory, structure of matter, evaporation and condensation, chemical change, physical change, and so on, are the result of our systematic imagination and thinking about the world. In the same way that we can use our imagination to envision "the whole universe is in a glass of wine" (Feynman, 1963, p. 66), we can find all chemistry within a single natural phenomenon. This systematic approach to the study of matter demands finding the answer to the following essential questions: How does matter exist? How can we classify it? What is its structure? How does it change chemically? How does it change physically? The answer to these questions requires the understanding of states of matter, classification of matter, atomic structure, chemical bonding, and intermolecular forces. Similarly, systematic thinking about energy demands the answer to the following questions: What is the form of energy? How can it be transferred? What is lost during transfer? How is it conserved? The answers to these questions involve the understanding of forms of energy, energy transfer, energy degradation, and energy conservation. Together, the above questions serve as a guide for systematically thinking about matter and its changes.

Using items designed specifically to measure high school chemistry students' understanding of matter and energy within the context of computer models and modeling, the present study was designed to answer the following questions:

1. What are the difficulty levels of items designed to measure students' understanding of matter and energy in the context of various high school chemistry topics? Is there a significant difference in the average difficulty of items addressing different chemistry topics?

2. How are the difficulty levels of items designed to measure students' understanding of models and modeling in the context of various high school chemistry topics? Is there a significant difference in the average difficulty of items addressing different chemistry topics?

3. Is there a significant interaction effect between the above mean difficulties of items for matter and energy and models and modeling?

The answers to these questions can advance our knowledge and understanding about students' development of the core concepts: matter, energy, and models. Such answers have critical implications for a learning progression within a high school chemistry course.

Data for this study came from a project called Connected Chemistry as Formative Assessment (CCFA) funded by the National Science Foundation (please refer to Liu et at., 2012; Waight et at., 2013 for more information). CCFA integrates formative assessment, computer models and modeling, and learning progression. The development of assessment items followed the construct modeling approach (Wilson, 2005). Progress variables on matter, energy, and models were conceptualized to consist of three levels. Table 1 presents the different levels within each of the progress variables associated with these core concepts.

Students' behaviors on each of the progress variables were solicited by computer model-based assessment questions. That is, students first interacted with a computer model, and then answered questions related to specific aspects of the model. Computer model-based assessments on matter, energy, and models were developed for the following nine commonly taught topics in a high school chemistry course: Atomic Structure, States of Matter, Solutions, Gases, Stoichiometry, Chemical Bonding, Chemical Equilibrium, Redox, and Acids and Bases. The topic of periodic properties was dropped due to lack of items fitting the Rasch model (described below). Each assessment included at least one Flash animation, one NetLogo model, and a set of assessment questions. Assessment questions were in the format of Ordered Multiple-Choice (OMC) (Briggs et al., 2006), with answer choices matching different levels of a progress variable (all computer models are available at http://wings.buffalo.edu/research/ConnectedChemistry/).

The following sample questions are illustrative of the items used when students worked with computer models for the topic of States of Matter:

Sample Matter Question: Which of the following statements best describes the commonalities or differences among three phases of the substance in the container?

a. The substance in different phases has the same composition. (level 1)
b. The chemical bonds of the substance in different phases are the same. (level 2)
c. The intermolecular forces in different phases are different. (level 3)

a. They represent different amounts of energy. (level 1)
b. They represent different amounts of transferred energy. (level 2)
c. They represent different processes of conserving energy. (level 3)

Sample Model Question: Which of the following statements best describes the white substance in the container?

a. ice at the room temperature. (level 1)
b. a solid substance at the room temperature. (level 2)
c. any solid substance of my choice. (level 3)

Students' responses to the nine computer model-based assessments were analyzed using the Rasch model. Rasch measurement is based on an equation originally developed by a Danish mathematician named George Rasch (Rasch, 1960/1980). We used the partial credit Rasch model (Masters, 1982), which takes the following form:

where P_nik is the probability for student n with an ability Bn responding successfully at level k instead of level k - 1 to item i; D_ik is the difficulty of level k of item i.

Although matter and energy are conceptually distinct concepts in chemistry, previous Rasch analysis indicated that they are psychometrically highly correlated and can be treated as one dimension. A combined matter and energy dimension is consistent with our emphasis on systematic thinking about a chemical system. The conceptual framework for next generation science standards also includes "energy and matter - flows, cycles, and conservation" as one of the crosscutting concepts in science and engineering. Multi-dimensional Rasch models, specifically the Multidimensional Random Coefficients Multinomial Logit (MRCML) model (Adams et al., 1997), together with Con-Quest computer software (Wu et al., 1997) were used for the above analysis.

Results

In Rasch analysis, item difficulty is measured in a logarithm unit defined as the natural logarithm of the odds of answering a question at one level over the probability for answering a question at a lower level. This unit is also called logit. The more positive the logit value for a question, the higher its difficulty. Logit measures are interval, and can be transformed linearly to a scale with any mean and standard deviation. Table 2 presents descriptive statistics of item difficulties related to different chemistry topics.

Data in Table 2 indicates that, for the matter-energy construct, items related to the topic of Gases had the highest mean difficulty, while items in the Redox category had the lowest mean difficulty. We can also note that difficulties among items within the same topic also varied, with the greatest variation associated with questions about bonding (sd = 0.50). For the model construct, items in the Bonding category had the highest mean difficulty, while items about states of matter had the lowest mean difficulty. Similarly, there was a variation in difficulty among items within a given topic, with the greatest variation associated with questions about chemical bonding (sd = 0.43).

A two-way analysis of variance on item difficulties using topics (e.g., Gases, Bonding) and construct (e.g., Matter-Energy, Models) as independent variables found that there was no topic by construct interaction effect. Differences in item difficulties were mainly linked to the nature of the topic, and not to the nature of the construct. Table 3 presents the quantitative data that supports these results.

Post-hoc comparison using a liberal method, i.e., Fisher's Least Significant Difference (LSD), indicated the following statistically significant differences in the mean difficulty of items associated with different pairs of topics:

— Acids and Bases when compared with Gases, Redox, or Stoichiometry;

— Atomic Structure when compared with Redox or Solutions;

— Bonding when compared with Gases, Redox, or Solutions;

— Equilibrium when compared with Gases or Stoichiometry;

— Gases when compared with Acids and Bases, Bonding, Equilibrium, Redox, Solutions, or States of Matter;

— Redox when compared with Acids and Bases, Atomic Structure, Bonding, Gases, States of Matter, or Stoichiometry;

— Solutions when compared with Atomic Structure, Bonding, Gases, States of Matter, or Stoichiometry;

]]> — States of Matter when compared with Gases, Redox, and Solutions;

— Stoichiometry when compared with Acids and Bases, Equilibrium, Redox, and Solutions.

Table 4 presents the quantitative data associated with these different pair-wise comparisons.

Figure 1 plots the marginal item difficulty means for all items within a given topic. Based on this data, topics can be arranged into the following four clusters from most difficult to least difficult: (a) Gases, (b) Atomic Structure and Stoichiometry, (c) Acids and Bases, Bonding, and States of Matter, and (d) Equilibrium, Solutions, and Redox.

Discussion

Learning progressions are "descriptions of the successively more sophisticated ways of thinking about a topic that can follow one another as children learn about and investigate a topic over a broad span of time" (NRC, 2007, p. 219). If we assume that differences in item difficulty correspond to differences in levels of sophistication in the ways of thinking about various chemistry topics, then the results presented in the previous section may suggest a possible learning progression. This progression would move from understanding chemical equilibrium, solutions, and redox reactions to understanding acids and bases, chemical bonding, and states of matter, to understanding atomic structure and stoichi-ometry, to finally understanding gases.

To make sense of the suggested learning progression for high school chemistry, we need to think about potential reasons for such a progression. Research in science education suggests that student development of sophisticated ideas takes time and may involve many intermediate ways of thinking, some of them naive and based on misconceptions. Thus, a learning progression is to make learning "gradual" or "progressive" in the sense that later learning is built upon previous learning. This is what a spiral curriculum intends to achieve (Bruner, 1960). Specifically for chemistry teaching, Taber (2012) has emphasized the importance of building student understanding over time in a spiral format.

Previous research has suggested many different ways to foster learning. For example, Ausubel's cognitive learning theory gives primary importance to developing new cognitive structures in learning from prior knowledge (Ausubel, 1968). Specifically, Ausubel suggested that an important mechanism for developing new cognitive structures is through integrative reconciliation. Contrary to progressive differentiation, integrative reconciliation involves building new learning based on commonalities or connections with existing knowledge, allowing for the introduction of more inclusive or broader concepts. Joseph Novak has opera-tionalized integrative reconciliation and progressive differentiation into concept mapping (Novak & Gowin, 1984). More recently, various scholars have proposed the notion of curriculum coherence (AAAS, 2001; Schmidt et at., 2005) as an approach to support learning. A coherent curriculum is one that over time follows a sequence of topics and performances consistent with the logical and, if appropriate, hierarchical nature of the disciplinary content from which the subject-matter derives (Schmidt et at., 2005). For high school chemistry, the logic and hierarchy of reasoning may be demonstrated in the following ways: qualitative to quantitative, descriptive to predictive, observational to experimental, macroscopic to microscopic, experimental to computer modeling, and simple to complex.

Based on the above possible mechanisms of learning progression, the sequence of topics that emerged from the analysis of item difficulty in our work may respond to the following logic. First, from primarily qualitative to quantitative, as the most difficult topics, such as Stoichiometry and Gases, involve quantifications of substances and mathematical relationships between and among substances, while other topics may be primarily studied in qualitative ways. Second, the elicited sequence of topics can also be considered as following a logic from descriptive to predictive, as Atomic Structure, Chemical Bonding and Kinetic Molecular Theory (KMT) are theoretical models used to explain chemical equilibrium, solutions processes, redox reactions, and acid-base reactions. Finally, the topical sequence can be considered as following a logic from simple to complex because studying phase changes and gas behavior requires an understanding of not only structure and properties of individual particles (atoms, ions, and molecules) but also of interactions among many particles — a system view.

The sequence of topics that emerged from our analysis is just one of many possible learning progressions a high school chemistry course may follow. Different learning progressions aim at developing different ideas foundational to chemistry. The sequence suggested in this study aims at developing student understanding about matter, energy and models; it is a promising way to conceptualize high school chemistry teaching and learning in today's digital age.

Acknowledgement

The materials reported in this paper are based upon work supported by the National Science Foundation under Grant No. DRL-0918295.

References

AAAS, American Association for the Advancement of Science, Designs for science titeracy, New York: Oxford University Press, 2001.         [ Links ]

Achieve, Inc., Next generation of science standards. Washington, D.C., 2013.         [ Links ]

Adams R. J., Wilson M. and Wang W., The multidimensional random coefficients multinomial logit model, Apptied Psychotogicat Measurement, 21, 1-23, 1997.         [ Links ]

Alonzo A. and Steedle J. T., Developing and assessing a force and motion learning progression, Science Education, 93(3), 389-421, 2009.         [ Links ]

Ardac D. and Akaygun S., Effectiveness of multimedia-based instruction that emphasizes molecular representations on students' understanding of chemical change, Journat of Research in Science Teaching, 41(4), 317-337, 2004.         [ Links ]

Ausubel D. P, Educationat Psychotogy: A Cognitive View, New York and Toronto: Holt, Rinehart and Winston, 1968.         [ Links ]

Briggs D. C., Alonzo A. C., Schwab C. and Wilson M., Diagnostic assessment with ordered multiple-choice items, Educationat Assessment, 11(1), 33-63, 2006.         [ Links ]

Bruner J. S., The Process of Education. London, Cambridge, MA: Harvard University Press, 1960/1977.         [ Links ]

Chang H.-Y., Quintana C. and Krajcik J., The impact of designing and evaluating molecular animations on how well middle school students understand the particulate nature of matter, Science Education, 94, 73-94, 2010.         [ Links ]

Dori Y. J. and Sasson I., Chemical understanding and graphing skills in an honors case-based computerized chemistry laboratory environment: The value of bidirectional visual and textual representations, Journat of Research in Science Teaching, 45(2), 219-250, 2008.         [ Links ]

Doymus K., Karacop A. and Simsek U., Effects of jigsaw and animation techniques on students' understanding of concepts and subjects in electrochemistry, Education Tech ResearchDev, 58, 671-691, 2010.         [ Links ]

Duncan R. Rogat A. and Yarden A., A learning progression for deepening students' understandings of modern genetics across the 5th-10th grades, Journat of Research in Science Teaching, 46(6), 655-674, 2009.         [ Links ]

Feynman R. P., Six easy pieces: Essentiats of physics exptained by its most brittiant teacher. Cambridge, MA: Perseus Books, 1963.         [ Links ]

Frailich M., Kesner M. and Hofstein A., Enhancing students' understanding of the concept of chemical bonding by using activities provided on an interactive website, Journat of Research in Science Teaching, 46(3), 289-310, 2009.         [ Links ]

Jin H. and Anderson C. W., A learning progression for energy in socio-ecological systems, Journat of Research in Science Teaching, 49(9), 1149-1180, 2012.         [ Links ]

Kind V., Beyond appearances: Students' misconceptions about basic chemicat ideas. School of Education, University of Durham, 2004.         [ Links ]

Lewis E. L. and Linn M. C., Heat energy and temperature concepts of adolescents, adults and experts: Implications for curricular improvements, Journat of Research in Science Teaching, 31(6), 657-677, 1994.         [ Links ]

Liu X., Waight N., Roberto G., G., Smith E. and Park M., Developing computer model-based assessment of chemical reasoning: A feasibility study, Journat of Computers in Mathematics and Science Teaching, 31(3), 259-281, 2012.         [ Links ]

Masters G. N., A Rasch Model for Partial Credit Scoring, Psy-chometrika, 47, 149-174, 1982.         [ Links ]

NRC, National Research Council, Taking science to schoot: Learning and teaching science in Grades K-8, Washington, D.C.: The National Academic Press, 2007.         [ Links ]

NRC, National Research Council, A framework for K-12 science education: Practices, crosscutting concepts, and core Ideas, Washington, DC: National Academy of Sciences, 2012.         [ Links ]

Novak J. and Gowin B., Learning How to Learn, Cambridge: Cambridge University Press, 1984.         [ Links ]

Novick S. and Nussbaum J., Pupils' understanding of the particulate nature of matter: A cross age study, Science Education, 65(2) 187-196, 1981.         [ Links ]

Ozmen H., Effect of animation enhanced conceptual change texts on 6th grade students' understanding of the partic-ulate nature of matter and transformation during phase changes, Computer & Education, 57, 1114-1126, 2011.         [ Links ]

Plass J. L., Milne C., Homer B. D., Schwartz R. N., Hayward E. O. and Jordan T., Investigating the effectiveness of computer simulations for chemistry learning, Journat of Research in Science Teaching, 49(3), 394-419, 2012.         [ Links ]

Rasch G., Probabitistic modets for some intettigence and attainment tests, Chicago: University of Chicago Press / Danmarks Paedogogiske Institut, 1960/1980.         [ Links ]

Rutten N., van Joolingen W. R. and van der Veen J. T., The learning effects of computer simulations in science education, Computers & Education, 58, 136-153, 2012.         [ Links ]

Schmidt W. H., Wang H. and McKnight C. C., Curriculum coherence: An examination of US mathematics and science content standards from an international perspective, Journat of Curricutum Studies, 37(5), 525-559, 2005.         [ Links ]

Scalise K., Timms M., Moorjani A., Clark L. and Holtermann K., Student learning in science simulations: Design features that promote learning gains, Journat of Research in Science Teaching, 48(9), 1050-1078, 2011.         [ Links ]

Shen J., Jing L., Chang H. and Namdar B., Technology-enhanced, modeling-based instruction (TMBI) in science education. In: Spector J. M., Merrill M. D., Elen J. and Bishop M. J. (eds.), Handbook of research on educationat communication and technotogy (4th ed.) (Chap. 46), New York: Springer, 2011.         [ Links ]

Smetana L. K. and Bell R. L., Computer simulations to support science instruction and learning: A critical review of the literature, Internationat Journat of Science Education, 34(9), 1337-1370, 2012.         [ Links ]

Snir J., Smith C. L. and Raz G., Linking phenomena with competing underlying models: A software tool for introducing students to the particulate model, Science Education, 87(6), 794-830, 2003.         [ Links ]

Songer N. B., Kelcey B. and Gotwals A. W., How and when does complex reasoning occur? Empirically driven development of a learning progression focused on complex reasoning about biodiversity, Journat of Research in Science Teaching, 46(6), 610-631, 2009.         [ Links ]

Stieff, M., Connected chemistry - A novel modeling environment for the chemistry Classroom, Journat of Chemicat Education, 82(3), 489-493, 2004.         [ Links ]

Taber, K. S. (ed.). Teaching Secondary Chemistry (2^nd ed.). London: Hodder Education 2012.         [ Links ]

Talib O., Matthews R. and Secombe M., Computer-animated instruction and students' conceptual change in electrochemistry: Preliminary qualitative analysis, Internationat Education Journat, 5(5), 29-42, 2005.         [ Links ]

Waight N., Liu X., Gregorius R. M., Smith E. and Park M. (2013), Teacher conceptions and approaches associated with an immersive instructional implementation of computer-based models and assessment in secondary chemistry classrooms, Internationat Journat of Science Education. DOI: 10.1080/09500693.2013.787506, 2013.         [ Links ]

Westbrook S. L. and Marek E. D., A cross-age study of student understanding of the concept of diffusion, Journat of Research in Science Teaching, 28, 649-660, 1991.         [ Links ]

Wilson M., Constructing measures: An item response modeting approach. Mahwah, N. J., Lawrence Erlbaum Associates Publishers, 2005.         [ Links ]

Wu M. L., Adams R. J. and Wilson M., Conquest: Generatized item response modetting software - Manuat. Melbourne: Australian Council for Educational Research, 1997.         [ Links ]