Using Learning Progressions to Inform Curriculum, Instruction and
Assessment Design
Namsoo Shin1, Shawn Y. Stevens1, César Delgado1, Joseph Krajcik1 & James Pellegrino3. Paper submitted to the annual meeting of the National Association for Research in Science Teaching,
New Orleans, Louisiana. Abstract
Introducing an emerging science discipline such as nanoscience into the grade 7-12 classroom creates many challenges. As emerging sciences tend to be interdisciplinary in nature, changes in instruction, assessment and curriculum development are required. In particular, this interdisciplinary nature requires that a greater priority must be placed on fostering connections between ideas from many different scientific topics. In this study we report on efforts to assess students’ conceptual understanding within the nature of matter. In so doing, we have characterized connections between concepts and ideas that students make successfully as well as those with which they have difficulty. We have used this data to create a multi-dimensional progression of ideas that we used to validate a potential learning progression for the nature of matter. The critical conceptual links can be translated into both classroom and large-scale assessment strategies/materials. In addition, the work informs both the curricular structure and instruction by providing insight into how students connect ideas from other science disciplines with a core scientific concept.
1 University of Michigan 3 University of Illinois-Chicago
Using Learning Progressions to Inform Curriculum, Instruction and
Assessment Design
Rationale, Research Problem, and Research Questions
One of the major challenges to bringing nanoscience and nanotechnology, as well as most emerging science, into the classroom is their interdisciplinary nature. For example, nanoscience and nanotechnology incorporate chemistry, physics, biology and engineering (Roco, 2001). This interdisciplinary nature requires students to be able to integrate ideas from several topic areas in order to explain most nanoscale phenomena. However, students often have difficulty making connections between different scientific concepts and ideas. For instance, students often have difficulty applying knowledge from one part of the particulate model of matter to another (Renström, Andersson, & Marton,1990). In addition, students often use models of different levels to describe different concepts related to the structure and behavior of matter. (Harrison & Treagust, 2000).
The integration of knowledge is made more difficult by typical large-scale and classroom assessments ostensibly based on the standards. Such assessments commonly focus on targeted, isolated topics that do not require students to connect currently taught concepts with concepts from other science areas that were previously learned (NRC, 2005; Pellegrino, Chudowsky, & Glaser, 2001). Instead, these assessments encourage teachers to focus on isolated bodies of knowledge that ultimately results in compartmentalized application of science concepts. As a result, the traditional curriculum often compartmentalizes the various aspects of the study of matter (e.g. structure of matter, conservation of matter, chemical reactions, phase changes). The authors of documents such as Benchmarks for Science Literacy (AAAS, 1993) and the National Science Education Standards (NRC, 1996) suggested connections between key concepts among multiple disciplines in the sciences. However, these connections have not been borne out in most science curricula nor are they a part of typical assessment practices. Thus, in order to generate literacy in emerging sciences, school curricula must begin to emphasize not only the learning of individual topics, but also the connections between them and assessments must be developed to support such a curriculum.
This study describes work towards developing and validating the sequence and assumptions behind a learning progression. The processes of assessment that we use might ultimately be translated into both classroom and large-scale assessment strategies/materials. In addition, the work informs both the curricular structure and instruction by providing insight into how students connect ideas from other science disciplines with a core scientific concept. Thus, this approach might provide a method for identifying the connections that are required to obtain a deep conceptual understanding of an interdisciplinary field such as nanoscience.
A learning progression describes what it means to move towards more expert understanding in an area and gauges students’ increased competence related to a core concept or a scientific practice (Smith et al., 2004). They consist of a sequence of successively more complex ways of thinking about an idea that might reasonably follow one another in the process of students developing understanding about that idea. However, as we address interdisciplinary subject matter, we can no longer consider learning progressions in a linear fashion. Rather, learning progressions may be viewed as strategic sequencing that promotes both branching out and
forming connections between ideas related to a core scientific concept. We can better assess students’ conceptual understanding by designing items that assess the connections between related science topics and ideas.
In order to provide a conceptual explanation of most nanoscale phenomena, a deep and thorough understanding about the nature of matter is required. This includes the structure and properties of matter and how it behaves under a variety of conditions. To make progress on how students’ understanding about the nature of matter develops, we need to construct a learning progression. Based upon the AAAS Atlas for Science Literacy, we created a hypothetical progression of ideas to describe the nature of matter. However, this progression is only theoretical as it is does not represent the curriculum that is followed in the classroom nor do we have much empirical evidence to support this progression. Therefore, although the suggested progression is consistent with national standards, it requires collection of empirical evidence to verify a possible learning progression that can be applied to describe the learning of our target populations. This study is directed towards assessing and characterizing how and when students make the connections between the ideas as they progress towards a deep conceptual understanding. It affords practical guideline to develop test items for measuring students’ learning progression. As we study student learning of the nature of matter, we explore these questions:
1. How do students’ ideas about concepts regarding the nature of matter develop over time? 2. How can we assess the requisite connections between concepts that traditionally have
been compartmentalized in instruction?
Study Design and Methods
AAAS Project 2061 created the Benchmarks for Science Literacy based on the goals stated for adult literacy in Science for All Americans (AAAS, 1989, 1993). Each benchmark describes an idea or skill that represents a defined goal for student learning. The next step AAAS took was to propose how students’ understanding of these ideas and skills might grow over time. To this end, a series of strand maps were created. Each map focuses on a single topic that is important for literacy in science, mathematics and technology. A set of relevant benchmarks were collected, then connected based on a logical progression within the subject matter and when possible, educational research about student learning. A series of these maps was incorporated into the Atlas of Science Literacy, which contains a set of four maps that describe the structure of matter: “Atoms and Molecules”, “States of Matter”, “Chemical Reactions” and “Conservation of
Matter.” We used these four strand maps and created two additional strand maps on related topics that we deem critical for understanding nanoscience, “Properties of Matter” and “Forces and Interactions”. New learning goals were added for concepts where the standards were found to be lacking. The six maps were then merged to form an interconnected set of learning goals that predict a progression of concepts and the necessary connections between ideas that students’ must be able to make in order to have a deep conceptual understanding of the nature of matter In order to test aspects of this hypothetical progression, we have conducted interviews with middle school and high school chemistry and pre-chemistry students to measure their conceptual understanding of the structure, properties and behavior of matter. To complete the progression, we will interview undergraduate science and non-science majors and experts. However, here we report only on middle school and high school students’ conceptual understanding of the nature of matter.
Instruments- In order to test the validity of this progression of ideas, we interviewed individual students to evaluate their conceptual understanding of the nature of matter. The 20-30 minute interview was designed to directly probe the students’ conceptual understanding of several topics within the nature of matter:
Structure of matter- (Students are handed a small sheet of metal) “Imagine we have an instrument that lets us zoom in and see what this sheet of metal is made of-- What do you think the surface would look like?”
Atomic size- “How many particles thick is this (~0.5mm thick) sheet of metal?”
Phase changes- “Now we’re going to melt the metal. Describe what happens as it melts.” Properties of matter- “How would you describe the differences between powdered and
granulated sugar?”
Electromagnetic forces- “What is keeping the particles of the solid (or liquid) together? (A demonstration involving pouring granulated and powdered sugar from a black plastic surface is performed. A significant amount of the powdered sticks to the plastic or clumps due to electric forces.) What causes the difference in the behavior of the two kinds of sugar?”
Atomic structure- “Please draw an atom showing what it’s made of, and explain it to me.” Nature of chemical reactions- “Can you explain why chlorine and sodium chloride form
stable substances? How do they differ?”
Population- Students were selected to fill out a 3-D matrix with dimensions of educational level (middle school, high school pre-chemistry and chemistry), ability and gender. The population consisted of Midwestern students from middle and high schools in a diverse, urban community where approximately 50% are considered to be economically disadvantaged, and predominantly Caucasian middle-class suburban and rural communities.
Data Analysis- Data analysis for this study incorporated both quantitative and qualitative methodology. A hierarchical coding scheme was developed based on previous research studies (Renström et al., 1990; Eskilsson, O. & Hellden 2003, Nicoll, G. 2003).
Table 1. Coding scheme for atomic structure- How do students define the structure of atom?
Code Description Examples
0 Does not know what an atom is. 1 An atom consists of a single, uniform
sphere.
Student draws a circle to represent atom. (PC apr06, C jun06)
2
An atom consists of some other entities.
Student draws a cluster of circles, but cannot identify them. (PC apr06) Student draws a ring with two spheres, an electron and a proton. (PC apr06)
3
An atom consists of electrons, protons and neutrons; protons are positively charged, electrons negatively charged and neutrons are neutral.
Components may not be in proper place (ex. Protons and electrons in orbitals around the nucleus) (PC apr06, C jun06)
4
An atom consists of electrons, protons and neutrons; the relative abundance of each of these is important.
Student correctly discusses the information on the periodic table. (PC apr06, C jun06) PC = pre-chemistry; C = chemistry
The levels were based initially on previous research studies of students’ understanding of concepts related to the structure of matter and then refined based upon student responses
obtained from a small portion of the data. The resulting data are then analyzed quantitatively to identify any correlations between topics that are thoroughly understood by students (level 4) to develop a logical learning progression.
Results
Students can hold diverse sets of beliefs within their models of the nature of matter. For instance, when describing the structure of atoms, students provided an array of descriptions of atomic structure. Even high-ability students in their second semester of chemistry often consider atoms as just solid spheres, having no concept of the components within them (protons, neutrons and electrons) or their purpose (PC SS apr06, C SS jun06).
As they describe phenomena involving different aspects of the behavior of matter, students’ models are often inconsistent. For example, Gilbert a mid/low-achieving HS chemistry student states that the edge of a piece of metal feels smooth because the atoms are too small to feel. Then when asked how many atoms stacked up would it take to fill about 0.5 mm he answered ‘one’, which illustrates the inconsistency in his knowledge about the size of atoms (G KH feb06). In addition, while most high school chemistry students will create drawings of a solid containing an ordered array of atoms, most do not believe that the arrangement of atoms is important; they fail to make the connection that the properties of a substance are derived from the arrangement of atoms (SS apr06, jun06).
In another example Andrew (all names are pseudonyms), an 11th
grade chemistry student who obtained one of the highest grades in his class, used the octet model to explain the difference between ionic and covalent bonding (sodium chloride and chlorine respectively). He described the single electron in the outer shell of the sodium atom to be transferred to the chlorine in order to complete the octet, while in the chlorine two electrons are shared between the atoms to form the bond (apparent Level 4). However when probed further, it became clear that he did not know what the symbols Na and Cl represent. He referred to them only as elements, never atoms. In fact, in the earlier part of the interview regarding atomic structure, his atom was just a sphere; in his model, atoms did not contain any components (Level 1) (A SS jun06). Thus, although he could provide a seemingly acceptable answer about bonding and how electrons govern the interactions, he did not realize that these interactions are between atoms and that atoms contain electrons. Rebecca, also a high-performing high school chemistry student, used the particulate model of matter and kinetic theory to explain the process of melting (Level 4). However, matter was not conserved in her model. She said that some atoms “magically disappear” when a solid melts into a liquid (RW KH feb06). In addition, she also confuses atoms and cells, believing that atoms are comprised of “nucleuses and cytoplasm”, which is a somewhat common
misconception for both middle school and high school students. Therefore, academic ability and achievement cannot be considered predictors of conceptual understanding, as some very high achieving students lack mastery of basic concepts. Indeed, this suggests that traditional assessment strategies are inadequate for measuring students’ conceptual understanding. In this paper we presented how we developed and validated a potential learning progression to use for the development of assessing student learning in nanoscience, specifically in the topic
areas involving the structure, properties and behavior of matter. Our work is a case study that illustrates the design of assessments based on a learning progression through research and development cycles. Our principles include (a) elaborating standards and Benchmarks to create a hypothetical learning progression, (b) collecting various data to validate the learning
progression, (c) revising the learning progression based on the collected data, and (d) developing items for assessing student’s placement on the learning progression. The difficulties that we have identified in students’ conceptual understanding of the nature of matter are not necessarily because the material is developmentally inappropriate. As more learning progressions are developed and validated, they can be used to guide teaching and curriculum development. If students were presented with an exemplary curriculum that helped to foster their understanding and facilitate the connections they must make between ideas to have a deep understanding of a science topic, the problems we observed might not occur.
Contribution to the Field and Interest to NARST Members
Bringing an emerging science discipline such as nanoscience into the grade 7-12 classroom creates many challenges. As emerging sciences tend to be interdisciplinary in nature, changes in instruction, assessment and curriculum development are required. In particular, this
interdisciplinary nature requires that a greater priority must be placed on fostering connections between ideas from many different scientific topics. The growth of these connections must be better characterized in the context of learning progressions such as the progression that we developed for the nature of matter. As such, these learning progressions can be used to inform not only instruction, but also the development of curriculum materials and assessment.
References
American Association for the Advancement of Science (1993). Benchmarks for Science Literacy. New York: Oxford University Press.
American Association for the Advancement of Science (1989). Science for all Americans. New York: Oxford University Press.
Harrison, A.G. & Treagust, D.F., (2000). Learning about atoms, molecules and chemical bonds: A case-study of multiple model use in grade-11 chemistry. Science Education, 84, 352-381. National Research Council (2005). Systems for State Science Assessments. In M.R. Wilson and
M.W. Bertenthal, (Eds.), Committee on Test Design for K-12 Science Achievement. Washington, DC: The National Academies Press.
National Research Council (2004). Implications of research on children’s learning for
assessment: Matter and atomic molecular theory. In C. Smith, M. Wiser, C.W. Anderson, J. Krajcik, & B. Coppola, (Eds), Committee on Test Design for K-12 Science Achievement. Washington, DC: The National Academy Press.
National Research Council (1996). National Science Educational Standards. Washington, DC: National Academy Press.
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academy Press. Renström, L., Andersson, B., & Marton, F. (1990). Students’ conception of matter. Journal of
Educational Psychology, 82, 555-569.
Roco, M.C. (2001). From vision to the implementation of the U.S. National Nanotechnology Initiative. Journal of Nanoparticle Research, 3, 5-11.
