This study addresses two research questions. The first question is does low- achieving fourth graders' participation in tutoring based on five key instructional practices for students with mathematics disabilities increase their understanding of
multiplication and division? The five key instructional practices are real-world problems, strategy instruction, CRA sequence, peer-assisted learning, and progress monitoring. The second research question is what are the patterns of deficits low achieving fourth graders have in relation to the six indicators of MLD? These six indicators are fluency with basic facts, judgment of magnitude, determination of unreasonable results, use of multiple representations, procedural memory, and visual-spatial skills. This chapter provides an overview of the methodology used in this study.
Context of Study
For the last several years, the state of Idaho has focused resources on a statewide effort to improve mathematics instruction with the passage of the Idaho Math Initiative in 2007. This initiative has required every teacher of mathematics of students in grades Kindergarten through 12 to take a state-sponsored class called Mathematical Thinking for Instruction (MTI). This class has encouraged teachers in the state to shift toward
mathematics instruction that develops conceptual understanding and embeds skill development in meaningful contexts. The course is based on an instructional framework for Core level instruction at all grade levels. This instructional framework was developed by Brendefur (2007, 2008) and is referred to as the Five Big Ideas of Teaching
Mathematics. The Five Big Ideas helps teachers focus on these five key instructional practices for general education mathematics instruction to build mathematical
understanding. The five key practices are as follows: 1. Focus on the structure of mathematics 2. Address misconceptions
3. Take students' ideas seriously 4. Press students conceptually
5. Encourage multiple representations
The implementation of this instructional framework across the state and in school- wide improvement efforts has led to greater student achievement as measured by the Idaho Standards for Achievement and project evaluation data (Brendefur, Pittman, & Thiede, in review; Brendefur, Thiede, Strother, Jesse, & Sutton, in review).
Although the MTI professional development efforts have shown success in improving general education mathematics instruction, there is still much work to be accomplished in Idaho to improve mathematics instruction at the Tier 2 and Tier 1 levels in an RTI model of intervention. The framework developed for this study is designed to complement the framework for Core Instruction being taught in the MTI classes while
still keeping in mind the unique needs of students with MLD and the need for diagnostic assessments.
Setting
This study took place in an elementary school in a rural school district in southwestern Idaho. The school has grades kindergarten through fifth grade and was chosen based on its diverse student population and convenience of location for the researcher. Based on the 2010 - 2011 School Report Card, the school has a student population of 510. The ethnicity of the students is comprised of 46.47% Caucasian, 42.35% Hispanic, 11.18% other ethnicities. Students who qualify for free and reduced lunch comprise 79.22% of the student population. English Language Learners are 19.80% of the student population, and 9.41% participate in special education. During the 2011- 2012 school year when the study was conducted, the school was in Year 5 of school improvement for reading but had 85% of the students meeting the state reading benchmark in 2010 and had met 18 of 20 goals. Missed reading goals included proficiency levels for Hispanic and Economically Disadvantaged students. In
mathematics, the school was in year 1 of school improvement due to missing goals in the past. They met all 20 goals in this subject in 2010 and had 87% of the students reach the state math benchmark.
Student Sample
The sample population for this study was all students in one fourth grade at this elementary school. From this classroom, four students were selected for a treatment
group based on multiple measures. A matrix was created that showed student scores on the following data:
Pre-test using researcher-created test on multiplication and division concepts
Timed multiplication fact assessment. Bottom 35th percentile were identified.
First trimester math grades
Teacher rating on scale of 1 to 4 indicating mathematics performance in class in comparison to grade level peers.
Idaho Standards for Achievement (ISAT) mathematics score Spring 2011. Bottom 35th percentile of the class were identified.
ISAT proficiency level: basic, below basic, proficient, or advanced
Noting those scores which were in the bottom 35th percentile of the population is consistent with the recommendations of leading researchers in this field (Mazzocco, 2005; Geary, 2005; Gersten et al., 2005). Based on this collection of data from multiple measures, the teacher and researcher selected four students who were performing below their peers in mathematics to participate in the tutoring group. Two of the selected students were female and two were male. Three of the four students were Hispanic and one was Caucasian. Three had been identified by the school as limited English proficient (LEP), and two were receiving special education services for speech articulation.
Parent Permission
Because this study involved children, all guidelines for research on vulnerable populations were closely followed. Parents of all students in the class needed to provide consent for their child's data to be used in this study. The parents of the students in the
tutoring group needed to be fully informed of the study and the intervention program their child was participating in. A parent permission slip was sent home with all of the students in the class. All students were asked to return the permission slip showing either consent or no consent. Students were provided a small prize for returning the permission slip. Only students whose parents provided consent were included in the population sample. Students whose parents did not provide consent still took the same assessments as their classmates, but their data was not used in the study.
Once the students for the tutoring group were selected by the teacher and
researcher, the researcher or an interpreter provided by the school called each parent and explained the study and the instruction their child would be receiving. Because some of the parents did not speak English, an employee of the school who spoke Spanish called instead of the researcher. After the phone conversation, a written permission slip was sent home to the parent. The parents of all four students provided consent. Once parents had signed the permission slip and returned it to the teacher, the researcher met with the student to discuss the study and obtain student assent. A copy of the parent permission slips, scripts for phone calls and conversations to obtain student assent are included in Appendix A. No student identifying information will be used in any presentation of this study and all identifying information will be kept confidential by the researcher and school staff.
Measurement Instrument
Research on valid and reliable measures of mathematical understanding and abilities beyond basic computation is in its early stages (Geary, 2004, 2005; Feifer & De
Fina, 2005; Gersten et al., 2005; Mazzocco, 2005; Chiappe, 2005). The lack of available measures of mathematical abilities limits the quality of research that can be conducted on the effectiveness of intervention programs. This study addresses this limitation by
utilizing a multiple measures approach. Achievement was measured with multiple sources of data, which included a researcher-created test on the topics taught, a basic mathematics fact assessment, the standardized state mathematics assessment, teacher and researcher observations and ratings, grades, work sample, and informal teacher and researcher-created assessments. While none of these measures is considered statistically valid or reliable, the collection of these multiple sources of data provided information on the mathematics achievement of the sample population as well as changes in their
understanding of multiplication and division. Copies of the assessments used in this study are included with the intervention curriculum in Appendix B.
Treatment
Students in the treatment group received 27 sessions of mathematics intervention based on the instructional framework created by the researcher. Students received
supplemental instruction on two core topics for the fourth grade mathematics curriculum. The group met three or four times a week for 45 minute sessions for a total of 27
sessions. The intervention took place at a time arranged with the school that did not remove students from their grade level mathematics instruction. The students participated in two instructional units aligned to grade level targets from the Common Core
Mathematics Standards, which Idaho has adopted. The first unit was called Number Patterns. It included 11 sessions focused on patterns with numbers, skip counting, and
basic number combinations with addition, subtraction, and multiplication. Students were introduced to the concept of using an iconic model to represent numbers. Examples of such models included a picture, open number line, arrow math, tree diagram, an array, and a set model. These models were also used in the second unit on multiplication. The second unit was 16 sessions focused on the topic of multiplication. It provided students with contextual problem solving experiences related to multiplication. The curriculum for each unit and lesson plans for tutoring sessions are included in Appendix B. Informal formative assessments were utilized throughout the intervention period to monitor progress and mastery of objectives related to lessons.
Instructional activities for the intervention period were based on the instructional framework created from the comprehensive review of literature supporting this study. The key instructional practices used for the intervention were use of real-world problems, strategy instruction, using a CRA sequence, peer-assisted learning, and progress
monitoring. Throughout the intervention period, the researcher recorded anecdotal observations and made notes on work samples and assessments, which demonstrated strength or weakness in the areas identified as a possible indicator of an MLD. These areas were fluency with basic facts, judgment of magnitude, determination of
unreasonable results, use of multiple representations, procedural memory, and visual- spatial skills. Learning activities emphasizing these skills were strategically integrated into lessons. All student work was collected in a portfolio for each student in the tutoring group. Work samples and anecdotal observations were then coded based on the indicator areas. At the end of the treatment period, the patterns of strength and weaknesses for each
child were summarized. Scores on the post-test were compared to pre-test scores to determine whether the students in the tutoring group had made changes in their
understanding of multiplication and division. Based on the post-test, the qualitative data and the researcher's experiences with the child, a progress report for each student was prepared by the researcher and provided to the teacher, principal, and parent of each student. Copies of these progress reports are included in Appendix E.
Control Group
All of the other students in the fourth grade class acted as a control group. These students participated in the same mathematics lessons provided by the classroom teacher as the tutoring group. The fourth grade at this school has a mathematics intervention period for all students. The rest of the class participated in a 25 minute math intervention period four days a week during the time of the tutoring for this study. For these classes, the entire grade level was ability grouped and teachers worked with students on
reteaching or enriching mathematical skills. During the treatment period, students in the tutoring group did not participate in this intervention program but instead worked with the researcher for an extended intervention period of 45 minutes. At the end of the treatment period, all students in the class took the same post assessments as the students in the tutoring group on the same schedule. The assessment data from the tutoring group was compared with the assessment data from the rest of the class.
Data Analysis
This study used a mixed-methods design to evaluate the effectiveness of the instructional approach used for the treatment group. Statistical analysis and a scatter plot was used to determine whether the students in the treatment moved closer to the group median as compared to the assessment given before the intervention. This analysis helped determine whether the treatment intervention was effective in closing the achievement gap between the low performing students and their peers.
The researcher created a pre-test and post-test to assess changes in understanding on multiplication and division. Copies of these assessments as well as their theoretical constructs are included in Appendix B. The students' ability to solve open ended contextual problems involving multiplication and division and the use of models to represent both equations and contexts was evaluated. Changes from pre-test to post-test for the treatment group were noted. Each subskill on the unit tests was evaluated using a four point rubric recommended by Van de Walle, Karp, and Bay-Williams (2010). Table 7 explains the rating for each level of the scoring.
Table 7. Four Point Rubric for Pre-Test and Post-Test Questions
Score 4 Got it
Excellent: Full Accomplishment
Evidence shows that the student essentially has the target concept or idea. Strategy and execution meet the content, processes and qualitative
demands of the task. Communication is judged by effectiveness, not length. May have minor errors.
Score 3 Got it
Proficient: Substantial Accomplishment
Evidence shows that the student essentially has the target concept or idea. Could work to full accomplishment with minimal feedback. Errors are minor, so teacher is confident that understanding is adequate to
Score 2 Not there yet
Marginal: Partial Accomplishment
Student shows evidence of major misunderstanding, incorrect concept or procedure, or failure to engage the task.
Part of the task is accomplished, but there is lack of evidence of understanding or evidence of not understanding. Direct input or further teaching is required.
Score 1 Not there yet
Unsatisfactory: Little Accomplishment
Student shows evidence of major misunderstanding, incorrect concept or procedure, or failure to engage the task.
The task is attempted and some mathematical effort is made. There may be fragments of accomplishment but little or no success. Note: From Van de Walle et al. (2010, p. 81)
Additionally, the work samples, informal assessments, and anecdotal notes collected for each student's portfolio provided qualitative data in a case study approach for each student. This data combined with the assessment data from the pre- and post- tests allowed the researcher to summarize each student's response to the intervention and make conclusions about patterns in strengths and weaknesses. The data for each student were coded and summarized based on the six indicators of an MLD. This qualitative data collection provided an opportunity to evaluate the effectiveness of organizing data about students according to the six indicators of an MLD listed on the framework.