https://doi.org/10.1177/0014402919857993 Exceptional Children 2019, Vol. 86(1) 77 –94 © The Author(s) 2019 DOI: 10.1177/0014402919857993 journals.sagepub.com/home/ecx Original Research
In the current climate of a globalized econ-omy, the demand for a highly skilled work-force has dramatically increased (National Science Board, 2018). Consequently, the notion of preparing all students, including those with mathematics difficulties (MD), for early mathematical learning has gained national prominence. The raised standards bar for mathematics teaching and learning is evidenced by the advent of the Common Core State Standards for Mathematics (CCSS-M; National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010). Under the CCSS-M, students at every grade level are expected to develop proficiency in all areas of mathematics, including length measure-ment and data analysis.
A robust understanding of measurement and data analysis is essential in the science,
technology, engineering, and mathematics (STEM) fields. Civil engineers, for example, use measurement to obtain precise estimates of mass and strain, whereas an epidemiologist will collect and analyze data to identify trends in health-related events, such as measles out-breaks (Paules, Marston, & Fauci, 2019). Academically, measurement and data analysis hold significant value. Students who acquire a robust understanding of early measurement and data analysis concepts are better positioned
1University of Texas at Austin 2University of Oregon 3Oregon Research Institute 4University of California–Berkeley
Corresponding Author:
Christian T. Doabler, Meadows Center for Preventing Educational Risk, University of Texas at Austin, 1 University Station, D5300, SZB 408B, Austin, TX 78712. E-mail: [email protected]
Efficacy of a First-Grade
Mathematics Intervention on
Measurement and Data Analysis
Christian T. Doabler
1, Ben Clarke
2, Derek Kosty
3,
Jessica E. Turtura
2, Allison R. Firestone
4, Keith Smolkowski
3,
Kathleen Jungjohann
2, Tasia L. Brafford
2, Nancy J. Nelson
2,
Marah Sutherland
2, Hank Fien
2, and Steven A. Maddox
1Abstract
Well-designed mathematics instruction focused on concepts and problem-solving skills associated with measurement and data analysis can build a foundational understanding for more advanced mathematics. This study investigated the efficacy of the Precision Mathematics Level 1 (PM-L1) intervention, a Tier 2 print- and technology-based mathematics intervention designed to increase first-grade students’ conceptual understanding and problem-solving skills around the areas of measurement and data analysis. Employing a randomized controlled trial, 96 first-grade students at risk for mathematics difficulties were randomly assigned within classrooms to either a treatment (PM-L1) or a control (business-as-usual) condition. A statistically significant positive effect was found on one of five outcome measures, with the other four showing positive but nonsignificant results. Results also suggested preliminary evidence of differential response based on students’ number sense and early literacy risk status. Implications for using mathematics interventions focused on measurement and data analysis to build comprehensive, multitiered service delivery models in mathematics are discussed.
to develop proficiency with later mathematics (Frye et al., 2013). In the early elementary grades (i.e., kindergarten through second grade), these areas of mathematics serve as optimal platforms for students to apply and strengthen their whole-number understand-ing, skills in statistical investigation, and application of scientific practices. For exam-ple, when first-grade students solve “com-pare” word problems centered on length measurement, they have opportunities to draw from their conceptual and procedural knowl-edge of (a) place value, (b) the underlying structures of word problem types, and (c) whole-number computations. The ability to solve problems involving early measurement and data analysis concepts is also essential for building a base to understand advanced math-ematics and complex statistical investigation, such as posing important research questions that can be answered through empirical research (Confrey et al., 2012; Frye et al., 2013; Shaughnessy, 2007).
Students who acquire a robust understanding of early measurement and data analysis concepts are better positioned to
develop proficiency with later mathematics.
Measurement and data analysis are integral areas of mathematics. Yet, few early mathe-matics interventions target areas beyond num-ber. As such, the current investigation focuses on testing the efficacy of a Tier 2 intervention designed to promote first-grade students’ pro-ficiency with concepts and problem-solving skills of measurement and data analysis.
U.S. Students’ Performance
With Measurement and Data
Analysis
Despite the important role that early measure-ment and data analysis play in students’ over-all mathematical learning, empirical evidence suggests that as students exit the lower ele-mentary grades, many continue to face deficits
in all areas of mathematics, including mea-surement and data analysis. And although the percentage of students reaching proficient lev-els in the areas of measurement and data anal-ysis has slightly increased over the past decade (National Center for Education Statistics [NCES], 2017), these increases in mathemati-cal performance are not uniform. Mounting evidence indicates that the mathematics achievement gap in students’ knowledge of measurement and data analysis is quite pro-nounced for students from subgroups with or at risk for MD.
For example, results from the 2017 National Assessment of Educational Progress in the areas of data analysis and measurement suggest that students eligible for free or reduced-price lunch programs scored approxi-mately 20 scale score points lower than stu-dents not eligible for such programs (NCES, 2017). In the same content areas, Hispanic, Black, and American Indian students scored between 22 and 33 points lower than their White peers. These disparities are also evident for English learners (ELs), who scored 28 points lower than their native English-speak-ing peers (NCES, 2017).
Why Focus on the Areas of
Measurement and Data Analysis?
A limited understanding of measurement and data analysis may hinder students’ opportuni-ties to strengthen their whole-number under-standing and, in turn, develop overall mathematical proficiency. As such, it is imper-ative that students with MD receive opportuni-ties to work with foundational concepts of measurement and data analysis. For example, one important measurement concept to target is length-unit iteration. An understanding of iterative measurement allows students to gain foundational knowledge of estimation strate-gies, fractions on a number line, and strategic use of rulers to determine length measurement (Parmar, Garrison, Clements, & Sarama, 2011). Extending the concept of length-unit iteration to rulers provides opportunities for students to learn about the correct alignment with a ruler and zero-point concepts. In
addition, knowledge of unit iteration provides a foundation for understanding angles and concepts of angle measurement in later grades (Parmar et al., 2011).
A limited understanding of measurement and data analysis may hinder students’ opportunities
to strengthen their whole-number understanding and, in turn, develop
overall mathematical proficiency. In terms of data analysis, students with MD require structured learning opportunities to develop early statistical understanding. Profi-ciency in early concepts and problem-solving skills of data analysis is often viewed as a pro-gression of mathematical learning (National Governors Association Center for Best Prac-tices & Council of Chief State School Officers, 2010). As students progress through different levels of statistical and probabilistic thinking, they form important connections with their current understanding of whole numbers and operations (Langrall & Mooney, 2011). For example, in the early grades, students can col-lect data (i.e., continuous and categorical) and represent their findings on different types of graphs. Then, they can use such data to solve various types of word problems, including put-together, take-apart, and comparison problems (e.g., calculating the difference between two length measurements).
Students with MD also need extended opportunities to learn and engage in the prac-tices of statistical investigation. The statistical investigation framework proposed by Confrey et al. (2012) highlights the importance of hav-ing students directly experience and work with the same methodology that practicing scien-tists use in their respective fields. The frame-work comprises a set of practices, including (a) formulating research questions, (b) collect-ing relevant data, (c) organizcollect-ing and represent-ing data on graphs and analyzrepresent-ing data to address the targeted research question, and (d) interpreting and sharing the results. When stu-dents with MD engage in these practices, they have opportunities to not only understand how
scientists work but also gain a deeper under-standing of statistical investigation.
Development of a Tier 2
Intervention Focused on
Measurement and Data Analysis
When mathematics instruction targeting the areas of measurement and data is effectively designed and delivered, it can serve as a vehi-cle for accelerating the learning of students with MD. Yet, there remains a paucity of empirical evidence on effective interventions for young students who struggle in these areas (Frye et al., 2013; Langrall & Mooney, 2011; Parmar et al., 2011). Even more concerning is the fact that reviews conducted by the National Center on Intensive Intervention (NCII; n.d.) and the What Works Clearinghouse (WWC; n.d.) suggest that no rigorous intervention studies have investigated these areas of math-ematics for students with MD in the early elementary grades.
When mathematics instruction targeting the areas of measurement and data is effectively designed and delivered, it can serve as a vehicle
for accelerating the learning of students with MD.
Recognizing this gap in the current knowl-edge base, our research team developed the Precision Mathematics Level 1 (PM-L1) pro-gram. PM-L1 is a Tier 2 mathematics inter-vention that is designed to teach first-grade students with MD the foundational concepts and problem-solving skills of measurement and data analysis (Doabler et al., 2015, 2018). Because the unique learning needs of stu-dents with MD require instruction that is more intensive and specific than that pro-vided to typically achieving students, PM-L1 incorporates empirically validated principles of explicit instruction to directly teach essen-tial mathematics concepts and skills that stu-dents would not otherwise acquire on their own (Gersten et al., 2009; Hughes, Morris, Therrien, & Benson, 2017). Such principles
include teacher modeling of targeted con-cepts and skills, scaffolded practice opportu-nities for students, and specific academic feedback. To develop mathematical profi-ciency with measurement and data analysis concepts, the PM-L1 intervention judiciously integrates print- and technology-based activi-ties. We contend this “hybrid” approach (Doabler et al., 2018) of interweaving tech-nology and conventional means of explicit mathematics instruction offers inherent ben-efits for students with MD. For example, the tablet-based platform of the PM-L1 interven-tion fosters a tactile quality and thus permits students with MD to become active problem solvers (e.g., collecting and graphing mea-surement data).
Purpose of the Study
The purpose of this study was to investigate the efficacy of the PM-L1 intervention for improving the mathematics achievement of first-grade students with MD. We also had strong interest in exploring student-level pre-dictors of differential response. A growing body of research has identified initial math-ematics achievement and language skills as risk factors of early MD and moderating variables of mathematics interventions (Clarke et al., 2019; Doabler et al., 2019; Powell, Cirino, & Malone, 2017). For exam-ple, some studies suggest that initial mathe-matics achievement is an important explanatory variable of response variation among students who receive empirically val-idated Tier 2 mathematics interventions (Clarke et al., 2019; Powell et al., 2017). Language skills, both oral and written, have also been found to underlie students’ mathe-matical development (Riccomini, Smith, Hughes, & Fries, 2015). For instance, when students engage in mathematical problem-solving activities, they are often required to read connected text, use general academic and mathematics-specific vocabulary, and convey their solution methods through writ-ten or verbal responses. Consequently, stu-dents who struggle with early literacy, such as understanding the alphabetic principle
(i.e., letter-sound correspondence), may experience challenges when solving mathe-matics problems given the complexities with the language of mathematics. As such, we examined whether students’ initial mathe-matics achievement and early literacy skills served as plausible explanatory variables of response variation among students who received the PM-L1 intervention. Two research questions were addressed.
1. What is the effect of the PM-L1 inter-vention on the mathematics achieve-ment of first-grade students with MD? 2. Is there evidence of differential
response to the PM-L1 intervention as a function of number proficiency and early literacy risk statuses?
Method
This study analyzed efficacy data collected during a federally funded design and develop-ment project (Doabler et al., 2015). The PM-L1 study, which was conducted in an Oregon school district during the 2017–2018 school year, employed a randomized con-trolled trial. Blocking on classrooms, 96 first-grade students were randomly assigned within first-grade classrooms to one of two condi-tions: (a) the PM-L1 intervention or (b) a con-trol (i.e., business-as-usual) condition. We elected to use this rigorous research design to establish the efficacy of the PM-L1 interven-tion and better posiinterven-tion it for more formal effi-cacy testing in future research (Institute of Education Sciences & National Science Foun-dation, 2013).
Participants
Schools. Five elementary schools from one Oregon school district participated in the cur-rent study. The participating school district had a student enrollment of approximately 18,000. Within the five schools, the propor-tion of students who were American Indian or Native Alaskan ranged from less than 1% to 2%; Asian, less than 1% to 2%; Black, less than 1% to 2%; Hispanic, 3% to 19%; Native
Hawaiian or Pacific Islander, less than 1% to 1%; White, 76% to 92%; and more than one race, 3% to 5%. Within these same schools, 7% to 14% of students received special educa-tion services, 1% to 12% were ELs, and 28% to 54% were eligible for free or reduced-price lunch.
Classrooms. Ten classrooms participated in the study. All classrooms provided mathemat-ics instruction in English and operated 5 days per week. The 10 classrooms were taught by 11 teachers (two half-time), and each class-room had an average of 20.3 students (SD = 2.3). Of the 11 teachers, 91% identified as female. Teachers had an average of 11.5 years of teaching experience and 5.1 years of first-grade teaching experience, 100% had a mas-ter’s degree in education, and 45% of teachers had completed an algebra course at the col-lege level.
Students and inclusion criteria. In each partici-pating classroom, all students with parental consent were screened in the winter of their first-grade year. The screening process, which included 221 first-grade students, comprised the three measures of the first-grade Assess-ing Student Proficiency in Early Number Sense (ASPENS; Clarke, Rolfhus, Dimino, & Gersten, 2012): Magnitude Comparison, Missing Number, and Basic Arithmetic Facts and Base-10. Students were considered eligi-ble for the PM-L1 intervention and thus con-sidered at risk for MD if they had an ASPENS composite score in the strategic or intensive categories (i.e., <43 on ASPENS composite) based on the winter benchmark. Composite scores at or below the strategic category sug-gest that students have less than a 50% chance of meeting end-of-year grade-level expecta-tions in mathematics (Clarke et al., 2012). Approximately 70% of students in both con-ditions had pretest ASPENS composite scores in the intensive category.
In each participating classroom, an inde-pendent evaluator rank ordered students with ASPENS composite scores less than 43, selected the 10 lowest-scoring eligible stu-dents for study inclusion, and then created
pairs of adjacent students with respect to their screener composite scores. One student from each pair was then randomly assigned to the PM-L1 intervention (treatment) and the other to business-as-usual Tier 2 mathematics sup-ports (control). In classrooms with an odd number of eligible students, we randomly assigned the last student to either treatment or control. Of the 221 students who were screened, 96 met the eligibility criteria and were randomly assigned within each of the 10 classrooms to the treatment (n = 49) or the control condition (n = 47).
Demographic data indicated that 8% of treatment and 13% of control students received special education services; 8% and 2% were ELs, and 58% and 64% were females, respectively. Although the majority racial group for both conditions was White (87%), 10% of treatment and 11% of control students were Hispanic. In both conditions, approximately 2% of students were multiple races.
Interventionists. PM-L1 intervention groups were taught by district-employed instructional assistants and by interventionists hired spe-cifically for this study. Among the interven-tionists, 100% identified as female. Most interventionists had previous experience pro-viding small-group mathematics instruction (60%) and held a bachelor’s degree or higher (80%). Interventionists had an average of 7.6 years of teaching experience, 20% had a cur-rent teaching license, and 80% had taken an algebra course at the college level.
Procedures
PM-L1 intervention. PM-L1 is a Tier 2 mathe-matics intervention aimed at building stu-dents’ conceptual understanding and problem-solving skills associated with early measurement and data analysis. Specifically, the intervention prioritizes standards, with exception of time and money, from the Mea-surement and Data domain of the first-grade CCSS-M and centers on principles of explicit and systematic mathematics instruction (Ger-sten et al., 2009; Hughes et al., 2017). In this
way, lessons include overt teacher modeling, guided and independent student practice opportunities, visual and concrete representa-tions of mathematics, and academic feedback to address potential misconceptions and iden-tify knowledge gaps in students’ problem-solving processes.
PM-L1 is composed of eight “investigative units” that are grounded in the disciplinary core ideas within the first-grade life science topic in the Next Generation Science Stan-dards (NGSS; National Research Council, 2013). Each unit is delivered across 4 consec-utive days, 30 min per session, for 8 weeks (32 lessons total). Disciplinary core ideas of the NGSS were used to provide students with meaningful science-based contexts when solving early measurement and data analysis problems. For example, students are taught to understand patterns of animal behavior, such as migration, while solving mathematics problems (see the PM-L1 scope and sequence in Supplemental Material online).
To promote students’ understanding of data analysis, the intervention utilizes the Confrey et al. (2012) framework of statistical investi-gation, which comprises four practices: (1) formulate research questions, (2) collect rele-vant data, (3) organize and represent data on graphs and analyze data to address the tar-geted research question, and (4) interpret the results. Students learn how to successfully use the framework when solving grade-level problems involving measurement and data analysis, such as measuring the length of foot-prints. In addition to statistical investigation, the PM-L1 intervention also explicitly teaches two other foundational aspects of early math-ematics. The first is mathematical language and vocabulary (Riccomini et al., 2015). PM-L1 offers opportunities for students to use precise mathematical language and vocabu-lary that is central to success within the inter-vention (e.g., centimeter, data, equation) and academic vocabulary that is important across content areas (e.g., explain, solve, justify, characteristic). Key vocabulary is presented in student-friendly language and systemati-cally reviewed across the investigative units. The second aspect prioritized in PM-L1 is the
word problem subtypes specified in the CCSS-M for first grade: add-to, take-from, put-together, take-apart, and compare situa-tions. Word problem solving is taught through a schema-broadening instructional approach (Powell, 2011), explicitly teaching students how to identify and use the structural features underlying each problem type to guide their solution method.
At the core of PM-L1 is a “hybrid” frame-work (Doabler et al., 2018) that utilizes tech-nology-based and hands-on problem-solving activities to promote a robust understanding of crucial measurement and data analysis con-cepts and problem-solving skills among stu-dents with MD. The technology activities, which are guided by the teacher’s iPad, offer students opportunities for individualized instruction and practice through their own iPads; whereas the hands-on problem-solving activities allow students to build conceptual understanding through concrete materials. Despite being different mediums of instruc-tion, each activity type allows students to engage in mathematical discourse with their teacher and peers around core concepts of measurement and data analysis.
To illustrate how PM-L1 is implemented, the following is an example of a lesson that involves the iPad for multiple activities (see sample lesson in Supplemental Material online). To begin, using iPads and the lesson script, the teacher guides students through a review of previously learned mathematical vocabulary—such as height, estimate, and centimeter—that is related to the mathemati-cal concepts targeted in the lesson. To review word meanings, the teacher uses explicit instruction to support students as they match definitions to corresponding words and depictions on their individual iPads. Follow-ing this activity, the teacher guides students through the lesson’s statistical investigation. In this lesson, students apply previously learned solution methods to answer a ques-tion regarding how long an earthworm grew in 1 week. Such methods include applying a virtual measurement tool to measure a worm’s length in centimeters, applying knowledge of the structural features of
“take-from” word problems, and collecting and representing data on a bar graph. To scaf-fold students’ learning, the teacher has the ability through the iPad interface to coordi-nate students’ iPads with the different instructional examples presented as well as error corrections as needed.
Using iPads and the lesson script, the teacher guides students through
a review of previously learned mathematical vocabulary—such as
height, estimate, and centimeter— that is related to the mathematical
concepts targeted in the lesson. In this study, the PM-L1 intervention was delivered in 30-min, small-group formats (i.e., four or five students per interventionist), 4 days per week for approximately 8 weeks, beginning in early winter. Because PM-L1 is designed as a supplemental intervention, treatment students continued to receive core Tier 1 mathematics instruction during the study. It is also important to note that inter-vention students did not receive Tier 2 math-ematics intervention support above and beyond the PM-L1 intervention.
Professional development. All interventionists participated in two 4-hr professional develop-ment workshops delivered by project staff. The first workshop was held prior to the start of PM-L1 implementation and focused on the instructional objectives and content of Lessons 1 through 16, whereas the second workshop targeted Lessons 17 through 32. Intervention-ists were provided opportunities to practice and receive feedback on lesson delivery from project staff. To promote implementation fidelity, all interventionists received two in-class coaching visits by a former educator with specialized knowledge and training in evi-dence-based mathematics instruction, the PM-L1 intervention, and effective small-group instructional practices. Coaching visits con-sisted of direct observations of lesson delivery, followed by feedback on instructional quality and fidelity of intervention implementation.
Fidelity of implementation. Fidelity of imple-mentation was measured via direct observa-tions by trained research staff. Each PM-L1 group was observed two times during the course of the intervention, documenting four features of implementation. The first compared the number of activities specified in the PM-L1 lesson materials with the number of activities taught in the observed lesson. On average, interventionists taught 4.55 out of 6 (SD = 1.23) of the prescribed activities. The remaining three features were measured using a 4-point scale (4 = all, 3 = most, 2 = some, 1 = none). Inter-ventionists were observed to meet mathe-matics objectives (M = 3.25, SD = 0.71), follow teacher scripting (M = 2.50, SD = 0.51), and use prescribed mathematics mod-els (M = 3.20, SD = 0.61).
Control condition. The control condition, which consisted of school-based Tier 1 and Tier 2 instructional supports, was docu-mented through teacher surveys. For their core (Tier 1) mathematics instruction, all teachers reported using the Bridges in Math-ematics curriculum, a preK–5 mathMath-ematics curriculum that is aligned with the CCSS-M and delivered in a variety of instructional formats, including whole-class, small-group, and 1:1 settings. Teachers noted that they provided an average of 71 min (SD = 11.79) of daily core mathematics instruction. All teachers indicated that teaching number and operations in base-10 was prioritized in their core instruction, followed by operations in algebraic thinking. As reported by teachers, approximately 40% of the control students received Tier 2 mathematics intervention support in addition to their core instruction. Teachers reported using the Bridges in Mathematics intervention program during the Tier 2 instruction. Such supports, which began in September and ran until the end of the school year, were delivered in small-group formats, 4 days per week in 30-min sessions, which was an equivalent dose of Tier 2 instruction to that of PM-L1. The reported focus of Tier 2 control instruction was on finding patterns in numbers, fact
families, identifying the relationship between addition and subtraction, and decomposing and composing numbers.
Measures
Students were administered five outcome measures at pretest and posttest. These mea-sures included a proximal assessment that measured skills identified in the CCSS-M for first grade, three distal mathematics outcome measures, and an early literacy curriculum-based measure (CBM) related to decoding and whole-word reading skills. Trained research staff administered all student mea-sures, and interscorer reliability criteria ≥.95 were met for all assessments.
PM-L1 proximal assessment. This individually administered measure comprises 22 items related to topics identified under the Measure-ment and Data domain in the CCSS-M for first grade. In an untimed setting, students are asked to sort objects, identify categories, interpret data, and create bar graphs. Coeffi-cient alpha for the current study sample was .71. Concurrent validity correlations with the ASPENS composite and easyCBM Math scores were greater than .54.
ASPENS (Clarke et al., 2012). ASPENS is a set of fluency-based measures (1 min each) vali-dated for screening and progress monitoring of students’ number sense proficiency, includ-ing number identification, magnitude com-parison, missing number, and arithmetic facts and base-10. Test authors report test-retest reliability ranges from the .70s to .90. Crite-rion concurrent validity with the TerraNova 3 is reported as ranging from .51 to .63. ASPENS composite (i.e., magnitude compari-son, missing number, and arithmetic facts and base-10) and Number Identification scores were examined as separate outcomes in this study. Composite scores below 28 represent the intensive performance category based on first-grade winter benchmarks and thus served as the cutoff used in the differential response analyses.
Early Measurement Curriculum-Based Measures (EM-CBM; Clarke, Sutherland, & Doabler, 2017). This set of four 1-min fluency-based CBMs focuses on concepts of early measure-ment identified in the CCSS-M for first grade. The EM-CBM battery comprises four mea-sures: (a) Comparison of Three Objects, (b) Iteration-1, (c) Iteration-2, and (d) Measurement of Two Items Using an Object. Specifically, the measures assess students’ understanding of how to compare the length of multiple objects, how to compare the lengths of two objects using a third object, and how to engage in iterative measurement with a nonstandard measure-ment tool. Because EM-CBM is a timed assessment, coefficient alphas were not calcu-lated. Concurrent validity correlations with the ASPENS composite and easyCBM Math scores were .31 and .26, respectively. EM-CBM pretest and posttest correlations were moderate (r = .55).
EasyCBM Math (Alonzo, Tindal, Ulmer, & Glasgow, 2006). This assessment is an online mathe-matics outcome measure designed for kinder-garten to eighth grade. The test items are multiple choice and assess all domains of the CCSS-M. Reliability and validity of the assessments are well established. Internal reli-abilities of first-grade easyCBM are high (.81–.84). Concurrent validity of easyCBM scores on the winter benchmark, with the Stanford Achievement Test, Tenth Edition, ranges from .75 to .82.
Dynamic Indicators of Basic Early Literacy Skills: Nonsense Word Fluency–Whole Words Read (DIBELS NWF-WWR; Dynamic Measurement Group, 2016). DIBELS NWF-WWR is a flu-ency-based (1 min) CBM validated for screen-ing and progress monitorscreen-ing early literacy skills (i.e., decoding) in first grade. Alternate form reliability of the first-grade DIBELS NWF measures is in the moderate to high range (.90–.96). Predictive validity of fall scores on the first-grade NWF measure, with scores on the Group Reading Assessment and Diagnostic Evaluation, ranges from .39 to .43. Student NWF-WWR scores below 8 represent below winter benchmark and thus served as
the cutoff for early literacy risk in the differ-ential response analyses.
Statistical Analysis
We tested the efficacy of the PM-L1 interven-tion using a mixed Time × Condiinterven-tion analysis designed to account for students (unit of ran-dom assignment) nested within small groups in the treatment condition and not nested in the control condition (Baldwin, Bauer, Stice, & Rohde, 2011; Bauer, Sterba, & Hallfors, 2008). These partially nested analyses tested for differences between conditions on change in outcomes from pretest to posttest and are described in detail by Doabler et al. (2016). The model included effects of condition, time, and the Time × Condition interaction, with condition coded 0 for control and 1 for treat-ment, and time coded 0 at pretest and 1 at posttest. The effect of condition represents the difference in outcome between the treatment and control conditions at pretest. The effect of time represents the change in outcome from pretest to posttest in the control condition. The Time × Condition interaction represents the difference in change in outcome between the treatment and control conditions. Hedges’ g effect size (Hedges, 1981) was reported to support interpretation of results.
We also examined whether early numeracy or early literacy risk status at pretest predicted differential response to the PM-L1 interven-tion. We expanded the statistical model to include dichotomous risk status and its inter-action with condition, time, and the Time × Condition term. The three-way interactions provided estimates of whether condition effects varied by risk status (i.e., early numer-acy or early liternumer-acy).
We performed the analyses with SAS PROC MIXED Version 9.4 (SAS Institute, 2016), restricted maximum-likelihood esti-mation, and Satterthwaite approximation to determine degrees of freedom. Maximum-likelihood estimation uses all available data and produces potentially unbiased results even in the face of substantial miss-ing data, provided the missmiss-ing data were missing at random (Schafer & Graham,
2002). We considered this assumption ten-able given the minimal rates of missing data. The statistical model also assumes independent and normally distributed observations. We addressed the first of these assumptions by modeling the multi-level nature of the data. The outcome mea-sures in the present study also did not markedly deviate from normality; skew-ness and kurtosis fell within ±2.3.
The trade-off between Type I and Type II errors represents a delicate balance, particu-larly in intervention development studies with limited resources. False conclusions that interventions affect performance (Type I errors) are problematic, as is failing to detect effects of potentially valuable interventions (Type II errors). To balance the likelihood of the two types of errors, Cohen (1990) and Rosnow and Rosenthal (1989) recommend an adjustment to alpha, the Type I error rate. Thus, with alpha adjusted to .10, our sample of 96 students, and a pre-/posttest outcome correlation of .50, we were powered (.80) to detect medium or larger effects (g ≥ 0.51). We also corrected for multiple tests using the Benjamini-Hochberg procedure (Benjamini & Hochberg, 1995) within the set of analyses for each research question.
Results
Baseline Equivalence and Attrition
Table 1 reports descriptive statistics for study outcome variables by assessment time and condition. Treatment and control groups did not significantly differ on pretest outcomes (ps ≥ .523, gs ≤ 0.15). Examination of attri-tion between pretest and posttest revealed 98% (48/49) of the treatment students com-pleted a posttest assessment compared to 100% (47/47) of the control students, χ2(1,
96) = 0.97, p = .325.
Evidence of Intervention Efficacy
We first tested the hypothesis that partici-pants in the PM-L1 intervention condition would experience greater increases in
mathematics outcomes than did students in the control condition. The model summa-rized in Table 2 tested fixed effects for time, condition, and their interaction on outcomes. Impact analyses included all 96 students (49 treatment, 47 control), as they all provided either pretest or posttest data for each out-come measure. The bottom two rows of the table show the likelihood-ratio test results that compared homoscedastic residuals with heteroscedastic residuals. Condition effect estimates and their statistical significance values were very similar for the heterosce-dastic and homosceheterosce-dastic models.
As shown in the fourth column of Table 2, students in the PM-L1 intervention condition experienced significantly greater increases in EM-CBM scores from pretest to posttest than
did students in the control condition (t = 2.65, df = 30, p = .013, Benjamini-Hochberg cor-rected p = .065). The model estimated differ-ences in gains between treatment and control conditions of 6.3 (g = 0.45). Although not statistically significant, small to medium intervention effects were also observed on the PM-L1 proximal assessment (g = 0.21), ASPENS composite (g = 0.03), ASPENS Number Identification (g = 0.19), and easy-CBM (g = 0.08).
Students in the PM-L1 intervention condition experienced
significantly greater increases in EM-CBM scores from pretest to posttest than did students in the
control condition. Table 1. Descriptive Statistics for Mathematics Outcome Measures by Assessment Time and
Condition.
Pretest Posttest
Measure Intervention Control Intervention Control
PM-L1 proximal assessment M 15.3 14.5 19.3 17.7 (SD) (4.9) (4.8) (5.6) (6.5) n 48 47 48 46 ASPENS composite M 18.5 17.6 31.5 30.0 (SD) (11.8) (11.9) (14.4) (16.0) n 49 47 47 45 ASPENS Number ID M 42.3 40.6 53.7 49.7 (SD) (14.2) (13.6) (11.3) (12.2) n 49 47 47 45 EM-CBM total M 41.5 41.2 48.4 41.8 (SD) (14.4) (12.2) (15.6) (12.1) n 48 47 45 45 easyCBM M 18.8 18.6 25.5 24.3 (SD) (4.3) (5.4) (5.6) (6.2) n 46 45 46 46
Note. The sample included 49 treatment students and 47 control students. The treatment and control conditions had
approximately 73% and 72% of students, respectively, with pretest ASPENS composite scores in the intensive risk category. Approximately 41% of treatment and 40% of control students had pretest DIBELS NWF-WWR scores below or well benchmark. PM-L1 = Precision Mathematics Level 1; ASPENS = Assessing Student Proficiency in Early Number Sense; ID = identification; EM-CBM = Early Measurement Curriculum-Based Measures; DIBELS NWF-WWR = Dynamic Indicators of Basic Early Literacy Skills: Nonsense Word Fluency–Whole Words Read.
Evidence of Differential Response
Table 3 presents tests of differential response to PM-L1 as a function of early numeracy risk status at pretest. Although not statistically sig-nificant, differential gains by condition were greater among students in the intensive range at pretest compared to students in the strategic
range for three of the five outcomes: ASPENS composite (estimated treatment gains vs. con-trol gains = 1.2 vs. –1.8, respectively; p = .571), ASPENS Number Identification (2.9 vs. –1.2, respectively; p = .388), and EM-CBM (7.3 vs. 3.8, respectively; p = .545).
Table 4 presents tests of differential response to PM-L1 as a function of early literacy risk at Table 2. Results of a Partially Nested Time × Condition Analysis of Pretest-Posttest Change in Math Outcomes.
Variable PM-L1 proximal assessment compositeASPENS Number IDASPENS EM-CBM total easyCBM Fixed effects Intercept 14.4*** (0.8) 17.6***(2.0) 40.6***(1.9) 41.2***(1.8) 18.6***(0.8) Time 3.3*** (0.9) 12.7***(1.7) (1.6)9.3*** (1.6)0.6 (0.9)5.8*** Condition 0.5 (1.1) (3.0)0.7 (2.7)1.7 (2.6)0.3 (1.4)0.2 Time × Condition 1.2 (0.9) (2.9)0.5 (2.5)2.2 (2.4)6.3* (1.7)0.5 Variances
Gains between groups -1.2**
(0.5) (13.7)18.8 (9.2)5.2 -10.2(8.5) (3.7)8.9* Pre-/posttest covariance 118.8*** (23.3) 105.4***(20.4) 12.2***(3.3) Residual 45.2*** (10.3) (11.5)55.9*** (2.2)8.8*** PM-L1 residual 8.5*** (1.9) 113.7***(26.6) Pre-/posttest covariance 22.9*** (5.6) 119.0**(36.8) Control residual 18.9*** (3.7) (15.5)71.1*** Pre-/posttest covariance 14.7** (5.2) (25.5)87.2*** Time × Condition Hedges’ g 0.21 0.03 0.19 0.45 0.08 p value .190 .877 .378 .013 .779 BH p value .475 .877 .630 .065 .877 df 54 19 25 30 28 Likelihood ratio χ2 8.47 0.97 0.48 4.65 1.65 p value .015 .617 .785 .098 .438
Note. Table entries show parameter estimates with standard errors in parentheses. Tests of fixed effects accounted
for small groups as the unit of analysis within the PM-L1 condition and unclustered students in the control condition.
P values provided with and without the Benjamini-Hochberg (BH) correction. Likelihood ratio test compared
homoscedastic residuals to heteroscedastic residuals with a criterion α of .20 and one degree of freedom. PM-L1 = Precision Mathematics Level-1; ASPENS = Assessing Student Proficiency in Early Number Sense; ID = identification; EM-CBM = Early Measurement Curriculum-Based Measures.
pretest. Although not statistically significant, differential gains by condition were greater among students below benchmark at pretest compared to students at or above benchmark
for two of the five outcome measures: ASPENS Number Identification (3.3 vs. 1.5, respec-tively; p = .700) and EM-CBM (11.4 vs. 3.1, respectively; p = .121).
Table 3. Tests of Early Numeracy Risk (ENR) as a Predictor of Differential Response to Intervention.
Variable PM-L1 proximal assessment compositeASPENS Number IDASPENS EM-CBM total easyCBM Fixed effects Intercept 16.9**** (1.6) 33.1****(2.9) 51.5****(3.3) 44.2****(3.3) 22.0****(1.4) Time 2.2 (1.7) 11.2***(3.2) (2.9)2.8 (3.1)3.7 (1.7)4.6* Condition 3.0 (2.0) (4.3)0.1 (4.8)3.4 (5.1)2.2 (2.2)0.9 Time × Condition 2.1 (1.9) –1.8(4.7) –1.2(4.3) (5.0)3.8 (2.6)1.2 ENR –3.4† (1.8) –21.5****(3.5) –15.2***(3.9) –4.2(3.9) –4.7**(1.7) ENR × Condition –3.2 (2.3) (4.9)1.3 –1.9(5.5) –2.6(6.1) –0.7(2.3) ENR × Time 1.6 (2.0) (3.7)2.1 (3.4)8.9** –4.2(3.7) (2.0)1.6
ENR × Time × Condition –1.2
(2.2) (5.2)3.0 (4.8)4.2 (5.9)3.6 –1.2(2.7)
Variances
Gains between groups –1.2*
(0.5) (11.8)14.3 (11.0)10.4 –10.1(9.0) 10.1**(3.9) Pre-/posttest covariance 50.3*** (13.0) (17.4)90.8**** (2.8)8.0** Residual 47.5**** (10.6) (10.8)40.8**** (2.1)8.6**** PM-L1 residual 8.7**** (2.0) 115.8****(27.4) Pre-/posttest covariance 14.5*** (4.0) 111.3**(36.1) Control residual 19.0**** (3.8) (15.7)70.2**** Pre-/posttest covariance 13.7** (5.1) (24.8)83.0***
ENR × Time × Condition
p value .603 .571 .388 .545 .667
BH p value .667 .667 .667 .667 .667
df 60 89 92 54 84
Likelihood ratio χ2 8.45 1.60 1.56 4.96 2.86
p value .015 .450 .459 .084 .239
Note. Table entries show parameter estimates with standard errors in parentheses. Analyses accounted for small
groups as the unit of analysis within the PM-L1 condition. P values provided with and without the Benjamini-Hochberg (BH) correction. Likelihood ratio tests compared homoscedastic residuals to heteroscedastic residuals with a criterion α of .20 and two degrees of freedom. PM-L1 = Precision Mathematics Level 1; ASPENS = Assessing Student Proficiency in Early Number Sense; ID = identification; EM-CBM = Early Measurement Curriculum-Based Measures. †p < .10. *p < .05. **p < .01. ***p < .001.
Discussion
The purpose of this study was to investigate the efficacy of a Tier 2 mathematics intervention
aimed at increasing the mathematics achieve-ment of first-grade students with MD. Specifically, the PM-L1 was designed to build students’ conceptual and procedural Table 4. Tests of Early Literacy Risk (ELR) as a Predictor of Differential Response to Intervention.
Variable PM-L1 proximal assessment compositeASPENS Number IDASPENS EM-CBM total easyCBM Fixed effects Intercept 15.6**** (1.1) 21.3****(2.5) 43.6****(2.4) 44.4****(2.2) 20.1****(1.0) Time 3.2** (1.2) 12.0****(2.2) (2.1)8.5*** (2.1)0.3 (1.2)6.0**** Condition 0.6 (1.4) –0.4(3.7) (3.5)0.8 –0.2(3.4) –0.7(1.6) Time × Condition 1.4 (1.3) (3.5)0.6 (3.1)1.5 (3.1)3.1 (1.9)0.6 ELR –2.9† (1.7) –9.2*(3.9) –7.5†(3.8) –8.0*(3.5) –3.7*(1.6) ELR × Condition –0.1 (2.3) (5.4)2.9 (5.3)2.3 (5.5)1.0 (2.2)2.0 ELR × Time 0.3 (1.8) (3.5)1.8 (3.4)2.0 (3.4)0.7 –0.3(1.8) ELR × Time × Condition –0.4(2.1) –0.3(4.7) (4.7)1.8 (5.3)8.3 –0.2(2.3) Variances
Gains between groups –1.2*
(0.5) (14.5)19.8 (10.0)6.8 –11.2(7.9) (3.6)8.8* Pre-/posttest covariance 107.9**** (22.5) 101.7****(20.1) 10.2***(3.0) Residual 45.9**** (10.5) (11.6)54.6**** (2.3)9.2**** PM-L1 residual 8.6**** (2.0) 108.2****(25.5) Pre-/posttest covariance 21.2**** (5.3) 122.2***(36.5) Control residual 19.2**** (3.9) (15.5)73.3**** Pre-/posttest covariance 13.2** (5.1) (23.4)75.1**
ELR × Time × Condition
p value .855 .944 .700 .121 .924
BH p value .944 .944 .944 .605 .944
df 66 86 90 73 86
Likelihood ratio χ2 8.48 0.77 0.64 4.64 0.95
p value .014 .680 .725 .098 .622
Note. Table entries show parameter estimates with standard errors in parentheses. Analyses accounted for small
groups as the unit of analysis within the PM-L1 condition. P values provided with and without the Benjamini-Hochberg (BH) correction. Likelihood ratio tests compared homoscedastic residuals to heteroscedastic residuals with a criterion α of .20 and two degrees of freedom. PM-L1 = Precision Mathematics Level 1; ASPENS = Assessing Student Proficiency in Early Number Sense; ID = identification; EM-CBM = Early Measurement Curriculum-Based Measures. †p < .10. *p < .05. **p < .01. ***p < .001.
knowledge of early measurement and data analysis concepts and problem-solving skills. A randomized controlled trial was employed to address two research questions.
First, it was hypothesized that the PM-L1 intervention would demonstrate efficacy, sug-gesting treatment students would demonstrate greater gains than their control peers on the five targeted outcome measures. Although findings for most of the outcome measures did not reach statistical significance, all treatment effects were positive and three outcome measures had effect sizes at or greater than 0.19. Another encourag-ing findencourag-ing was that PM-L1 students made sta-tistically significant greater gains than their control peers on a set of researcher-developed CBMs focused on early measurement skills (g = 0.45). Collectively, a pattern of “promise” emerged from our first research question, and this may be important for the field because little methodologically rigorous research has been conducted involving interventions focused on measurement and data analysis for students with MD (Frye et al., 2013).
All treatment effects were positive and three outcome measures had effect sizes at or greater than 0.19. For our second research question, we explored whether and to what extent two stu-dent-level variables might influence students’ response to the PM-L1 intervention: initial mathematics achievement and early literacy skills. A growing body of research has identi-fied initial mathematics achievement as a risk factor for MD and as an explanatory variable of students’ differential response to whole-number-focused mathematics interventions (Clarke et al., 2019; Powell et al., 2017). Yet few studies have investigated the potential role early literacy skills might play in stu-dents’ response to mathematics interventions. Given that recent federal law (i.e., READ Act) pushes strongly for educational research in the STEM fields to focus on dyslexia and other reading learning disabilities, future research is needed to explore the predictive utility of early literacy skills in mathematics intervention research.
Preliminary evidence of differential response to PM-L1 surfaced for both risk variables. Although not statistically signifi-cant, results suggested differential gains in outcomes by conditions were greater for treatment students with more intensive needs around initial understanding of whole num-bers and word reading skills on three of the four distal outcome measures. It may be that the intervention’s explicit instructional framework was able to cater to the instruc-tional needs of these students. However, future research is warranted to more formally examine these moderating variables of impact. Generally speaking, exploring stu-dent-level characteristics, such as academic skills at pretest, may help the field better understand students who respond to well-designed interventions as well as those stu-dents who are minimally responsive to such interventions (Fuchs, Fuchs, & Gilbert, 2019; Miller, Vaughn, & Freund, 2014; Tran, San-chez, Arellano, & Swanson, 2011). Such information may also guide curriculum developers in revising interventions to better meet the needs of students who make mini-mal gains. For example, we used the differen-tial response information generated in the current study to revise some of the different student practice opportunities offered in PM-L1, with the aim of catering more to stu-dents with lower initial skill levels in mathe-matics and early literacy.
Implications
Although preliminary, findings from this study raise several considerations for the field. Above all, this study adds support for devel-oping and testing mathematics interventions that target areas of mathematics outside of whole numbers, such as measurement and data analysis. Findings from reviews of the mathematics research conducted by the NCII (n.d.) and the WWC (n.d.) suggest a signifi-cant lack of rigorous research being conducted on interventions focused on measurement and data analysis topics. Of the 150 mathematics programs reviewed by the WWC, for exam-ple, none targeted these areas of mathematics.
Given that acquisition of foundational mea-surement and data analysis skills is crucial for working with more complex mathematics content and that there is compelling evidence of an early achievement gap in these areas of mathematics, we contend there is need for early and sustained interventions on these “other” topics. Such interventions may offer meaningful opportunities for students with MD to build important bridges across areas of mathematical learning (Frye et al., 2013).
This study adds support for developing and testing mathematics
interventions that target areas of mathematics outside of whole
numbers.
Relatedly, meeting the instructional needs of students with MD may require a more coherent progression and comprehensive sys-tem of mathematics support. Within a multi-tier system of support (MTSS) model of service delivery, for example, it may be neces-sary to complement interventions that provide a specific focus on number and operations with interventions that target other areas of mathematics, such as measurement and data analysis. When designed to flexibly work in MTSS delivery models, it is plausible that these “other” mathematics interventions con-tribute to gains in student mathematics achievement over and above whole-number interventions. Future research in this area is warranted.
Finally, we believe this study lends support for collecting initial evidence of efficacy dur-ing intervention development projects. Researchers from prominent research fields, such as medicine (De Mattos-Arruda & Rodon, 2013), gather preliminary evidence of intervention efficacy prior to conducting more formal clinical trials. In these fields, collection of such data represents a first, albeit central, phase in the linear progression of intervention development, efficacy and effectiveness test-ing, and wide-scale diffusion of interventions. It seems warranted, therefore, that a similar process is employed in educational research.
Large-scale efficacy trials can be expensive, particularly when they take place in different geographical regions. Obtaining trustworthy and usable information about key research design features, such as intervention dosage levels, could help researchers design more methodologically rigorous efficacy trials.
Limitations
Several limitations highlight directions for future research. First, the study’s sample was not racially diverse and, moreover, was rela-tively small, which provided power to detect only medium to large intervention effects and even less power to detect moderation effects. As such, we are reluctant to overstate practi-cal implications from our preliminary find-ings. Another limitation was the limited availability of psychometrically sound out-come measures related to measurement and data analysis. Because of this shortage, we developed two assessments around these areas of mathematics. Results from these measures should be interpreted cautiously given their initial technical properties. Additionally, we used students’ word reading skills as a predic-tor of differential response to PM-L1. Other indices of mathematical language, such as an expressive measure of mathematics vocabu-lary, should be considered. We also did not include a follow-up measure to document whether effects were maintained beyond the intervention.
Low fidelity of implementation was also noted, and this may have affected our find-ings. Specifically, observers documented that some interventionists loosely followed the prescribed teacher scripting. It may be that we need to better instruct intervention-ists on how to rely on the teacher scripting in future studies on PM-L1. This could be done through in-class coaching. On the other hand, the low fidelity may have been attributable to factors outside of the interventionists’ con-trol. It is plausible, for example, that han-dling behavior management issues and technology problems may have forced inter-ventionists to adapt some of the teaching script in order to complete lessons in the
30-min time period allotted for PM-L1. Regardless of the reason for the low fidelity with teacher scripting, future research is needed to better understand implementation fidelity and best support interventionists in delivering the intervention as intended. Another limitation was the lack of direct observations of Tier 2 instruction received by students in the control condition. Relat-edly, only two observations were conducted per intervention group. Additional observa-tions may have provided a richer description of program implementation and the instruc-tional interactions that occurred between teachers and students around measurement and data analysis. The possible discrepancy between the amount of mathematics instruc-tional time received by the treatment and control groups may be another limitation.
Additionally, monies available for investi-gating technology-based mathematics inter-ventions is another limitation worth noting. Because most of the participating schools had dated technology, we were forced to purchase iPads for all participating interventionists and students. Consequently, this posed major financial strains on our project and, in turn, restricted the study’s sample size. Finally, and above all, although all our findings demon-strated positive effects on all outcome mea-sures, statistical significance was obtained for only one measure. Thus, the field could ques-tion whether our effects are “substantively important” (WWC, 2017). However, consid-ering the nascent nature of our work in the area and the field’s as a whole, coupled with the number of students who struggle with measurement and data analysis, we contend our results represent a promising avenue to pursue in a crucial area of mathematics inter-vention research.
Conclusion
To date, the majority of curriculum develop-ment work in early mathematics has had an exclusive focus on whole numbers (Gersten et al., 2009; NCII, n.d.; WWC, n.d.). And although these cogent efforts have produced a host of empirically validated programs, at the
same time, there are calls by experts to broaden students’ mathematical experiences beyond number and operations, including measure-ment and data analysis (Frye et al., 2009). Designing effective interventions around these other areas of mathematics has strong poten-tial not only to strengthen students’ whole-number understanding and help build a base for future mathematical learning but also to support schools in implementing more com-prehensive mathematics MTSS models.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/ or publication of this article: Research reported here was supported by the National Science Foun-dation through Grant 1503161 awarded to Drs. Christian T. Doabler, Ben Clarke, Hank Fien, and Nancy Nelson at the Center on Teaching and Learning at the University of Oregon.
Supplemental Material
The supplemental material is available in the online version of the article.
References
Alonzo, J., Tindal, G., Ulmer, K., & Glasgow, A. (2006). EasyCBM online progress
monitor-ing assessment system. Eugene: University of
Oregon.
Baldwin, S. A., Bauer, D. J., Stice, E., & Rohde, P. (2011). Evaluating models for partially clustered designs. Psychological Methods, 16, 149–165. doi:10.1037/ a0023464
Bauer, D. J., Sterba, S. K., & Hallfors, D. D. (2008). Evaluating group-based interventions when control participants are ungrouped.
Multivariate Behavioral Research, 43, 210–
236. doi:10.1080/00273170802034810 Benjamini, Y., & Hochberg, Y. (1995). Controlling
the false discovery rate: A practical and pow-erful approach to multiple testing. Journal
of the Royal Statistical Society. Series B (Methodological), 57, 289–300. Retrieved
from http://www.jstor.org/stable/2346101 Clarke, B., Sutherland, M., & Doabler, C. T.
(2017). Early Measurement
Curriculum-Based Measures. Unpublished measures,
Center on Teaching and Learning, University of Oregon, Eugene.
Clarke, B., Doabler, C. T., Smolkowski, K., Turtura, J., Kurtz-Nelson, E., Fien, H., & Baker, S. (2019). Exploring the relationship between initial math skill and a kindergarten mathemat-ics intervention. Exceptional Children, 85, 129–146. doi:10.1177/0014402918799503 Clarke, B., Rolfhus, E., Dimino, J., & Gersten,
R. M. (2012). Assessing Student Proficiency
of Number Sense (ASPENS). Longmont, CO:
Cambium Learning Group, Sopris Learning. Cohen, J. (1990). Things I have learned (so far).
American Psychologist, 45, 1304–1312.
Confrey, J., Maloney, A., Nguyen, K., Lee, K. S., Panorkou, N., Corley, D., . . . Gibson, T. (2012). TurnOnCCMath.net learning
trajec-tories for the K–8 Common Core math stan-dards. Retrieved from http://turnonccmath.
net/index.php
De Mattos-Arruda, L., & Rodon, J. (2013). Pilot studies for personalized cancer medicine: Focusing on the patient for treatment selec-tion. Oncologist, 18, 1180-1188. doi:10.1634/ theoncologist.2013-0135
Doabler, C. T., Clarke, B., Fien, H., Nelson, N. J., & Smolkowski, K. (2015). Precision
Mathematics: Using interactive gaming tech-nology to build student proficiency in the foun-dational concepts and problem solving skills of measurement and data analysis (Award
No. 1503161). Washington, DC: National Science Foundation, Division of Research on Learning, Directorate for Education and Human Resources, Research on Learning in Formal and Informal Settings, Discovery Research K–12. Doabler, C. T., Clarke, B., Firestone, A., Turtura,
J. Jungjohann, K., Brafford, T., . . . Fien, H. (2018). Applying the curriculum research framework in the design and development of a technology-based Tier 2 mathematics intervention. Journal of Special Education
Technology. Advance online publication.
doi:10.1177/0162643418812051
Doabler, C. T., Clarke, B., Kosty, D., Kurtz-Nelson, E., Fien, F., Smolkowski, K., & Baker, S. K. (2016). Testing the efficacy of a Tier-2 math-ematics intervention: A conceptual replica-tion study. Excepreplica-tional Children, 83, 92–110. doi:10.1177/0014402916660084
Doabler, C. T., Clarke, B., Kosty, D., Smolkowski, K., Kurtz-Nelson, E., Fien, H., & Baker, S. (2019). Building number sense among English learners: A multisite randomized controlled trial of a tier 2 kindergarten mathematics intervention. Early Childhood Research
Quarterly, 47, 432–444. doi:10.1016/j.
ecresq.2018.08.004
Dynamic Measurement Group. (2016). Dynamic
Indicators of Basic Early Literacy Skills Next.
Eugene, OR: Author.
Frye, D., Baroody, A. J., Burchinal, M., Carver, S. M., Jordan, N. C., & McDowell, J. (2013).
Teaching math to young children: A prac-tice guide (Pracprac-tice Guide No. NCEE
2014-4005). Washington, DC: National Center for Education and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
Fuchs, L. S., Fuchs, D., & Gilbert, J. K. (2019). Does the severity of students’ pre-inter-vention math deficits affect responsiveness to generally effective first-grade interven-tion? Exceptional Children, 85, 147–162. doi:10.1177/0014402918782628
Gersten, R. M., Chard, D., Jayanthi, M., Baker, S. K., Morphy, P., & Flojo, J. (2009). Mathematics instruction for students with learning disabili-ties: A meta-analysis of instructional compo-nents. Review of Educational Research, 79, 1202–1242. doi:10.3102/0034654309334431 Hedges, L. V. (1981). Distribution theory for
Glass’s estimator of effect size and related estimators. Journal of Educational and
Behavioral Statistics, 6, 107–128.
Hughes, C. A., Morris, J. R., Therrien, W. J., & Benson, S. K. (2017). Explicit instruc-tion: Historical and contemporary contexts.
Learning Disabilities Research & Practice, 32, 140–148. doi:10.1111/ldrp.12142
Institute of Education Sciences & National Science Foundation. (2013). Common
guide-lines for education research and develop-ment. Retrieved from https://www.nsf.gov/
pubs/2013/nsf13126/nsf13126.pdf
Langrall, C. W., & Mooney, E. S. (2011). Data analysis and probability. In F. M. Fennell (Ed.), Achieving fluency: Special education
and mathematics (pp. 219–242). Reston, VA:
National Council of Teachers of Mathematics. Miller, B., Vaughn, S., & Freund, L. S. (2014).
Learning disabilities research studies: Findings from NICHD-funded projects. Journal of
Research on Educational Effectiveness, 7,
225–231. doi:10.1080/19345747.2014.927251 National Center for Education Statistics. (2017).
The nation’s report card: 2017 mathematics assessment. Retrieved from https://nces.ed.gov
National Center on Intensive Intervention. (n.d.).
Academic intervention. Retrieved from https://
intensiveintervention.org
National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards
for Mathematics. Washington, DC: Author.
National Research Council. (2013). Next Generation
Science Standards: For states, by states.
Washington, DC: National Academies Press. National Science Board. (2018). Science and
engi-neering indicators 2018 (Report No.
NSB-2018-02). Arlington, VA: National Science Foundation. Retrieved from https://www.nsf. gov/statistics/2018/nsb20181/
Parmar, R. S., Garrison, R., Clements, D. H., & Sarama, J. (2011). Measurement. In F. (Skip) Fennell (Ed.), Achieving fluency: Special
education and mathematics (pp. 197–218).
Reston, VA: National Council of Teachers of Mathematics.
Paules, C. I., Marston, H. D. & Fauci, A. S. (2019). Measles in 2019: Going backward. New
England Journal of Medicine. Retrieved from
https://www.nejm.org/doi/pdf/10.1056/NEJM p1905099?articleTools=true
Powell, S. R. (2011). Solving word problems using schemas: A review of the literature. Learning
Disabilities Research & Practice, 26, 94–108.
Powell, S. R., Cirino, P. T., & Malone, A. S. (2017). Child-level predictors of responsive-ness to evidence-based mathematics interven-tion. Exceptional Children, 83, 359–377. Riccomini, P. J., Smith, G. W., Hughes, E. M., & Fries,
K. M. (2015). The language of mathematics:
The importance of teaching and learning mathematical vocabulary. Reading & Writing
Quarterly, 31, 235–252. doi:10.1080/1057356
9.2015.1030995
Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowl-edge in psychological science. American
Psychologist, 44, 1276–1284.
SAS Institute. (2016). SAS/STAT 14.2 user’s guide. Cary, NC: Author. Retrieved from the SAS Product Documentation website: http://sup-port.sas.com/documentation/index.html Schafer, J. L., & Graham, J. W. (2002). Missing
data: Our view of the state of the art.
Psychological Methods, 7, 147–177.
Shaughnessy, J. M. (2007). Research on statistics learning and reasoning. In F. K. Lester (Ed.),
Second handbook of research on mathematics teaching and learning (Vol. 2, pp. 957–1010).
Charlotte, NC: Information Age.
Tran, L., Sanchez, T., Arellano, B., & Swanson, H. L. (2011). A meta-analysis of the RTI litera-ture for children at risk for reading disabilities.
Journal of Learning Disabilities, 44, 283–295.
doi:10.1177/0022219410378447
What Works Clearinghouse. (2017). Procedures and
standards handbook (Version 4.0). Washington
DC: U.S. Department of Education, Institute of Education Sciences.
What Works Clearinghouse. (n.d.). What works in math? Retrieved from https://ies.ed.gov/ncee/ wwc/FWW/Results?filters=,Math
Manuscript received November 2018; accepted May 2019.
