Dissertations
9-2020
The Impact of Reading Intervention on Retained Elementary-Age
The Impact of Reading Intervention on Retained Elementary-Age
Students
Students
Elizabeth LazarFollow this and additional works at: https://digitalcommons.nl.edu/diss
Part of the Curriculum and Instruction Commons, Elementary Education Commons, and the Language and Literacy Education Commons
Recommended Citation Recommended Citation
Lazar, Elizabeth, "The Impact of Reading Intervention on Retained Elementary-Age Students" (2020). Dissertations. 488.
https://digitalcommons.nl.edu/diss/488
This Dissertation - Public Access is brought to you for free and open access by Digital Commons@NLU. It has been accepted for inclusion in Dissertations by an authorized administrator of Digital Commons@NLU. For more information, please contact [email protected].
The Impact of Reading Intervention on Retained Elementary-Age Students
Elizabeth Anne Lazar
Educational Leadership Doctoral Program
Submitted in Partial Fulfillment of the Requirements of
Doctor of Education
In the Foster G. McGaw Graduate School
National Louis University 2018
Copyright by Elizabeth Anne Lazar, 2020 All rights reserved
ABSTRACT
Due to historically high rates of retained students, one low-performing school needed to fix the endless cycle of student failure. After implementation of a close reading program, participating students made growth on the state reading test. My main goal was to examine which key elements of this program supported retained third grade students resulting in learning gains. My data showed t-test findings with a statistically significant difference in both scale scores and achievement levels on the state reading assessment. My recommended policy change provides elementary students with a district-wide close reading program to teach comprehension strategies. Through policy measures, I
considered the need for a change vision and a detailed action plan to support the needs of elementary students’ reading skills.
PREFACE
After witnessing the incredible growth made by a small cohort of retained third grade students on a state reading assessment, I wanted to find the reason for their success on the state reading assessment. Prior to the spring testing season, a new reading
intervention program was implemented at low-performing elementary schools to support third grade students that were retained based on low performance on the state reading test. Teachers of the close reading program provided small group instruction, both with grade level text and with independent reading level text. Additionally, students were taught close reading strategies to increase comprehension of complex text, like they would encounter on the state reading assessment. And they succeeded. After participation in the program, 86% of the group made learning gains on the reading assessment that spring. What about this reading program led to this major success? Could this program be replicated at other school sites? It became my mission to find out this information and explore the possibility of expanding a close reading program to all upper elementary students within a school district.
ACKNOWLEDGEMENTS
I would like to thank my National Louis University faculty members for their unending support throughout my doctoral journey.
DEDICATION
TABLE OF CONTENTS ABSTRACT ... iv PREFACE ...v ACKNOWLEDGEMENTS ... vi DEDICATION ... vii LIST OF TABLES ... xi
LIST OF FIGURES ... xii
CHAPTER ONE ...1
Introduction ...1
Purpose of the Program Evaluation ...1
Rationale ...4 Goals ...7 Definition of Terms ...9 Research Questions ... 11 Conclusion ...12 CHAPTER TWO ...13
Review of the Literature ...13
Characteristics of Retained Students ...13
Early Intervention and Retention ...14
Environmental Factors to Support Retained Students ...15
Comprehension Strategies ...17
Conclusion ...19
CHAPTER THREE ...21
Methodology ...21
Research Design Overview ...21
Participants ...22
Data Gathering Techniques ...23
Extant Data...23
Interviews ...24
Data Analysis Techniques ...24
Ethical Considerations ...25 Limitations. ...25 Conclusion ...26 CHAPTER FOUR ...27 Results ...27 Findings...28
Student Growth Demonstrated on State Assessment Test ...29
Trends in Student Performance on State Reading Assessment ...32
Diagnostic Data ...34
Grade Level Status ...35
Areas of Change ...38 Context ...38 Culture...40 Conditions ...41 Competencies ...43 Interpretation ...44 Judgments ...46 Recommendations ...48 Conclusion ...49 CHAPTER FIVE ...51 To-Be Framework ...51
Envisioning the Success To-Be ...51
Context ...52 Culture...53 Conditions ...55 Competencies ...57 Conclusion ...58 CHAPTER SIX ...59
Strategies and Actions ...59
Urgency ...59
Guiding Coalition...61
Build Organizational Capacity ...65
Sustain Acceleration...68
Institute the Change ...69
Conclusion ...71
CHAPTER SEVEN ...72
Implications and Policy Recommendations ...72
Policy Statement ...72 Analysis of Needs ...75 Educational Analysis ...75 Economic Analysis ...77 Social Analysis...78 Political Analysis ...79 Legal Analysis ...80
Moral and Ethical Analysis ...81
Implications for Staff and Community Relationships ...81
Conclusion ...83 CHAPTER EIGHT ...84 Conclusion ...84 Discussion ...84 Leadership Lessons ...87 Conclusion ...89
References ...90
Appendices ...95
Appendix A: Interview Questions: Authors of the Slam Dunk My Reading Program ...96
Appendix B: As Is 4 C’s Analysis ...97
Appendix C: Interview Transcript ...98
Appendix D: To Be 4 C’s Analysis ...102
TABLES
Table 1. t-Test Paired Two Sample for Means: Achievement Level ...31 Table 2. t-Test Paired Two Sample for Means Scale Score ...32 Table 3. Average Performance in Reporting Categories for State Reading Assessment ..33 Table 4. t-Test Paired Two Sample for Means: Craft and Structure ...34 Table 5. Growth of Students in the Domains Tested on the iReady Diagnostic Test and
FIGURE
Figure 1. Gains made in reading by number of levels gained, number of students per levels gained, and level movement ...30
CHAPTER ONE Introduction
To better support retained third grade students, leaders of a mid-size school district in the United States decided to implement a reading intervention in the months prior to the state reading test. The schools chosen to pilot the program were among the lowest 300 elementary schools within the state based on reading achievement scores. School leaders selected retained third grade students for participation in the new reading intervention program. Historically, retained third graders were a low-performing group, scoring nonproficient on the state reading assessment without making any significant growth to achieve a learning gain.
Purpose of the Program Evaluation
The purpose of this program evaluation was to examine the elements of a reading intervention program and how it was related to students making learning gains on the state reading assessment. By examining the key elements of a close reading intervention program, Slam Dunk My Reading, and the success rate of participating students, I
determined the overall effectiveness of the program supported by data. The identification of key elements led to deeper questions to pinpoint the reasons that make such a program successful in supporting student learning gains. According to Shanahan and Shanahan (2008), “most students need explicit teaching of sophisticated genres, specialized language conventions, disciplinary norms of precision and accuracy, and higher-level interpretive processes” (p. 43). Once students received explicit reading instruction using both grade level text and their own instructional level text, then it was the goal of the school leaders that each student made learning gains and achieved a passing scale score
on state testing. I used a combination of three types of evaluation including an attribution focus, context focus, and effectiveness focus (Patton, 2008) to examine the impact of the
Slam Dunk My Reading close reading program on student learning. An attribution focus
allowed me to determine the relationship between the Slam Dunk My Reading close reading program as an intervention with the resulting outcome of student performance on the state reading assessment (Patton, 2008). I examined key evidence including a history of each participant’s foundational skill levels, domain performance on diagnostic testing, and achievement level on the state reading tests. Each set of data included information from before students received the intervention and after students participated in the close reading program.
The context focus provided a detailed look into the environment within the program and how political, social, and economic contexts impacted the effectiveness of the program (Patton, 2008). The Slam Dunk My Reading close reading program was implemented at seven of the lowest performing schools in a school district in the
Southeastern United States based on proficiency levels from the state reading assessment in Spring 2016. In addition to serving struggling readers, each of these schools had a high percentage of students who qualified for free or reduced lunch and a highly transient student population. Historically, these schools served a high-needs population for many years. The physical location of the school under study in this evaluation was an idyllic, suburban area, surrounded by horse farms and country estates. However, most students in attendance did not live in the immediate vicinity and face long bus rides to and from school, ultimately passing five other schools on their daily trek due to outdated school assignments. Attendance zones had not been recalculated in recent years and the
attendance area of the school was heavily impacted by the recent economic downturn. Transient families trying to find affordable housing were attracted to the low-cost housing and mobile parks in the area. The families faced a high unemployment rate in this district and the cycle of poverty had continued for generations.
Finally, the use of an effectiveness focus allowed me to look at the extent to which the program was effective in attaining its goals and how the program could be replicated at similar sites (Patton, 2008). The purpose of Slam Dunk My Reading close reading program was to increase the reading comprehension skills of retained third grade students. The effectiveness of the program was determined by thoroughly examining student performance on subsequent tests.
Student performance on the state reading assessment was reported by one of five achievement levels. The lowest performing level, a level 1, had three sublevels of low, middle, and high. A level 2 had two sublevels, low and high. Students showed
satisfactory performance by scoring a level 3, level 4, or level 5 on the state reading assessment. Student performance across reporting categories measured how students could answer questions in three separate areas: Key Idea and Details, Craft and Structure, and Integration of Knowledge and Ideas. For the Key Idea and Details reporting category, students were expected to read closely to understand information which evolved into interpreting information to show mastery of Craft and Structure skills. The demand of the tasks grew as students then had to show their ability to integrate and evaluate content in the Integration of Knowledge and Ideas reporting category. I considered the strengths of the program to determine the possibility of replication at other sites in the district.
Rationale
Slam Dunk My Reading close reading program was designed as a reading
intervention program to teach close reading techniques, mainly using a strategy by the acronym SPADE that required readers to survey the text, predict what the text will be about, annotate the text as they read, dissect any questions asked, and provide evidence for answers to asked questions. Additionally, the program incorporated text marking codes to increase comprehension of text as students analyzed the text. Applying the SPADE framework to reading passages and related tasks assisted students with the comprehension of the text and provided evidence to answer the higher order questions required on the state reading assessment.
The first group of third grade retained students to participate in the Slam Dunk My
Reading program at an elementary school in the Southeastern United States was a group
with whom I worked over three years. I served as a reading coach during the 2014-2015 school year and managed the intervention programs that they received; most of these students received intensive phonics instruction in small groups for two 45-minute blocks of time a day. During School Year 2015-16, the cohort of third grade students remained in intensive phonics programs, showing progress but were not able to show mastery required for graduation from the program. At the end of the 2015-2016 school year, 21 students out of 106 (19.8%) were retained in third grade at my school due to a score of Level 1 out of 5 on the reading portion of the state reading assessment; historically, 20% of third grade students were retained each year at this elementary school.
During School Year 2016-17, I transitioned to a coordinator position at the same school and it was then my responsibility to strategically assign the retained students into
classrooms and intervention blocks. While the students continued in their intervention programs, it became clear they were lacking exposure to the grade level material on which they would be tested during state assessments. The rigorous task demands on the state assessments required students to read and comprehend text passages written at the current grade level of student placement. In addition to the text passages, assessment items were written at varied levels of questioning and required higher order thinking processes. Without exposure to these text passages and assessment items, students were not prepared for the state reading assessment. In the winter of 2017, district curriculum leaders designed a program to specifically support retained third grade students and directed school leaders to implement Slam Dunk My Reading, a close reading program, at district schools ranked among the lowest 300 of the state’s schools based on reading achievement. Our administrative team made the decision of which students to place into two leveled groups; we also selected two staff members to teach each group and acquired the necessary materials to implement the program with fidelity. Independent level text passages along with grade level text passages were prepared for students to apply the SPADE strategy over several days with rereads and annotation of the text. Each student was supplied with support documents, including a bookmark outlining the SPADE strategy and color pencils for text marking. Additionally, state assessment sample questions were used in both teacher-led instruction and independent formative assessments.
The success of Slam Dunk My Reading was important to many stakeholders as 86% (18 out of 21) of retained third graders at the elementary school under study were able to maintain their achievement level or made learning gains on the state reading
assessment after participating in the program. It was important that each retained student understood their current level and set a goal to work towards closure of the achievement gap between themselves and their grade level peers during elementary school so that they did not continue to fall further behind in subsequent years. The low-performing schools ranked among the lowest 300 elementary schools in the state according to reading
assessment performance needed to increase their state assigned school grade. Within the school grade calculation, earned learning gains and increased achievement levels in the reading scores of retained students caused the school to gain percentage points in the school grade categories of reading achievement, reading learning gains, and reading learning gains of students within the bottom quartile.
School grade calculations were based on three student performance ratings on both the reading and mathematics state assessments; overall achievement level, learning gains when achievement levels were increased from the previous year’s achievement level, and learning gains when achievement levels were increased from the previous year’s achievement level for the lowest performing students. Additionally, the overall achievement levels of fifth grade students on the state science assessment were also part of the school grade calculation. An improved school grade could result from students reaching a satisfactory status in reading as the now stronger readers would be able to comprehend the grade level material required to tackle both the math and science state assessments.
Reading counted for three of seven categories in the school grade calculation completed by leaders in the state Department of Education. As students made learning gains in their growth towards proficiency, points towards the school grade were be earned
in “reading learning gains” and “reading learning gains of the bottom quartile” in addition to those “reading proficiency” points earned by students scoring a level 3 or higher on the state reading assessment.
Overall achievement levels on state assessments ranged from the lowest level 1, indicating subpar performance, up to a level 5, or mastery of the state standards for that subject area. Due to the low scale scores that designated students at level 1 on their state reading assessment, most retained students were part of a bottom quartile of lowest performers, and therefore, impacted the school grade in three categories. The lowest performing students were at a numerical advantage as they could make a learning gain by moving into the subsets of level 1 or level 2 which divided the scale scores into smaller intervals. A student could gain five points and be able to move from a low level 1 to a medium level 1 to count as a learning gain, whereas a student at level 3 may have needed to earn forty points to make a learning gain by moving to level 4.
As Anyon (1980) claimed in the article “Social Class and the Hidden Curriculum of Work,” successful communities are grounded in successful schools, and therefore, all community stakeholders must support the preparation of all students to reach proficiency in reading. The school grading system in the state under study was applied to every public school and provided evidence of overall performance. Such grades were built on student achievement and learning gains and served as a determining factor for a
successful school rating. Goals
The intended goals of the program evaluation were to determine if educators using the Slam Dunk My Reading close reading program provided students the skills
necessary to become stronger readers that could, in turn, make learning gains on the state reading test. The program provided intensive, standards-based, whole group instruction along with small group instruction with a focus on close reading strategies on both grade level reading passages and independent reading level text passages. Slam Dunk My
Reading was designed and implemented in schools to prepare students to make learning
gains when taking the state reading test for a second time as a retained third grade student. If students made learning gains on the state reading test after participating in the program, the increase in scale score points would show the effectiveness of Slam Dunk
My Reading in developing reading comprehension skills. Students would also build their
testing endurance, increase their fluency, and comprehend complex vocabulary.
Second Elementary School (pseudonym) and its comparable six low-performing schools in the school district under study were facing the consequences of their failing school grades. A vote by the school board of the district under study granted an external operator to take over the daily operation of Second Elementary School for a second year since the school received a fourth consecutive school grade of D, on a scale of A-F, in Spring 2019 from the state Department of Education. This move came after six years of underperformance in both student achievement and student learning gains. During the six-year period of underperformance, the school grade fluctuated between a D and an F rating. A relinquishment of control over a local school had never been experienced in the district under study. The outcry from the public, including groups such as the National Association for the Advancement of Colored People (NAACP) and local politicians, demanded an answer as to what could be done to assist the school in raising and maintaining student achievement at Second Elementary School.
Definition of Terms
I identified several terms specific to my study that I used throughout this dissertation.
• Reading intervention is a program, supplementary to an existing literacy curriculum, that is provided to students for the primary purpose of increasing reading levels.
• Retained students are those students who are kept back to repeat their current grade level the next school year.
• Slam Dunk My Reading is a reading intervention program focused on increasing student reading comprehension.
• Student learning gains are calculated by computing how many points a student’s score increases from year to year on a state assessment.
• Independent reading level is the level at which students can read 99% of the words correctly in a given text with 90% comprehension.
• Instructional reading level is defined as 85% correct word recognition with at least 75% comprehension; a student instructional level is his or her reading level.
• Proficiency refers to an academic rating once students have demonstrated they have learned the knowledge and skills they are expected to learn as they progress through their education. Foundational literacy skills are the skills needed for reading and writing; they include things such as
awareness of the sounds of language, awareness of print, and the relationship between letters and sounds.
• Domain performance are the broad reporting categories for the state reading assessment; this includes Key Ideas and Details, Craft and
Structure, Integration of Knowledge and Ideas, and Language and Editing with Text-Based Writing added for grades 4 and up.
• Diagnostic testing is an assessment to determine a student’s specific areas of weakness and strength.
• Reading comprehension is the ability to process text, understand its meaning, and to integrate what the reader already knows.
• Phonics instruction is a way of teaching reading that stresses the
acquisition of letter-sound correspondences and their use in reading and spelling.
• State reading assessment is a standards-based test to measure education gains and progress in the area of reading.
• Bottom quartile students are the students that scored the lowest in each grade level and subject. School grades are a measure of student
proficiency and learning gains on the state assessment for the specific grades in those schools.
• Standards-based instruction is a system of instruction and assessment based on students demonstrating mastery of the state academic standards. • Close reading strategies apply a thoughtful, critical analysis of a test that
focuses on key details to develop an understanding of the craft and structure of the text to allow the reader to integrate ideas from the text with their own understandings and experiences.
• Test endurance is the ability to maintain focus and put forth extended effort during an assessment.
• Reading fluency is the ability to read with speed, accuracy, and proper expression.
Research Questions
The exploratory questions that guided my research were:
1. What key elements of Slam Dunk My Reading provide students with the skills to make gains?
2. What factors do the students have in common who passed the state assessment test after being in the program?
3. What parts of the program can educational leaders replicate at other schools? 4. Will this program be effective with struggling readers at other grade levels?
To address Question 1, Wagner et al. (2006) reminded us that competencies are the various technical, social, and leadership skills that bring about change. When considering the key elements of a reading intervention program, skills acquired by students lead to successful reading comprehension. Additionally, according to Wagner (2006), context encompasses those influences in politics, culture, and economics that impact our work. Determining the common factors of successful students completing the intervention program showed influences of home and school life to address Question 2.
Conditions, as discussed by Wagner et al. (2006), are those internal, tangible factors that shape the overall environment that allowed me to answer Question 3. The success of a replicated intervention program will be dependent on the culture of the organization where it is started. Lastly, Wagner et al. (2006) described culture as the
reality of how things are done in an organization which addresses Question 4 and the nature of struggling readers. Slam Dunk My Reading, a close reading program, was implemented at the lowest performing schools in a school district. Each of these schools was also designated as a Title I school and received funds due to the high percentage of economically disadvantaged students enrolled at each school. The culture of the schools implementing Slam Dunk My Reading included support for struggling readers in a Title I setting.
Conclusion
To best support retained third grade students, an effective reading intervention program needed to be implemented. Struggling readers needed a strong close reading framework to use while reading for text comprehension. Slam Dunk My Reading was designed to provide close reading strategies to assist readers in strengthening their ability to comprehend grade level text.
CHAPTER TWO Review of the Literature
Retention of students in elementary school is a well-researched topic; especially when considering the detrimental effects of being held back to the future academic careers of students. Specifically, reading proficiency by third grade is a predictor of future learning abilities, and most importantly, comprehension skills. The state under study required students to show basic proficiency on the state’s third grade reading test or a similar measure such as successful completion of a portfolio. Authors of studies I have described also examined various ways to lessen the achievement gap for retained
students. While types of reading intervention strategies made up a majority of the research, it is also notable that the learning environment and the characteristics of the learners can influence the possibility of retention. I collected all articles from the EBSCOhost database and only included full text, scholarly peer-reviewed material. I analyzed both qualitative and quantitative studies to allow for varied types of data that included test performance, classroom observations, and student demographics.
Characteristics of Retained Students
Students who have been retained in elementary school share similar
characteristics. Authors Moser, West, and Hughes (2012) used a longitudinal growth curve analysis to show that retained students received an initial one-year boost in achievement which then fully dissipated by the end of elementary school. Also, to be considered were those students identified with a diagnosed exceptionality as these students faced a higher probability of retention in their elementary years.
Vaughn et al. (2015) found that high school students with reading disabilities who were randomly assigned to treatment groups improved their reading skills necessary for graduation from high school after participating in a reading intervention program. Additionally, a reduction in the achievement gap of minority students with the sample of students within the treatment group also improved their reading skills needed to pass the state assessment to meet graduation requirements.
Early Intervention and Retention
Several studies supported a need for early intervention when students were showing a lack of foundational skills required for reading, mainly a deficiency in decoding abilities. Dombek and Connor (2012) found that students were significantly less likely to be retained in first grade if their reading instruction included intervention while their struggling peers not receiving the intervention component were retained. Also of importance, according to O’Connor, Briggs, and Forbes (2013), was the documentation of a student’s response to intervention and the effectiveness of the
intervention, including the need for differentiation to best fit the needs of each individual student.
Authors Reutzel, Petscher, and Spichtig (2012) examined the effectiveness of an online-program that provided guided, silent-reading practice for individual students. After a year of participation, 80% of students receiving the online intervention
component were able to obtain proficiency on the state reading assessment at the end of the school year. Early intervention played a critical role in the development of a primary student’s reading ability. According to Scheffel, Lefly, and Houser (2016), the
such as Dynamic Indicators of Basic Early Literacy (DIBELS) provides teachers with descriptive, qualitative data to identify young students in need of reading intervention. The ability of DIBELS to predict student reading performance was noted for both native English-speaking students and their English Language Learner peers.
Environmental Factors to Support Retained Students
Gottfried (2012) examined the connection between classroom peer achievement and low-performing, high-poverty minority students. He found that retained students placed in classrooms with higher average peer ability tend to continue to earn lower testing outcomes in their post retention years compared to their continuously promoted peers. In a second publication, Gottfried (2013) continued his research into the post-retention academic performance of retained students. Through data gathered from five cohorts Gottfried found a statistically negative decline in the reading performance of non-retained students when placed in classrooms with non-retained students in elementary
classrooms. If teachers responded to the educational needs of retained students by allocating regular class time, then non-retained students may potentially be adversely affected as classroom instruction is slowed for everyone. Large numbers of retained peers in the classroom would suggest that even greater portions of instruction would be
dedicated to remediation and slow the educational advancement for other classmates. The scheduling of retained students must take the potential negative impact into
consideration.
Huddleston (2015) found that socio-economic status, described in terms of class, played into human action of the social field surrounding education, and therefore, influenced retentions based on low performance on state reading assessments. With a
history of unsuccessful experiences in school, these students lacked institutional cultural capital or credentials, and therefore, symbolic capital or prestige that would label them as being an asset to their school’s scores. Rather, they were a liability, students who would require extensive help and still might not pass the state assessments. Competition existed for what counted as learning and what determineed promotion. Although this study did offer some hope, it also served as a reminder of the continued work that must be done to help make schools instruments of social change. A greater effort should be made to educate the public who largely still believe that retaining students prevents them from dropping out of school.
Researchers Rodriguez and Tarango (2016) mapped the educational equity, as measured by access to early childhood programs and access to highly qualified teachers, across a geographic region to find that the more rural the setting, the greater
concentration of English Language Learners and economically disadvantaged students. They found that access to early childhood education and access to qualified teachers declined in the rural settings. Inversely, the highest educational equity was evident in the schools associated with areas of higher property values. Equity was not merely a leveling of the playing field of resources for all groups of students but was rather best applied from a social justice perspective as a reflection of unique needs, and approaches for diverse populations to achieve an equality of outcomes. This equality led to equal opportunities towards college access and success, career readiness, and overall citizenship.
Fidelity of intervention programs must be held to established protocols, supported by the study done by Jesson and Limbrick (2014). The sustainability of interventions
must be considered at both a district and school level to support all tiers of students. In addition to the fidelity of implementation of interventions, teacher attitudes and beliefs about retention were examined by Range, Pijanowski, Holt, and Young (2012) to see how primary grade teachers and elementary principals differed in their views about reasons for grade retention and the most appropriate time to retain students. Teachers agreed
significantly more than principals that retention helped prevent future failure, assisted with maintaining high standards, provided additional support, and motivated attendance for struggling learners. Both principals and teachers agreed that academic performance, followed by ability level, is the most important factor to consider when deciding to retain students.
Comprehension Strategies
Muller, Richter, Krizan, Hecht, and Ennemoser (2016) examined the influence of peer-tutoring using a specific set of questioning techniques. Their research indicated that specific cognitive strategies such as summarizing, generating questions, and activating prior knowledge can enhance reading comprehension skills in primary school. In later studies, the focus switched to how to combine multiple strategies to increase
comprehension of text. Heterogeneous dyads were set up and pre-test reading comprehension scores were compared to post-test reading comprehension after
participation in the partnership between tutor and tutee. The researchers found a positive and significant partner effect on the tutee’s post-test reading comprehension abilities.
Tabrizi and Vafakhah (2014) researched the relationship between the opportunity to preview the stem and answer choices of an assessment item prior to reading passages and the performance of the students utilizing the preview strategy to increase
comprehension. They concluded that a significant relationship did exist between the proficiency level of student performance and the use of an item preview cognitive strategy for comprehension. Hagaman, Casey, and Reid (2016) investigated the effects of a paraphrasing strategy on students who identified as fluent readers yet lacked comprehension skills. After taking a baseline assessment, students were taught a
paraphrasing strategy to assist with recall of text evidence and responding to short-answer questions. All participants showed measured improvement in identification of main idea and key details.
Worrell, Duffy, Brady, Dukes, and Gonzalez (2016) investigated the issues caused by computer-based testing when students had been taught using paper-based strategies. For this study, students were taught a mnemonic device to pull details from a text to increase comprehension of the text. The researchers found that the explicit teaching of a generalizable strategy that transferred from paper-based teaching to
computer-based application allowed students to increase their comprehension on digitally presented text. The state under study in my evaluation switched from computer-based testing to paper-based testing to avoid the issues such as technology outages that transpired during the first year of computer-based testing. However, students were still tested through computer-based programs in the district under study to gather local assessment data.
Sonmez and Sulak (2018) examined the effect of a think-aloud strategy on reading comprehension and discovered that the use of this strategy was statistically significant between pre-test and post-test scores on a reading comprehension test. While investigating the effect of a note-taking instruction on performance of students’ reading
comprehension, authors Chang and Ku (2015) found that the use of an explicitly taught note-taking strategy raised student performance scores on comprehension assessments. Goldstein et al. (2017) quantified the effect of explicitly taught vocabulary strategies on the learning of domain-specific words to assist students with comprehension. The authors found that participants were able to improve their understanding of vocabulary in the short term, but the strategies did not translate to lasting learning.
Legislation in the state under study required teachers to earn a reading endorsement on their teaching certificate to teach struggling readers, such as those in intensive reading intervention programs or those who had been retained. Teachers had to prove
competency in identifying characteristics of conditions such as dyslexia, along with a focus on multisensory interventions. Additionally, teachers had to become proficient with instructional strategies to support explicit, systematic, and sequential approaches to teaching phonemic awareness, phonics, vocabulary, fluency, and comprehension (citation withheld to protect anonymity).
Mahdavi and Tensfelt (2013) examined the effectiveness of teaching reading comprehension strategies to primary students. Overall findings pointed to the need for strategies to actively engage students with the text and to monitor their own
comprehension by moving from passive listeners to active participants. As students interacted with the text, they made connections that allowed them to answer the high demands of the state reading assessment.
Conclusion
In this section of my dissertation, I shared summaries of current literature
types of interventions on struggling readers in addition to the potential effect of the learning environment on students facing retention. I found trends in the scholarly research to support providing struggling students with a practiced reading strategy, so students actively engaged with the text. Also, of note was the potential for retained students to be placed in a reading intervention that was actively monitored by school administrators for fidelity of implementation of the program and progress monitoring of the students’ performance. Additionally, the socioeconomic status of a student dictated ability to compete for learning which then equaled access to promotion. Students from lower socioeconomic households did not have the same access to educational
opportunities as their more well-off peers which led to a higher potential for the poor students to be retained in school. Lastly, the early identification of struggling learners was critical to provide such students with academic interventions.
CHAPTER THREE Methodology Research Design Overview
This program evaluation examined the elements of a reading intervention program that seemed to result in learning gains as measured by the state reading assessment. By examining the key elements of Slam Dunk My Reading and the success rate of
participating students, I determined the overall effectiveness of the program based on analysis of gathered data. The identification of key elements within the Slam Dunk My
Reading close reading program led to deeper questions to pinpoint the reasons that made
this program successful in supporting student learning gains. I used a combination of three types of evaluation including an attribution focus, context focus, and effectiveness focus (Patton, 2008).
An attribution focus allowed me to determine the relationship between the Slam
Dunk My Reading close reading program as a treatment with the resulting outcome of
student performance on the state reading assessment (Patton, 2008). Examined evidence included a history of each participant’s foundational skill levels, domain performance on diagnostic testing, and proficiency ratings on the state reading assessments. Each set of data included information from before students received the treatment and after students participated in the program.
The context focus provided a detailed look into the environment within the program and how political, social, and economic contexts impacted the effectiveness of the program (Patton, 2008). The Slam Dunk My Reading close reading program was placed at seven of the lowest performing schools in Marshall County (pseudonym)
according to a ranking by the state based on the proficiency levels from the state reading assessment. In addition to serving struggling readers, each of these schools had a high percentage of students qualifying for free or reduced-price lunch, and historically they served a high-needs population. The physical location of the school under study in this evaluation was in a suburban area surrounded by rural farming communities and expensive equestrian estates. However, most students in attendance did not live in the immediate vicinity and faced long bus rides to and from school due to out of date boundary lines. Years of economic struggle and high unemployment rates in Marshall County and the state under study caused a high mobility rate for most of the families attending Second Elementary School.
Finally, using an effectiveness focus, I looked at the extent to which the program was effective in attaining its goals and how the program could be replicated at similar sites (Patton, 2008). The purpose of Slam Dunk My Reading close reading program was to increase the reading comprehension skills of retained third grade students. Through the examination of the performance on subsequent testing by looking at performance levels and points earned in each reporting category, I was able to determine the effectiveness of the program. I considered the strengths of the program to see the possibility of replication at other sites in the district.
Participants
I analyzed the extant data of a cohort of 18 third grade students retained during the 2016-2017 school year. The students in this study were part of a larger group of 21 retained students at the school under study. In addition, I interviewed an author of the reading intervention program.
Data Gathering Techniques
I examined both quantitative and qualitative data. I gathered quantifiable data from two major student information systems within the school district under study. Demographic information was stored in Skyward, the Marshall Public School student information system, while student performance data was stored in Performance Matter’s Unify program. I obtained permission from the district contact in the School Counseling and Assessment Department to use student data in my program evaluation. Within these two software programs, I was able to see a variety of test scores, student demographics, and overall student performance on district progress monitoring diagnostics.
Qualitatively, I interviewed an individual closely involved with the design and implementation of the Slam Dunk My Reading close reading program.
Extant data. Using the school district student information system, I reviewed several pieces of historic data including Spring 2016 and Spring 2017 scale scores, proficiency levels, and points earned versus points possible in each of the reading reporting categories from the state reading assessment. Reporting categories included Main Idea and Details, Craft and Structure, and Integration of Knowledge and Ideas. In addition, I studied diagnostic scale scores and level placement from iReady to understand student performance from 2016-2017 in the areas of phonological awareness, phonics, high-frequency words, vocabulary, and comprehension. Used as a diagnostic and
progress monitoring program, iReady is a computer-based program that tested students in reading and math to determine their academic levels. The diagnostic test was given three times a year in fall, winter, and spring while progress monitoring took place monthly. The data collected from iReady was used to determine student academic progress with
the goal of obtaining grade level mastery within both reading and math. I analyzed extant data for patterns and trends regarding subsequent performance on the state reading assessment and iReady diagnostics after students participated in the reading intervention
Interviews. I conducted an interview of one if the authors of the Slam Dunk My
Reading close reading program. I focused questions for the authors of the program on the
research done to support the implementation of the program in addition to the decision-making process for the elements included for Slam Dunk My Reading close reading program. I examined any feedback they received from schools participating in the program.
Data Analysis Techniques
I examined an interview transcript to find the essential elements in the design and implementation of the reading intervention. I analyzed extant data for patterns and trends regarding subsequent performance on the state reading assessment and iReady
diagnostics after students participated in the reading intervention. I determined growth in achievement levels in addition to performance in each reporting category of the state reading assessment.
Using a paired t-test, I determined if data were statistically significant comparing pre- and posparticipation in the reading program. The independent variable for my t-test was participation in a reading intervention program. The dependent variable was student performance in the categories of achievement level, scale score, and reporting categories. I applied controls to avoid variances by creating equivalent groups that received the same instruction, and took the same test administered under the same conditions to determine mastery of the same state reading standards.
Ethical Considerations
Slam Dunk My Reading close reading program was only implemented at low
performing, low socioeconomic schools within Marshall County (pseudonym). In addition to the schools’ placement on the list of the state’s lowest performing schools according to the state reading assessment, there was also a high number of retained students at each school and transient student population. As part of the program evaluation, I examined not only the reasons why such high poverty communities had a higher number of retained students that needed a program such as Slam Dunk My
Reading close reading program, but also what characteristics these students shared in
historic achievement data. I protected the identification of all involved students and I redacted any identifying information to insure student anonymity.
Limitations
A limitation for this study lay in the small sample size of 18 students. Also notable is the time students spent in the reading intervention; it was a six-week program during which students attended two sessions daily each week. There were also several key variables that influenced student learning gains and could not be controlled during my evaluation, such as student background knowledge and the testing environment on the day of testing. The format of the test was changed from computer based to paper based which may have affected student performance. As far as the abilities of the individual teachers, as a researcher, I was unable to account for their pedagogical experience and the level of fidelity with which they implemented the program as I was not present for direct observation of lessons.
Conclusion
In Section Three, I discussed the methodology I used to analyze extant data along with my interview with an author of the Slam Dunk My Reading close reading program. Through thorough review of state reading assessment data, pre- and post-participation in the close reading program, along with iReady performance, I determined patterns of student growth. I shared my plan to find causal relationships between the Slam Dunk My
Reading close reading program and the learning gains made by retained third grade
CHAPTER FOUR Results
Retention of students in elementary school can be a contentious topic; especially when considering the detrimental effects of being held back on the future academic careers of students as noted by Gottfried (2012). Mahdavi and Tensfelt (2013) supported reading proficiency by third grade as a predictor of future learning abilities, and most importantly, comprehension skills. The state under study required students to show basic proficiency on the state’s third grade reading test or a similar measure such as successful completion of portfolio, and thus showing mastery of third grade language arts standards. While types of reading intervention strategies make up much the research on this topic, it is also notable that the learning environment and the characteristics of the learners can influence the possibility of retention. I analyzed both qualitative and quantitative data to allow for varied types of data that included student performance from preparticipation and post-participation in the Slam Dunk My Reading close reading program in addition to an interview with the author of the program.
This chapter relays my findings of a mixed method approach through which I examined the Slam Dunk My Reading close reading intervention program both qualitatively and quantitatively by answering the following research questions:
1. What key elements of Slam Dunk My Reading close reading program provide students with the skills to make gains?
2. What factors do the students who pass the state reading assessment have in common after being in the program?
4. Will this program be effective with struggling readers at other grade levels? In this section I also discussed my analysis of extant data to determine the impact of the reading intervention program, including students’ foundational skill levels from before and after participation in the program, state reading assessment proficiency levels and scale scores from before and after participation in the program, and domain
performance on the state reading assessment based on reporting categories from before and after participation in the program. The process I used to examine extant data is described in this section.
Findings
Low performing elementary school educators implemented Slam Dunk My
Reading close reading program to target retained third grade students with the immediate
purpose of increasing their reading proficiency on the state reading assessment. Due to the timing of the program launch in the winter, a basketball theme built around March Madness helped to motivate students. Analysis of student reading scores showed that 17 of 21 students within the studied cohort increased their scale score with one student maintaining scale score points to achieve a high level 1 score. Levels on the state reading assessment were reported as achievement levels, with level 1 scores assigned as low, middle, or high, level 2 scores assigned as low or high. A level 3 achievement level was considered satisfactory with level 4 showing proficiency and level 5 reflecting mastery of grade level material.
T-test findings showed a statistically significant difference between the means of
my two trials (before participation/after participation in the Slam Dunk My Reading close reading program) in both scale scores and achievement levels on the state reading
assessment. Additionally, I found a statistically significant difference in the performance of students in the reporting category of Craft and Structure on the state reading
assessment questions.
Student Growth Demonstrated on State Assessment Test
When determining student growth, the cohort averaged a 1.5 achievement level average gain in the number of levels they achieved on the state reading assessment. While four students did not make a gain in levels, two students were able to maintain medium level 1 score while two students maintained their high level 1 scores. Six students made a complete level gain in achievement level, resulting in movement to a higher level; three students grew from a medium sublevel 1 score to a high sublevel 1 score, two students grew from a low sublevel 1 score to a medium sublevel 1 score, and one student grew from a high sublevel 1 achievement score to a low sublevel 2.
I observed the greatest growth in achievement levels in seven out of the 18 students. One student made a four-level gain in achievement level by growing from a low level 1 to a high level 2. Three students showed a three-level gain in achievement level with two growing from a low level 1 to a low level 2 and one student growing from a high level 1 to a satisfactory score of a level 3. Four students grew by two achievement levels with three students growing from a low level 1 to a high level 1 and one student going from a medium level 1 to a low level 2. The figure below illustrates the gains made in achievement level by students.
Number of Levels Gained
Number of
Students Level Movement
0 4 Two students remained 1M
Two students remained 1H
1 6 Three students moved from 1M to 1H Two students moved to 1L to 1M One student moved from 1H to 2L 2 4 Three students moved from 1L to 1H
One student moved from 1M to 2L 3 3 Two students moved from 1L to 2L
One student moved from 1H to 3 4 1 One student moved from 1L to 2H
Figure 1. Gains made in reading by number of levels gained, number of students per
levels gained, and level movement; L, M, and H designate low sublevel, medium sublevel, and high sublevel within the overall levels of 1-3.
T-test results for Student Achievement Level found a statistically significant
difference in the average mean between the two groups; preparticipation in Slam Dunk
My Reading close reading program and post participation in Slam Dunk My Reading
close reading program. Achievement levels ranged from a level 1 (inadequate), the lowest score that a student can earn, followed by a level 2 (below satisfactory); these
achievement levels mean that a student has not mastered grade level standards. Achievement levels 3 (satisfactory), 4 (proficiency), and 5 (mastery) are considered passing scores, and students scoring level 3, 4, or 5 are considered on grade level. The p-value was less than 0.05, and therefore, rejecting the null hypothesis, there was a positive impact of the Slam Dunk My Reading close reading program on student achievement levels on the state reading assessment.
Table 1.
t-Test Paired Two Sample for Means: Achievement Level Variable 1 and Variable 2
t-Test: Paired Two Sample for Means
ACHIEVEMENT LEVEL Variable 1 Variable 2
Mean 1.293333 1.783333
Variance 0.075447 0.183847
Observations 18 18
Pearson Correlation 0.175810
Hypothesized Mean Difference 0
df 17 t Stat -4.453700 P(T<=t) one-tail 0.000174 t Critical one-tail 1.739607 P(T<=t) two-tail 0.000349 t Critical two-tail 2.109816
T-test for Student scale scores found a statistically significant difference in the
average mean between the two groups; preparticipation in Slam Dunk My Reading close reading program and post participation in Slam Dunk My Reading close reading program. Student scale scores are the individual score assigned to each student. Each Achievement Level is assigned a score range on the state assessment scale. The p-value was less than 0.05, and therefore, rejecting the null hypothesis, there was a positive impact of the Slam
Dunk My Reading close reading program on student scale scores on the state reading
Table 2.
t-Test Paired Two Sample for Means Scale Score Variable 1 and Variable 2
t-Test: Paired Two Sample for Means
SCALE SCORE Variable 1 Variable 2
Mean 256.9444444000000 280.6111
Variance 174.5261438000000 139.8987
Observations 18 18
Pearson Correlation 0.221586072
Hypothesized Mean Difference 0
df 17 t Stat -6.4125970680000 P(T<=t) one-tail 0.0000032163708 t Critical one-tail 1.7396067260000 P(T<=t) two-tail 6.43274E-06 t Critical two-tail 2.109815578
Trends in Student Performance on State Reading Assessment
I examined student performance in each reporting category of the state reading assessment. The three reporting categories were Key Ideas and Details, Craft and
Structure, and Integration of Knowledge and Ideas. State education leaders shared results as a percentage of possible points earned so that values could be compared year to year since points in each reporting category could change based on the test blueprint for the assessment that testing season. In the Key Ideas and Details category, students from this cohort showed an overall negative percentage in growth while the average percentage of points earned in Craft and Structure grew by twelve percentage points, and the percentage of points earned in Integration of Knowledge and Ideas showed zero growth but did not decline. The figure below illustrates the average performance in reporting categories on the state reading assessment from Spring 2016 to Spring 2017 by the participants before and after participation in the Slam Dunk My Reading close reading program.
Table 3.
Average Performance in Reporting Categories for State Reading Assessment
Percentage of Possible Points Earned Key Ideas
and Details Craft and Structure Knowledge and Ideas Integration of
Spring 2016 28 23 24
Spring 2017 26 36 24
The t-test results for the reporting category of Craft and Structure demonstrated a statistically significant difference in the average mean between the two groups:
preparticipation in Slam Dunk My Reading close reading program and post participation in Slam Dunk My Reading close reading program. The Craft and Structure reporting category required students to determine the meaning of words and phrases in a text and analyze how the author used the meanings of key terms. A large part of Slam Dunk My Reading close reading program contained vocabulary strategies and use of context clues which allowed students to score higher in this reporting category. The p-value was less than 0.05, and therefore, rejecting the null hypothesis, there was a positive impact of the
Slam Dunk My Reading close reading program on student achievement levels on the state
reading assessment. However, a statistically significant difference was not found in the other two reporting categories of Key Ideas and Details and Integration of Knowledge and Ideas.
Table 4.
t-Test Paired Two Sample for Means: Craft and Structure Variable 1 and Variable 2
t-Test: Paired Two Sample for Means
CRAFT and STRUCTURE Variable 1 Variable 2
Mean 23.20244 35.76389 Variance 124.04570 279.43730 Observations 18 18 Pearson Correlation -0.02645 Hypothesized Mean Difference 0 df 17 t Stat -2.621360 P(T<=t) one-tail 0.008936 t Critical one-tail 1.739607 P(T<=t) two-tail 0.017872 t Critical two-tail 2.109816 Diagnostic Data
I gathered diagnostic data to examine student growth in foundational skill categories from before and after participation in the reading intervention. In the domain of High Frequency words from Assessment Period One in August of 2016 to Assessment Period Three in May of 2017, five of 18 students made growth in scale score points, showing improvement in their ability to identify high frequency words. The most commonly used words in printed text are called high frequency words; these words tend to be phonetically irregular, and therefore, must be memorized by students if reading is to be quick and fluent. Seventeen out of 18 students made growth in their understanding of vocabulary words through use of context clues which allowed them a deeper insight into the text. The students showed growth in comprehension skills with 14 out of 18 students making growth in the category of Informational Text and 16 out of 18 students growing in the category of Literature. Comprehension is an intentional and interactive process
done by a reader that occurs before, during, and after the reading of a text.
The growth of students towards obtaining grade level status was significant because the state reading assessment was based on grade level academic standards along with text on a third grade level. If students were to reach proficiency on the state reading assessment, they must have been able to read and comprehend grade level material. The figure below illustrates the growth of students in the domains tested on the iReady diagnostic test and the growth they made from Assessment Period One (prior to participation in the program) to Assessment Period Three (after participation in the program).
Table 5.
Growth of Students in the Domains Tested on the iReady Diagnostic Test and Growth by Assessment Period
Domain Placement AP1* Placement AP3
Growth from AP1 to AP3 by
students High Frequency
Words 14 showed mastery 17 showed mastery 5/18** Vocabulary 2 on grade level 4 on grade level 17/18 Comprehension,
Informational
Text 2 on grade level 6 on grade level 14/18
Comprehension,
Literature 0 on grade level 3 on grade level 16/18 *AP – Assessment Period
**number of students out of n=18 Grade Level Status
Achievement of grade level status was a key to success on the state reading assessment as students had to read and comprehend passages and questions that were written with the expectation of grade level mastery. I examined diagnostic data to see how many students achieved grade level placement in each of the foundational skills
domains. Most students had previously mastered High Frequency Words with 14 students already on grade level for Assessment Period One while four more students reached grade level on Assessment Period Three, resulting in 17 of 18 students testing out of this category having shown mastery. In the Vocabulary domain, two students were on grade level for Assessment Period One and an additional four students reached grade level during Assessment Period Three for a total of six students being on a third grade level at the end of the year. Students reaching grade level on the Comprehension domain grew from two students on grade level for Assessment Period One to six students on grade level by Assessment Period Three for Informational Text, and grew from zero students on grade level for Assessment Period One to three students on grade level for Assessment Period Three for Literature.
An Author’s Perspective
I conducted an interview with one author of the Slam Dunk My Reading close reading program to explore the reasoning for creation of the Slam Dunk My Reading close reading program. According to the author (citation withheld to protect anonymity of the school district under study), school and district leaders were concerned with the growing number of retained students in the third grade. In previous years, third grade students could be retained multiple times and, as such, became “three time third graders.” These struggling learners were placed into a specific classroom where class size was kept small, a full-time paraprofessional acted as an additional support dedicated solely to those classrooms, and student work throughout the day was devoted to reading strategies and phonics intervention throughout the whole school day (Citation withheld to protect anonymity of the school district).
District curriculum leaders decided to bring back a similar program in January of 2017 as pilot classrooms on a smaller scale to specifically target retained third grade students. District leaders proposed the new program to the principals of low-performing elementary schools. Slam Dunk My Reading close reading program was designed as a reading intervention program that would enable teachers to teach close reading strategies to students in a small group setting, using both independent level and grade level text passages. Due to the program launch coinciding with March Madness college basketball tournament, the program used a basketball theme to engage students and allow for themed celebrations.Comprehension strategies along with a close reading framework made up the main parts of the program. The basis of Slam Dunk My Reading close reading program was the SPADE strategy that required students to survey and make predictions about the text prior to reading, annotate the text following given coding symbols, dissect the questions and find evidence to support answers. The purpose of SPADE was to assist students in the comprehension of text on the state reading assessment.
According the author of the program, the strengths in the program were found in the staffing of teachers and classroom paraprofessionals chosen to implement the program. The ability to connect with and motivate students made a difference with the participating students. Additionally, maintaining a student focus that provided a
framework for students to use during the state reading assessment contributed to student success on the state reading assessment. Two weaknesses of the program were lack of funding and lack of monitoring for fidelity by school administrators (Citation withheld to protect anonymity of the district).
A rebranding of the Slam Dunk My Reading close reading program into a new release called Text Detectives, launched prior to testing in Spring of 2019. A major part of the new release was an intensive training of teachers and administrators in eight-hour sessions to teach the fundamentals of the program. In the 2019-2020 school year, district leaders planned to expand Text Detectives to all tested grade levels in all elementary schools across the district at the request of teachers and administrators in need of a close reading program to increase student comprehension skills in preparation for the state reading assessment.
Areas of Change
To lead effective and sustainable change in education, opportunities must be examined at all levels. Focusing on the four areas of change allows for multiple
perspectives to be considered. Analysis of the areas of context, culture, conditions, and competencies is crucial to begin the change process (See Appendix B).
Context. My research question for the context area of change was “What factors do the students have in common who passed the state reading assessment after being in the program?” The demographics of this cohort included a total of 18 students. Seven of the students were female and 11 of the students were male. Reported ethnicity categories included four Black females and seven Black males, two White females and one White male, one Hispanic female and three Hispanic males. Eleven of the 18 students were identified as a “student with a disability” and qualified to receive extra classroom support and/or accommodations on assessments. A common demographic shared by this cohort was being economically disadvantaged with 16 out of 18 students coming from a low-income home.
The context of this study was a cohort of 18 third grade students who were retained to repeat the third grade during the 2016-2017 school year. The students in the cohort attended one of the lowest ranked elementary schools within the state under study according to a three-year average of reading achievement on the state reading assessment. The purpose of implementing the Slam Dunk My Reading close reading intervention program was to lower the number of students retained in third grade because historically, 20 to 25 students were retained in third grade at lower performing schools which required the addition of a teaching unit for that grade level, pulling resources away from other areas to fill the need created by large numbers of retained third grader students. A secondary purpose to the program was to increase reading achievement on the state reading assessment to raise the school grade based on successful student performance. An increase in the school grade would result in state leaders removing the school from state accountability monitoring and move the school off the lowest 300 list of elementary schools in the state based on students’ performance on the state reading assessment.
The reality of the situation in the Marshall County school district was that the overall reading proficiency of students on the 2018 state reading assessment in grades 3, 4, and 5 was 45% compared to the state average proficiency of 54%. Comparatively, the school attended by this cohort of students held a 24% proficiency rate during this same time. Faced with a high rate of poverty and a transient student population, Second Elementary (pseudonym) also battled a lack of parental involvement and low
expectations for the students from both their families and their teachers. Additionally, years of failing school grades placed the school under scrutiny by the state accountability department along with a mandate to undergo management by an external operator. The
